From patchwork Mon Apr 19 18:40:36 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christopher Baines X-Patchwork-Id: 28684 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id 6FAC127BC7A; Mon, 19 Apr 2021 20:13:46 +0100 (BST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-2.9 required=5.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_MSPIKE_H4,RCVD_IN_MSPIKE_WL,SPF_HELO_PASS,UNPARSEABLE_RELAY autolearn=unavailable autolearn_force=no version=3.4.2 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTPS id E254627BC77 for ; Mon, 19 Apr 2021 20:13:45 +0100 (BST) Received: from localhost ([::1]:39202 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lYZL7-000123-1J for patchwork@mira.cbaines.net; Mon, 19 Apr 2021 15:13:45 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:40096) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lYYpS-00043x-Rl for guix-patches@gnu.org; Mon, 19 Apr 2021 14:41:02 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:40518) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lYYpS-0002Ky-K7 for guix-patches@gnu.org; Mon, 19 Apr 2021 14:41:02 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1lYYpS-0008Qq-I4 for guix-patches@gnu.org; Mon, 19 Apr 2021 14:41:02 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#47897] [PATCH] substitutes: Don't cache negative lookups or transient errors. Resent-From: Christopher Baines Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Mon, 19 Apr 2021 18:41:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: report 47897 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 47897@debbugs.gnu.org X-Debbugs-Original-To: guix-patches@gnu.org Received: via spool by submit@debbugs.gnu.org id=B.161885765632392 (code B ref -1); Mon, 19 Apr 2021 18:41:02 +0000 Received: (at submit) by debbugs.gnu.org; 19 Apr 2021 18:40:56 +0000 Received: from localhost ([127.0.0.1]:52064 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lYYpM-0008QO-K3 for submit@debbugs.gnu.org; Mon, 19 Apr 2021 14:40:56 -0400 Received: from lists.gnu.org ([209.51.188.17]:40328) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lYYpH-0008QB-48 for submit@debbugs.gnu.org; Mon, 19 Apr 2021 14:40:55 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:40064) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lYYpD-0003xE-E8 for guix-patches@gnu.org; Mon, 19 Apr 2021 14:40:50 -0400 Received: from mira.cbaines.net ([2a01:7e00:e000:2f8:fd4d:b5c7:13fb:3d27]:59795) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lYYp9-00026t-2y for guix-patches@gnu.org; Mon, 19 Apr 2021 14:40:47 -0400 Received: from localhost (unknown [IPv6:2a02:8010:68c1:0:8ac0:b4c7:f5c8:7caa]) by mira.cbaines.net (Postfix) with ESMTPSA id 92E0A27BC6C for ; Mon, 19 Apr 2021 19:40:37 +0100 (BST) Received: from localhost (localhost [local]) by localhost (OpenSMTPD) with ESMTPA id b370b44f for ; Mon, 19 Apr 2021 18:40:36 +0000 (UTC) From: Christopher Baines Date: Mon, 19 Apr 2021 19:40:36 +0100 Message-Id: <20210419184036.18632-1-mail@cbaines.net> X-Mailer: git-send-email 2.30.1 MIME-Version: 1.0 Received-SPF: pass client-ip=2a01:7e00:e000:2f8:fd4d:b5c7:13fb:3d27; envelope-from=mail@cbaines.net; helo=mira.cbaines.net X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001, UNPARSEABLE_RELAY=0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: "Guix-patches" X-getmail-retrieved-from-mailbox: Patches Caching these responses can delay users getting substitutes, since they'll have to wait at least as long as the cache TTL to fetch the substitute, even if it's available sooner. * guix/substitutes.scm (%narinfo-negative-ttl, %narinfo-transient-error-ttl): Remove variables. (cache-narinfo!): Adjust to not use the negative TTL. (fetch-narinfos): Don't cache non 200 status code responses. --- guix/substitutes.scm | 22 +++------------------- 1 file changed, 3 insertions(+), 19 deletions(-) diff --git a/guix/substitutes.scm b/guix/substitutes.scm index 08f8c24efd..55daa6c964 100644 --- a/guix/substitutes.scm +++ b/guix/substitutes.scm @@ -70,14 +70,6 @@ ;; 'Cache-Control' response header. (* 36 3600)) -(define %narinfo-negative-ttl - ;; Likewise, but for negative lookups---i.e., cached lookup failures (404). - (* 1 3600)) - -(define %narinfo-transient-error-ttl - ;; Likewise, but for transient errors such as 504 ("Gateway timeout"). - (* 10 60)) - (define %narinfo-cache-directory ;; A local cache of narinfos, to avoid going to the network. Most of the ;; time, 'guix substitute' is called by guix-daemon as root and stores its @@ -104,8 +96,7 @@ entry is stored in a sub-directory specific to CACHE-URL." (define (cache-narinfo! cache-url path narinfo ttl) "Cache locally NARNIFO for PATH, which originates from CACHE-URL, with the -given TTL (a number of seconds or #f). NARINFO may be #f, in which case it -indicates that PATH is unavailable at CACHE-URL." +given TTL (a number of seconds or #f)." (define now (current-time time-monotonic)) @@ -113,8 +104,7 @@ indicates that PATH is unavailable at CACHE-URL." `(narinfo (version 2) (cache-uri ,cache-uri) (date ,(time-second now)) - (ttl ,(or ttl - (if narinfo %narinfo-ttl %narinfo-negative-ttl))) + (ttl ,(or ttl %narinfo-ttl)) (value ,(and=> narinfo narinfo->string)))) (let ((file (narinfo-cache-file cache-url path))) @@ -214,16 +204,10 @@ if file doesn't exist, and the narinfo otherwise." (cache-narinfo! url (narinfo-path narinfo) narinfo ttl) (cons narinfo result)) result)) - (let* ((path (uri-path (request-uri request))) - (hash-part (basename - (string-drop-right path 8)))) ;drop ".narinfo" + (begin (if len (get-bytevector-n port len) (read-to-eof port)) - (cache-narinfo! url (hash-part->path hash-part) #f - (if (or (= 404 code) (= 202 code)) - ttl - %narinfo-transient-error-ttl)) result)))) (define (do-fetch uri)