From patchwork Mon Mar 2 21:07:33 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christopher Baines X-Patchwork-Id: 20495 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id 33AC427BBEA; Mon, 2 Mar 2020 21:08:13 +0000 (GMT) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-2.9 required=5.0 tests=BAYES_00,MAILING_LIST_MULTI, UNPARSEABLE_RELAY autolearn=unavailable autolearn_force=no version=3.4.2 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTP id E9B7E27BBE4 for ; Mon, 2 Mar 2020 21:08:12 +0000 (GMT) Received: from localhost ([::1]:38678 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1j8sIO-0006Ow-Fe for patchwork@mira.cbaines.net; Mon, 02 Mar 2020 16:08:12 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:43438) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1j8sIF-0006Oh-KU for guix-patches@gnu.org; Mon, 02 Mar 2020 16:08:04 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1j8sIE-0003Fm-Nh for guix-patches@gnu.org; Mon, 02 Mar 2020 16:08:03 -0500 Received: from debbugs.gnu.org ([209.51.188.43]:60682) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1j8sIE-0003FW-Iv for guix-patches@gnu.org; Mon, 02 Mar 2020 16:08:02 -0500 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1j8sIE-0001uv-Ei for guix-patches@gnu.org; Mon, 02 Mar 2020 16:08:02 -0500 X-Loop: help-debbugs@gnu.org Subject: [bug#39873] [PATCH 1/3] substitute: Use the same port for multiple request batches. Resent-From: Christopher Baines Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Mon, 02 Mar 2020 21:08:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 39873 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 39873@debbugs.gnu.org Received: via spool by 39873-submit@debbugs.gnu.org id=B39873.15831832607329 (code B ref 39873); Mon, 02 Mar 2020 21:08:02 +0000 Received: (at 39873) by debbugs.gnu.org; 2 Mar 2020 21:07:40 +0000 Received: from localhost ([127.0.0.1]:38418 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1j8sHs-0001u8-HV for submit@debbugs.gnu.org; Mon, 02 Mar 2020 16:07:40 -0500 Received: from mira.cbaines.net ([212.71.252.8]:48634) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1j8sHq-0001to-G5 for 39873@debbugs.gnu.org; Mon, 02 Mar 2020 16:07:38 -0500 Received: from localhost (unknown [46.237.162.11]) by mira.cbaines.net (Postfix) with ESMTPSA id E2AD027BBE4 for <39873@debbugs.gnu.org>; Mon, 2 Mar 2020 21:07:37 +0000 (GMT) Received: from localhost (localhost [local]) by localhost (OpenSMTPD) with ESMTPA id 6b8fc5cb for <39873@debbugs.gnu.org>; Mon, 2 Mar 2020 21:07:35 +0000 (UTC) From: Christopher Baines Date: Mon, 2 Mar 2020 21:07:33 +0000 Message-Id: <20200302210735.13337-1-mail@cbaines.net> X-Mailer: git-send-email 2.25.0 In-Reply-To: <87imjmeagu.fsf@cbaines.net> References: <87imjmeagu.fsf@cbaines.net> MIME-Version: 1.0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.51.188.43 X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: "Guix-patches" X-getmail-retrieved-from-mailbox: Patches In http-multiple-get. * guix/scripts/substitute.scm (http-multiple-get): Switch port to p in one occurrence. --- guix/scripts/substitute.scm | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/guix/scripts/substitute.scm b/guix/scripts/substitute.scm index dfb975a24a..5ed43d36c9 100755 --- a/guix/scripts/substitute.scm +++ b/guix/scripts/substitute.scm @@ -545,7 +545,7 @@ initial connection on which HTTP requests are sent." (() (reverse result)) (remainder - (connect port remainder result)))) + (connect p remainder result)))) ((head tail ...) (let* ((resp (read-response p)) (body (response-body-port resp)) From patchwork Mon Mar 2 21:07:34 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Christopher Baines X-Patchwork-Id: 20496 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id C623E27BBEB; Mon, 2 Mar 2020 21:08:15 +0000 (GMT) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-2.9 required=5.0 tests=BAYES_00,MAILING_LIST_MULTI, UNPARSEABLE_RELAY autolearn=unavailable autolearn_force=no version=3.4.2 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTP id 875FB27BBE4 for ; Mon, 2 Mar 2020 21:08:15 +0000 (GMT) Received: from localhost ([::1]:38682 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1j8sIR-0006Qg-3W for patchwork@mira.cbaines.net; Mon, 02 Mar 2020 16:08:15 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:43444) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1j8sIG-0006Oi-1r for guix-patches@gnu.org; Mon, 02 Mar 2020 16:08:05 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1j8sIF-0003GH-4P for guix-patches@gnu.org; Mon, 02 Mar 2020 16:08:03 -0500 Received: from debbugs.gnu.org ([209.51.188.43]:60683) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1j8sIF-0003G5-0A for guix-patches@gnu.org; Mon, 02 Mar 2020 16:08:03 -0500 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1j8sIE-0001v3-Sh for guix-patches@gnu.org; Mon, 02 Mar 2020 16:08:02 -0500 X-Loop: help-debbugs@gnu.org Subject: [bug#39873] [PATCH 2/3] substitute: Make http-multiple-get batch size configurable. Resent-From: Christopher Baines Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Mon, 02 Mar 2020 21:08:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 39873 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 39873@debbugs.gnu.org Received: via spool by 39873-submit@debbugs.gnu.org id=B39873.15831832617336 (code B ref 39873); Mon, 02 Mar 2020 21:08:02 +0000 Received: (at 39873) by debbugs.gnu.org; 2 Mar 2020 21:07:41 +0000 Received: from localhost ([127.0.0.1]:38420 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1j8sHs-0001uA-Nx for submit@debbugs.gnu.org; Mon, 02 Mar 2020 16:07:40 -0500 Received: from mira.cbaines.net ([212.71.252.8]:48636) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1j8sHq-0001tn-G2 for 39873@debbugs.gnu.org; Mon, 02 Mar 2020 16:07:38 -0500 Received: from localhost (unknown [46.237.162.11]) by mira.cbaines.net (Postfix) with ESMTPSA id 002AF27BBEA for <39873@debbugs.gnu.org>; Mon, 2 Mar 2020 21:07:37 +0000 (GMT) Received: from localhost (localhost [local]) by localhost (OpenSMTPD) with ESMTPA id 8974fb65 for <39873@debbugs.gnu.org>; Mon, 2 Mar 2020 21:07:35 +0000 (UTC) From: Christopher Baines Date: Mon, 2 Mar 2020 21:07:34 +0000 Message-Id: <20200302210735.13337-2-mail@cbaines.net> X-Mailer: git-send-email 2.25.0 In-Reply-To: <20200302210735.13337-1-mail@cbaines.net> References: <87imjmeagu.fsf@cbaines.net> <20200302210735.13337-1-mail@cbaines.net> MIME-Version: 1.0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.51.188.43 X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: "Guix-patches" X-getmail-retrieved-from-mailbox: Patches * guix/scripts/substitute.scm (http-multiple-get): Add batch-size parameter. --- guix/scripts/substitute.scm | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/guix/scripts/substitute.scm b/guix/scripts/substitute.scm index 5ed43d36c9..a88cb5bcfe 100755 --- a/guix/scripts/substitute.scm +++ b/guix/scripts/substitute.scm @@ -501,7 +501,8 @@ MAX-LENGTH first elements." (loop (+ 1 len) tail (cons head result))))))) (define* (http-multiple-get base-uri proc seed requests - #:key port (verify-certificate? #t)) + #:key port (verify-certificate? #t) + (batch-size 1000)) "Send all of REQUESTS to the server at BASE-URI. Call PROC for each response, passing it the request object, the response, a port from which to read the response body, and the previous result, starting with SEED, à la @@ -511,7 +512,7 @@ initial connection on which HTTP requests are sent." (requests requests) (result seed)) (define batch - (at-most 1000 requests)) + (at-most batch-size requests)) ;; (format (current-error-port) "connecting (~a requests left)..." ;; (length requests)) From patchwork Mon Mar 2 21:07:35 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christopher Baines X-Patchwork-Id: 20497 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id 03DD327BBE4; Mon, 2 Mar 2020 21:08:16 +0000 (GMT) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-2.9 required=5.0 tests=BAYES_00,MAILING_LIST_MULTI, UNPARSEABLE_RELAY autolearn=unavailable autolearn_force=no version=3.4.2 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTP id A1E8127BBEA for ; Mon, 2 Mar 2020 21:08:15 +0000 (GMT) Received: from localhost ([::1]:38680 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1j8sIR-0006Qf-6r for patchwork@mira.cbaines.net; Mon, 02 Mar 2020 16:08:15 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]:43452) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1j8sIG-0006Oj-IB for guix-patches@gnu.org; Mon, 02 Mar 2020 16:08:05 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1j8sIF-0003H8-Gn for guix-patches@gnu.org; Mon, 02 Mar 2020 16:08:04 -0500 Received: from debbugs.gnu.org ([209.51.188.43]:60684) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1j8sIF-0003Gu-DP for guix-patches@gnu.org; Mon, 02 Mar 2020 16:08:03 -0500 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1j8sIF-0001vA-9T for guix-patches@gnu.org; Mon, 02 Mar 2020 16:08:03 -0500 X-Loop: help-debbugs@gnu.org Subject: [bug#39873] [PATCH 3/3] substitute: Close port at the end of http-multiple-get. Resent-From: Christopher Baines Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Mon, 02 Mar 2020 21:08:03 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 39873 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 39873@debbugs.gnu.org Received: via spool by 39873-submit@debbugs.gnu.org id=B39873.15831832617342 (code B ref 39873); Mon, 02 Mar 2020 21:08:03 +0000 Received: (at 39873) by debbugs.gnu.org; 2 Mar 2020 21:07:41 +0000 Received: from localhost ([127.0.0.1]:38422 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1j8sHs-0001uH-Ua for submit@debbugs.gnu.org; Mon, 02 Mar 2020 16:07:41 -0500 Received: from mira.cbaines.net ([212.71.252.8]:48638) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1j8sHq-0001tp-GG for 39873@debbugs.gnu.org; Mon, 02 Mar 2020 16:07:39 -0500 Received: from localhost (unknown [46.237.162.11]) by mira.cbaines.net (Postfix) with ESMTPSA id 0FDAB27BBEB for <39873@debbugs.gnu.org>; Mon, 2 Mar 2020 21:07:38 +0000 (GMT) Received: from localhost (localhost [local]) by localhost (OpenSMTPD) with ESMTPA id 570a5acb for <39873@debbugs.gnu.org>; Mon, 2 Mar 2020 21:07:35 +0000 (UTC) From: Christopher Baines Date: Mon, 2 Mar 2020 21:07:35 +0000 Message-Id: <20200302210735.13337-3-mail@cbaines.net> X-Mailer: git-send-email 2.25.0 In-Reply-To: <20200302210735.13337-1-mail@cbaines.net> References: <87imjmeagu.fsf@cbaines.net> <20200302210735.13337-1-mail@cbaines.net> MIME-Version: 1.0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.51.188.43 X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: "Guix-patches" X-getmail-retrieved-from-mailbox: Patches * guix/scripts/substitute.scm (http-multiple-get): Add close-port call. --- guix/scripts/substitute.scm | 1 + 1 file changed, 1 insertion(+) diff --git a/guix/scripts/substitute.scm b/guix/scripts/substitute.scm index a88cb5bcfe..e3f5837a8e 100755 --- a/guix/scripts/substitute.scm +++ b/guix/scripts/substitute.scm @@ -544,6 +544,7 @@ initial connection on which HTTP requests are sent." (() (match (drop requests processed) (() + (close-port p) (reverse result)) (remainder (connect p remainder result))))