From patchwork Thu Dec 29 18:42:12 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Julien Lepiller X-Patchwork-Id: 45686 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id 1FB4927BBED; Thu, 29 Dec 2022 18:56:27 +0000 (GMT) X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-3.7 required=5.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,MAILING_LIST_MULTI,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS, URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.6 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTPS id 57C0027BBE9 for ; Thu, 29 Dec 2022 18:56:24 +0000 (GMT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pAxs7-00084I-IX; Thu, 29 Dec 2022 13:43:19 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pAxrr-0007z5-2V for guix-patches@gnu.org; Thu, 29 Dec 2022 13:43:04 -0500 Received: from debbugs.gnu.org ([209.51.188.43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1pAxrq-0006mJ-Eq for guix-patches@gnu.org; Thu, 29 Dec 2022 13:43:02 -0500 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1pAxrq-00088V-AP for guix-patches@gnu.org; Thu, 29 Dec 2022 13:43:02 -0500 X-Loop: help-debbugs@gnu.org Subject: [bug#60408] [PATCH wip] guix: Support showing status in parallel. Resent-From: Julien Lepiller Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Thu, 29 Dec 2022 18:43:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: report 60408 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 60408@debbugs.gnu.org X-Debbugs-Original-To: guix-patches@gnu.org Received: via spool by submit@debbugs.gnu.org id=B.167233936931251 (code B ref -1); Thu, 29 Dec 2022 18:43:02 +0000 Received: (at submit) by debbugs.gnu.org; 29 Dec 2022 18:42:49 +0000 Received: from localhost ([127.0.0.1]:32852 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pAxrc-00087y-2s for submit@debbugs.gnu.org; Thu, 29 Dec 2022 13:42:49 -0500 Received: from lists.gnu.org ([209.51.188.17]:59066) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1pAxrY-00087p-UU for submit@debbugs.gnu.org; Thu, 29 Dec 2022 13:42:46 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pAxrY-0007w3-6n for guix-patches@gnu.org; Thu, 29 Dec 2022 13:42:44 -0500 Received: from lepiller.eu ([2a00:5884:8208::1]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pAxrU-0006iW-UH for guix-patches@gnu.org; Thu, 29 Dec 2022 13:42:43 -0500 Received: from lepiller.eu (localhost [127.0.0.1]) by lepiller.eu (OpenSMTPD) with ESMTP id da4e5937 for ; Thu, 29 Dec 2022 18:42:31 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed; d=lepiller.eu; h=date:from :to:subject:message-id:mime-version:content-type; s=dkim; bh=11L AcLwX5WV2Sa1J1RtrT3tTg0Ywy/ZXSdMsNszpC0A=; b=iUFRNjPwZscRvbns+HS v3NoJ1xUud0SDPTgUAssKmxlcMRUuCrNKI7heyS6h23uSBbT79DMiZ4nJe+XBW7E Bez1GgrJooYWIpgl7ez1uHR+BRyiH6FwGj0EGNyoVaFxYWA2iw6bG/vvXIzYl3lC /11lhgesgTjiDtbZJAGYyC/EiLmTo6KmYZlP1unSFXTFQUlOkMlBn2GFt4+r+qtS gmjrunoeZ34pXcbmbQecBnCvhrUmXPBjA2BHbap3TrraXGMulACnRRaxL2Dn6UvR 48QVcFXPFYCKWogHHNGwUkg3FhvcWvo9QnmQ8sj5mkbgUsxxzfB8SObqqoxFToOF X0g== Received: by lepiller.eu (OpenSMTPD) with ESMTPSA id a17d1b78 (TLSv1.3:AEAD-AES256-GCM-SHA384:256:NO) for ; Thu, 29 Dec 2022 18:42:30 +0000 (UTC) Date: Thu, 29 Dec 2022 19:42:12 +0100 From: Julien Lepiller Message-ID: <20221229194212.44e5cabf@tachikoma.lepiller.eu> X-Mailer: Claws Mail 4.1.1 (GTK 3.24.30; x86_64-pc-linux-gnu) MIME-Version: 1.0 Received-SPF: pass client-ip=2a00:5884:8208::1; envelope-from=julien@lepiller.eu; helo=lepiller.eu X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org X-getmail-retrieved-from-mailbox: Patches Hi Guix! The attached patch allows showing more detailed status in parallel. One of the reasons for doing that is for supporting another of my patches, that allows specifying download tasks and build tasks numbers separately, with a default of 1 for each (so 2 tasks in parallel by default). With verbosity level 2, all messages from all builds are shown (interleaved, obviously), and messages for other levels are the same. When only one build is performed, no changes are visible. When multiple builds are happening in parallel, this patch shows one line per running job (whether build or download), and messages above. This will look like this: module-import-compiled.drv 75% ▕█████████████████████████ ▏ openjdk-9.181-jdk 337.6MiB 4.3MiB/s 00:07 ▕█▌ ▏ 8.8% If there is no progress lines in the output of a build, it doesn't show a spinner (yet), so no feedback that something is happening, but that's planned for v2. One other issue is probably caused by having lots of events (caused by build log output), that make status lines blink. Another issue is when you ^C, the cursor is on the first status line. My ideas for this patch are: First, it's possible to go back a few lines with an ANSI escape code, so basically print all status lines, then go back to the first line. When an even needs to print a new line, let it do it from the first status line, and print the status lines from below it. Sometimes, an event will contain an incomplete line (for instance, it ends with \r instead of \n), so I want to record the line to prevent it from being overwritten by a status line. It's printed together with the status lines, so we can go back to that line and print the rest of the line when we get more of it. Thoughts, ideas? From c40fc712dec93299657e916907bc603d30178327 Mon Sep 17 00:00:00 2001 Message-Id: From: Julien Lepiller Date: Thu, 29 Dec 2022 19:20:34 +0100 Subject: [PATCH] guix: Support showing status in parallel. * guix/status.scm (build-status): Add `last-daemon-line` field. (build): Add `last-line` and `start` fields. (update-build): Record partial lines (not ending with \n) in the last-line field of the new build or status record. (print-build-event): Always print status of all current builds and downloads at the end. Update all status lines. --- guix/status.scm | 358 +++++++++++++++++++++++++++++++++--------------- 1 file changed, 245 insertions(+), 113 deletions(-) diff --git a/guix/status.scm b/guix/status.scm index 2c69f49fb5..5eb3ebc46b 100644 --- a/guix/status.scm +++ b/guix/status.scm @@ -49,6 +49,7 @@ (define-module (guix status) build-status-downloading build-status-builds-completed build-status-downloads-completed + build-status-last-daemon-line build? build @@ -57,6 +58,8 @@ (define-module (guix status) build-log-file build-phase build-completion + build-start + build-last-line download? download @@ -100,11 +103,13 @@ (define-record-type* build-status make-build-status (builds-completed build-status-builds-completed ;list of (default '())) (downloads-completed build-status-downloads-completed ;list of - (default '()))) + (default '())) + (last-daemon-line build-status-last-daemon-line ;string + (default ""))) ;; On-going or completed build. (define-immutable-record-type - (%build derivation id system log-file phase completion) + (%build derivation id system log-file phase completion start last-line) build? (derivation build-derivation) ;string (.drv file name) (id build-id) ;#f | integer @@ -113,11 +118,17 @@ (define-immutable-record-type (phase build-phase ;#f | symbol set-build-phase) (completion build-completion ;#f | integer (percentage) - set-build-completion)) + set-build-completion) + (start build-start ;