From patchwork Fri Sep 2 09:52:37 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Roman Scherer X-Patchwork-Id: 42144 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id 901AA27BBEA; Fri, 2 Sep 2022 11:03:55 +0100 (BST) X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-2.7 required=5.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,MAILING_LIST_MULTI,SPF_HELO_PASS autolearn=unavailable autolearn_force=no version=3.4.6 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTPS id A21CD27BBE9 for ; Fri, 2 Sep 2022 11:03:54 +0100 (BST) Received: from localhost ([::1]:42630 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1oU3Wj-0003B5-Pa for patchwork@mira.cbaines.net; Fri, 02 Sep 2022 06:03:53 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:46130) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1oU3Vu-00038r-CG for guix-patches@gnu.org; Fri, 02 Sep 2022 06:03:08 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:55160) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1oU3Vu-0004ju-4K for guix-patches@gnu.org; Fri, 02 Sep 2022 06:03:02 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1oU3Vt-0004l2-ST for guix-patches@gnu.org; Fri, 02 Sep 2022 06:03:01 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#56604] [PATCH 0/8] Update Clojure to 1.11.1. Resent-From: Roman Scherer Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Fri, 02 Sep 2022 10:03:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 56604 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: Ludovic =?utf-8?q?Court=C3=A8s?= Cc: 56604-done@debbugs.gnu.org Received: via spool by 56604-done@debbugs.gnu.org id=D56604.166211297118272 (code D ref 56604); Fri, 02 Sep 2022 10:03:01 +0000 Received: (at 56604-done) by debbugs.gnu.org; 2 Sep 2022 10:02:51 +0000 Received: from localhost ([127.0.0.1]:44909 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1oU3Vi-0004kd-FD for submit@debbugs.gnu.org; Fri, 02 Sep 2022 06:02:51 -0400 Received: from mail-ej1-f46.google.com ([209.85.218.46]:42874) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1oU3Vf-0004kQ-UR for 56604-done@debbugs.gnu.org; Fri, 02 Sep 2022 06:02:48 -0400 Received: by mail-ej1-f46.google.com with SMTP id p16so2797410ejb.9 for <56604-done@debbugs.gnu.org>; Fri, 02 Sep 2022 03:02:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=burningswell-com.20210112.gappssmtp.com; s=20210112; h=mime-version:message-id:in-reply-to:date:subject:cc:to:from :user-agent:references:from:to:cc:subject:date; bh=eh0gZYj43wLm7dfsjxmOa6fsPWMbur+VbAw6qtQ3Cu4=; b=GOfJM1hFalnuneJyk0dJbe5Nrv61o0efaWrBGCUcVVW+4LpS8dBok75ndUERnbRbzH nJFxz2mvp33TIt9VzTqPYYFuy4vxEin9muRH7ADWUZGCFgT+Qo8bNQmCqJE3diWsTOe7 H9lk4/AO6wqUgpsiiJRwA4ytTpAPd1RPTVkOcO+1UyeNS0dB+nsFXxLCHkQuoUXMfuMg xZOaLTm3cVj+KJHRekUep3cxj2EiUHQbZuW+hJRU+0g8MU2V69PjqrvtKjM7ZauWiKUP c/d3vCZ6/mUeeRUgPU69QJ0QdYNzrRhjqp0g22EmJYdhKgOE0MLL1ff/iqKuyYp8pqH+ OKLQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=mime-version:message-id:in-reply-to:date:subject:cc:to:from :user-agent:references:x-gm-message-state:from:to:cc:subject:date; bh=eh0gZYj43wLm7dfsjxmOa6fsPWMbur+VbAw6qtQ3Cu4=; b=bh4D25DL4mLWgjIiPRysJiZt8+zuvbdQyzPpoFetAiBddyyVS914UUc+a3TuShrEMI 5kCEYqRuCWoB9Vufzb8wnt9g6Esci6ix9K+4LRPkz9IxZf5YZWHpibyOXrPGr4cmEU7f xLcca2uUAiVHMgP9wJwIKjA/vmZFj2rDAIwN2F+xjMeRV/uvtB8eyajNifum4qX5UG+m tlqm/7JC0bdnMUIzjbGe5jDVyIA9AA1rn7DrD/SkqWs+RkZTeBlGfWsaXwT1DfK7zXym /PEGBg1ixHROIqi+IiUb8+zmQUEGEp8X85JNXDeNFFzFQKxJnPsyTzk0htT0hqGyImzJ 92yQ== X-Gm-Message-State: ACgBeo3ie/3qQmaSn1venA8IWkK+chBNLrd/Ag98OZ+0q8bPbelXi84v PtQjdewnrPJO7FOUW7OOTCsd+Xbf7DQlzAaP X-Google-Smtp-Source: AA6agR4AV8GzwvtP46cvvr2l3LdiBQC2Ymg2dvLy0YeUn73Dtx46fzjiixt69iAmtpvK14aJsV8t/w== X-Received: by 2002:a17:906:3bd2:b0:731:3f03:1697 with SMTP id v18-20020a1709063bd200b007313f031697mr27624725ejf.289.1662112961571; Fri, 02 Sep 2022 03:02:41 -0700 (PDT) Received: from precision (tmo-119-93.customers.d1-online.com. [80.187.119.93]) by smtp.gmail.com with ESMTPSA id m14-20020a50ef0e000000b0043d5ead65a6sm1072333eds.84.2022.09.02.03.02.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 02 Sep 2022 03:02:40 -0700 (PDT) References: <87fsisg5mu.fsf@gnu.org> <87zgg55wyq.fsf@burningswell.com> <87tu5r4g20.fsf@gnu.org> User-agent: mu4e 1.8.9; emacs 28.1 From: Roman Scherer Date: Fri, 02 Sep 2022 09:52:37 +0000 In-reply-to: <87tu5r4g20.fsf@gnu.org> Message-ID: <87k06mksap.fsf@burningswell.com> MIME-Version: 1.0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: "Guix-patches" X-getmail-retrieved-from-mailbox: Patches Hi Ludo, thanks for taking a look. Here's a new patch. I changed the patch to not use define-public and I'm exporting the repack-jar function now as per your suggestion. I also updated the commit message to (hopefully) match the changelog style. Is that correct now? I did not add the #:jar-timestamp parameter because there is one more difference. The strip-jar-timestamps function in the Ant build system changes the timestamp of all files in the JAR. The reset-class-timestamps function in the Clojure build system sets the timestamp of only the class files. I could add another parameter called "extension" to the strip-jar-timestamps function if you prefer that. It would default to "*", matching all filenames and use it with "*.class" from the Clojure build system. I have the feeling the repack-jar function is more flexible as is, but I'm open to doing this change or any other suggestion you have. Wdyt? Roman Ludovic Courtès writes: > Hi Roman, > > (Catching up after vacation…) > > Roman Scherer skribis: > >> here's the promised patch to follow up with the code duplication I >> introduced in my previous patch. > > Awesome. > >> When I run the following commands after modifying the build systems they >> run for quite some time, because they were compiling a ton (the jdk, >> jetty) of things. >> >> ./pre-inst-env guix build clojure >> ./pre-inst-env guix build clojure-tools >> >> I guess this is expected, since a change in a build system might affect >> all packages being built with it. But I was wondering if there is a way >> to force only building the packages specified on the command line. Does >> such a thing exists? > > No, it doesn’t exist, because that would be building something > different. In this case, building everything that depends on > ‘ant-build-system.scm’ is unavoidable. > >> I was wondering what is the most efficient way to quickly iterate on >> changes to a build system, without recompiling the whole world for that >> build system. How would you do that? > > There’s no ideal solution as you’ll have to recompile the world anyway. > > When changing build systems, I’d usually stare at my changes for quite > some time first, to make sure I don’t have to rebuild the world on the > next day because of a typo. :-) > > Then, for small local changes, I’d build just the bottom of the > dependency graph to check for breakage (in this case, making sure the > ‘strip-jar-timestamps’ phase still works as intended). Then we can let > ci.guix build the whole thing afterwards, and make sure nothing goes > wrong. > >> From 756bfd3458ded38e1041ebb255c6b6ffe737732d Mon Sep 17 00:00:00 2001 >> From: Roman Scherer >> Date: Mon, 15 Aug 2022 15:29:25 +0000 >> Subject: [PATCH] build-system: Add repack-jar and use it in Ant & Clojure >> build systems >> >> * guix/build/ant-build-system.scm: Add repack-jar and use it in strip-jar-timestamps >> * guix/build/clojure-build-system.scm: Use repack-jar in reset-class-timestamps > > Please use the ChangeLog format, specifying procedure/variable names and > their actual changes. > > [...] > >> +(define-public (repack-jar outputs repack-fn) >> + "Unpack all jar archives, invoke repack-fn for each JAR with the directory >> +it has been unpacked to, and pack them again." > > Instead of ‘define-public’, I’d move ‘repack-jar’ to #:export at the > top, like the other procedures. > > But… > >> @@ -206,13 +205,7 @@ (define (repack-archive jar) >> (with-directory-excursion dir >> (invoke "jar" "xf" jar)) >> (delete-file jar) >> - ;; XXX: copied from (gnu build install) >> - (for-each (lambda (file) >> - (let ((s (lstat file))) >> - (unless (eq? (stat:type s) 'symlink) >> - (utime file 0 0 0 0)))) >> - (find-files dir #:directories? #t)) >> - >> + (repack-fn dir) > > Looking at the code, the only difference between the two repack > functions is the timestamp, right? In that case, I’d lean towards > adding a #:jar-timestamp parameter to ‘strip-jar-timestamps’ (rather than > this ‘repack-fn’ argument) that’d be passed to ‘utime’. The default for > #:jar-timestamp would be '(0 0 0 0); for Clojure we’d set the default > #:jar-timestamp in (guix build-system clojure) to: > > #:jar-timestamp (list early-1980 early-1980) > > WDYT? > > Thanks, > Ludo’. From c3e06172044264ff871cbe8637236415bfd6077f Mon Sep 17 00:00:00 2001 From: Roman Scherer Date: Mon, 15 Aug 2022 15:29:25 +0000 Subject: [PATCH] build-system: Remove code duplication in Ant & Clojure build systems * guix/build/ant-build-system.scm (repack-jar): Add repack-jar function to unpack, modify, and repack a JAR file. * guix/build/ant-build-system.scm (strip-jar-timestamps): Use the repack-jar function to set the timestamps of all files in the JAR file to 1980-01-01. * guix/build/clojure-build-system.scm (reset-class-timestamps): Use the repack-jar function from the Ant build system to set the timestamps of all class files in the JAR file to 1980-01-02 to prevent the Clojure compiler from re-compiling Clojure source files. --- guix/build/ant-build-system.scm | 29 ++++++++------- guix/build/clojure-build-system.scm | 55 +++++------------------------ 2 files changed, 26 insertions(+), 58 deletions(-) diff --git a/guix/build/ant-build-system.scm b/guix/build/ant-build-system.scm index fae1b47ec5..dec261cb82 100644 --- a/guix/build/ant-build-system.scm +++ b/guix/build/ant-build-system.scm @@ -27,7 +27,8 @@ (define-module (guix build ant-build-system) #:use-module (srfi srfi-1) #:use-module (srfi srfi-26) #:export (%standard-phases - ant-build)) + ant-build + repack-jar)) ;; Commentary: ;; @@ -195,10 +196,9 @@ (define (generate-index jar) outputs) #t) -(define* (strip-jar-timestamps #:key outputs - #:allow-other-keys) - "Unpack all jar archives, reset the timestamp of all contained files, and -repack them. This is necessary to ensure that archives are reproducible." +(define (repack-jar outputs repack-fn) + "Unpack all jar archives, invoke repack-fn for each JAR with the directory +it has been unpacked to, and pack them again." (define (repack-archive jar) (format #t "repacking ~a\n" jar) (let* ((dir (mkdtemp! "jar-contents.XXXXXX")) @@ -206,13 +206,7 @@ (define (repack-archive jar) (with-directory-excursion dir (invoke "jar" "xf" jar)) (delete-file jar) - ;; XXX: copied from (gnu build install) - (for-each (lambda (file) - (let ((s (lstat file))) - (unless (eq? (stat:type s) 'symlink) - (utime file 0 0 0 0)))) - (find-files dir #:directories? #t)) - + (repack-fn dir) ;; The jar tool will always set the timestamp on the manifest file ;; and the containing directory to the current time, even when we ;; reuse an existing manifest file. To avoid this we use "zip" @@ -237,6 +231,17 @@ (define (repack-archive jar) outputs) #t) +(define* (strip-jar-timestamps #:key outputs + #:allow-other-keys) + "Unpack all jar archives, reset the timestamp of all contained files, and +repack them. This is necessary to ensure that archives are reproducible." + (repack-jar outputs (lambda (dir) + (for-each (lambda (file) + (let ((s (lstat file))) + (unless (eq? (stat:type s) 'symlink) + (utime file 0 0 0 0)))) + (find-files dir #:directories? #t))))) + (define* (check #:key target (make-flags '()) (tests? (not target)) (test-target "check") #:allow-other-keys) diff --git a/guix/build/clojure-build-system.scm b/guix/build/clojure-build-system.scm index cacbefb386..b82ebc30fe 100644 --- a/guix/build/clojure-build-system.scm +++ b/guix/build/clojure-build-system.scm @@ -19,7 +19,7 @@ (define-module (guix build clojure-build-system) #:use-module ((guix build ant-build-system) #:select ((%standard-phases . %standard-phases@ant) - ant-build)) + ant-build repack-jar)) #:use-module (guix build clojure-utils) #:use-module (guix build java-utils) #:use-module (guix build syscalls) @@ -112,54 +112,17 @@ (define* (check #:key jar-names))) #t) -(define (regular-jar-file? file stat) - "Predicate returning true if FILE is ending on '.jar' -and STAT indicates it is a regular file." - (and (string-suffix? ".jar" file) - (eq? 'regular (stat:type stat)))) - -;; XXX: The only difference compared to 'strip-jar-timestamps' in -;; ant-build-system.scm is the date. TODO: Adjust and factorize. (define* (reset-class-timestamps #:key outputs #:allow-other-keys) "Unpack all jar archives, reset the timestamp of all contained class files, and repack them. This is necessary to ensure that archives are reproducible." - (define (repack-archive jar) - (format #t "resetting class timestamps and repacking ~a\n" jar) - - ;; Note: .class files need to be strictly newer than source files, - ;; otherwise the Clojure compiler will recompile sources. - (let* ((early-1980 315619200) ; 1980-01-02 UTC - (dir (mkdtemp! "jar-contents.XXXXXX")) - (manifest (string-append dir "/META-INF/MANIFEST.MF"))) - (with-directory-excursion dir - (invoke "jar" "xf" jar)) - (delete-file jar) - (for-each (lambda (file) - (let ((s (lstat file))) - (unless (eq? (stat:type s) 'symlink) - (when (string-match "^(.*)\\.class$" file) - (utime file early-1980 early-1980))))) - (find-files dir #:directories? #t)) - ;; The jar tool will always set the timestamp on the manifest file - ;; and the containing directory to the current time, even when we - ;; reuse an existing manifest file. To avoid this we use "zip" - ;; instead of "jar". It is important that the manifest appears - ;; first. - (with-directory-excursion dir - (let* ((files (find-files "." ".*" #:directories? #t)) - ;; To ensure that the reference scanner can detect all - ;; store references in the jars we disable compression - ;; with the "-0" option. - (command (if (file-exists? manifest) - `("zip" "-0" "-X" ,jar ,manifest ,@files) - `("zip" "-0" "-X" ,jar ,@files)))) - (apply invoke command))) - (utime jar 0 0))) - (for-each (match-lambda - ((output . directory) - (for-each repack-archive - (find-files directory regular-jar-file?)))) - outputs)) + (repack-jar outputs (lambda (dir) + (for-each (lambda (file) + (let ((s (lstat file)) + (early-1980 315619200)) ; 1980-01-02 UTC + (unless (eq? (stat:type s) 'symlink) + (when (string-match "^(.*)\\.class$" file) + (utime file early-1980 early-1980))))) + (find-files dir #:directories? #t))))) (define-with-docs install "Standard 'install' phase for clojure-build-system." -- 2.37.2