From patchwork Fri Aug 27 16:44:23 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Marius Bakke X-Patchwork-Id: 32338 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id 5105F27BBE3; Fri, 27 Aug 2021 17:45:17 +0100 (BST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-2.9 required=5.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.2 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTPS id 4ABB027BBE1 for ; Fri, 27 Aug 2021 17:45:16 +0100 (BST) Received: from localhost ([::1]:48494 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mJeyg-0007Ag-Mc for patchwork@mira.cbaines.net; Fri, 27 Aug 2021 12:45:14 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:44520) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mJeyU-0007AX-40 for guix-patches@gnu.org; Fri, 27 Aug 2021 12:45:02 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:41289) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1mJeyT-0002Gg-Nd for guix-patches@gnu.org; Fri, 27 Aug 2021 12:45:01 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1mJeyT-0007TC-LO for guix-patches@gnu.org; Fri, 27 Aug 2021 12:45:01 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#50227] [PATCH] build-system/go: Trim store references using the native compiler option. Resent-From: Marius Bakke Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Fri, 27 Aug 2021 16:45:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 50227 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 50227@debbugs.gnu.org Received: via spool by 50227-submit@debbugs.gnu.org id=B50227.163008268228675 (code B ref 50227); Fri, 27 Aug 2021 16:45:01 +0000 Received: (at 50227) by debbugs.gnu.org; 27 Aug 2021 16:44:42 +0000 Received: from localhost ([127.0.0.1]:52835 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mJey5-0007SN-CB for submit@debbugs.gnu.org; Fri, 27 Aug 2021 12:44:41 -0400 Received: from eggs.gnu.org ([209.51.188.92]:56570) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mJexz-0007S6-Bq for 50227@debbugs.gnu.org; Fri, 27 Aug 2021 12:44:36 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:54008) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mJexu-0001wH-3c for 50227@debbugs.gnu.org; Fri, 27 Aug 2021 12:44:26 -0400 Received: from host-37-191-231-185.lynet.no ([37.191.231.185]:38010 helo=localhost) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mJext-0000bo-N8 for 50227@debbugs.gnu.org; Fri, 27 Aug 2021 12:44:25 -0400 From: Marius Bakke Date: Fri, 27 Aug 2021 18:44:23 +0200 Message-Id: <20210827164423.17109-1-marius@gnu.org> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210827151052.12611-1-marius@gnu.org> References: <20210827151052.12611-1-marius@gnu.org> MIME-Version: 1.0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: "Guix-patches" X-getmail-retrieved-from-mailbox: Patches * guix/build/go-build-system.scm (build): Add '-trimpath' to the 'go install' invocation. (remove-store-references, remove-go-references): Remove procedures. (%standard-phases): Don't include remove-go-references. * gnu/packages/docker.scm (docker)[arguments]: Add the '-trimpath' option to the build flags. Remove phase remove-go-references. * gnu/packages/uucp.scm (nncp)[arguments]: Likewise. --- gnu/packages/docker.scm | 7 ++-- gnu/packages/uucp.scm | 8 ++-- guix/build/go-build-system.scm | 70 +++------------------------------- 3 files changed, 14 insertions(+), 71 deletions(-) diff --git a/gnu/packages/docker.scm b/gnu/packages/docker.scm index 8bac1b89ce..108f355aa7 100644 --- a/gnu/packages/docker.scm +++ b/gnu/packages/docker.scm @@ -535,6 +535,8 @@ built-in registry server of Docker.") ;; Respectively, strip the symbol table and debug ;; information, and the DWARF symbol table. (setenv "LDFLAGS" "-s -w") + ;; Trim store references from the compiled binary. + (setenv "BUILDFLAGS" "-trimpath") ;; Make build faster (setenv "GOCACHE" "/tmp") #t)) @@ -568,10 +570,7 @@ built-in registry server of Docker.") (install-file "bundles/dynbinary-daemon/dockerd" out-bin) (install-file (string-append "bundles/dynbinary-daemon/dockerd-" (getenv "VERSION")) - out-bin) - #t))) - (add-after 'install 'remove-go-references - (assoc-ref go:%standard-phases 'remove-go-references))))) + out-bin))))))) (inputs `(("btrfs-progs" ,btrfs-progs) ("containerd" ,containerd) ; for containerd-shim diff --git a/gnu/packages/uucp.scm b/gnu/packages/uucp.scm index 120417dea1..9d39c88fe5 100644 --- a/gnu/packages/uucp.scm +++ b/gnu/packages/uucp.scm @@ -127,6 +127,10 @@ between computers.") (substitute* (list "bin/default.do" "bin/hjson-cli.do" "test.do") ((" -mod=vendor") "") ((" -m") "")) + (substitute* (list "bin/default.do" "bin/hjson-cli.do") + ;; Prevent reference to the Go inputs in the compiled binaries. + (("\\$GO build") + "$GO build -trimpath")) ;; Use the correct module path. `go list` does not report the ;; correct module path since we have moved the source files. (substitute* "bin/default.do" @@ -138,9 +142,7 @@ between computers.") (replace 'check (lambda* (#:key tests? #:allow-other-keys) (when tests? - (invoke "contrib/do" "-c" "test")))) - (add-after 'install 'remove-go-references - (assoc-ref go:%standard-phases 'remove-go-references))))) + (invoke "contrib/do" "-c" "test"))))))) (inputs `(("go-github-com-davecgh-go-xdr" ,go-github-com-davecgh-go-xdr) ("go-github-com-dustin-go-humanize" ,go-github-com-dustin-go-humanize) diff --git a/guix/build/go-build-system.scm b/guix/build/go-build-system.scm index 37936fe5ca..fc5ee39c8d 100644 --- a/guix/build/go-build-system.scm +++ b/guix/build/go-build-system.scm @@ -28,8 +28,6 @@ #:use-module (ice-9 match) #:use-module (ice-9 ftw) #:use-module (srfi srfi-1) - #:use-module (rnrs io ports) - #:use-module (rnrs bytevectors) #:export (%standard-phases go-build)) @@ -47,12 +45,9 @@ ;; structure called a 'workspace' [1]. This workspace can be found by Go via ;; the GOPATH environment variable. Typically, all Go source code and compiled ;; objects are kept in a single workspace, but GOPATH may be a list of -;; directories [2]. In this go-build-system we create a file system union of -;; the Go-language dependencies. Previously, we made GOPATH a list of store -;; directories, but stopped because Go programs started keeping references to -;; these directories in Go 1.11: -;; . -;; +;; directories [2], which we rely on here, with the caveat that the current +;; package must appear first on GOPATH. +; ;; Go software, whether a package or a command, is uniquely named using an ;; 'import path'. The import path is based on the URL of the software's source. ;; Because most source code is provided over the internet, the import path is @@ -88,7 +83,6 @@ ;; a tmpdir when creating the inputs union. ;; * Use Go modules [4] ;; * Re-use compiled packages [5] -;; * Stop needing remove-go-references (-trimpath ? ) ;; * Remove module packages, only offering the full Git repos? This is ;; more idiomatic, I think, because Go downloads Git repos, not modules. ;; What are the trade-offs? @@ -194,6 +188,8 @@ unpacking." (apply invoke "go" "install" "-v" ; print the name of packages as they are compiled "-x" ; print each command as it is invoked + ;; Trim store references from the compiled binaries. + "-trimpath" ;; Respectively, strip the symbol table and debug ;; information, and the DWARF symbol table. "-ldflags=-s -w" @@ -236,59 +232,6 @@ the standard install-license-files phase to first enter the correct directory." unpack-path)) (apply (assoc-ref gnu:%standard-phases 'install-license-files) args))) -(define* (remove-store-reference file file-name - #:optional (store (%store-directory))) - "Remove from FILE occurrences of FILE-NAME in STORE; return #t when FILE-NAME -is encountered in FILE, #f otherwise. This implementation reads FILE one byte at -a time, which is slow. Instead, we should use the Boyer-Moore string search -algorithm; there is an example in (guix build grafts)." - (define pattern - (string-take file-name - (+ 34 (string-length (%store-directory))))) - - (with-fluids ((%default-port-encoding #f)) - (with-atomic-file-replacement file - (lambda (in out) - ;; We cannot use `regexp-exec' here because it cannot deal with - ;; strings containing NUL characters. - (format #t "removing references to `~a' from `~a'...~%" file-name file) - (setvbuf in 'block 65536) - (setvbuf out 'block 65536) - (fold-port-matches (lambda (match result) - (put-bytevector out (string->utf8 store)) - (put-u8 out (char->integer #\/)) - (put-bytevector out - (string->utf8 - "eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee-")) - #t) - #f - pattern - in - (lambda (char result) - (put-u8 out (char->integer char)) - result)))))) - -(define* (remove-go-references #:key allow-go-reference? - inputs outputs #:allow-other-keys) - "Remove any references to the Go compiler from the compiled Go executable -files in OUTPUTS." -;; We remove this spurious reference to save bandwidth when installing Go -;; executables. It would be better to not embed the reference in the first -;; place, but I'm not sure how to do that. The subject was discussed at: -;; - (if allow-go-reference? - #t - (let ((go (assoc-ref inputs "go")) - (bin "/bin")) - (for-each (lambda (output) - (when (file-exists? (string-append (cdr output) - bin)) - (for-each (lambda (file) - (remove-store-reference file go)) - (find-files (string-append (cdr output) bin))))) - outputs) - #t))) - (define %standard-phases (modify-phases gnu:%standard-phases (delete 'bootstrap) @@ -299,8 +242,7 @@ files in OUTPUTS." (replace 'build build) (replace 'check check) (replace 'install install) - (replace 'install-license-files install-license-files) - (add-after 'install 'remove-go-references remove-go-references))) + (replace 'install-license-files install-license-files))) (define* (go-build #:key inputs (phases %standard-phases) #:allow-other-keys #:rest args)