From patchwork Wed Sep 30 08:45:08 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Danny Milosavljevic X-Patchwork-Id: 24383 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id 842E027BBE9; Wed, 30 Sep 2020 09:46:55 +0100 (BST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-2.9 required=5.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_MSPIKE_H4,RCVD_IN_MSPIKE_WL,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.2 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTPS id 4510527BBE8 for ; Wed, 30 Sep 2020 09:46:55 +0100 (BST) Received: from localhost ([::1]:35600 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kNXlG-0005Xo-Fo for patchwork@mira.cbaines.net; Wed, 30 Sep 2020 04:46:54 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:53186) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kNXkS-0005Aj-55 for guix-patches@gnu.org; Wed, 30 Sep 2020 04:46:04 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:46545) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1kNXkR-0008EZ-Sa for guix-patches@gnu.org; Wed, 30 Sep 2020 04:46:03 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1kNXkR-0005Gx-RS for guix-patches@gnu.org; Wed, 30 Sep 2020 04:46:03 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#43591] [PATCH core-updates v2 1/5] gnu: glibc-final: Catch all cases of a glibc user not requesting 64-bit offsets and then using readdir regardless. Resent-From: Danny Milosavljevic Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Wed, 30 Sep 2020 08:46:03 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 43591 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 43591@debbugs.gnu.org Cc: Danny Milosavljevic Received: via spool by 43591-submit@debbugs.gnu.org id=B43591.160145556120218 (code B ref 43591); Wed, 30 Sep 2020 08:46:03 +0000 Received: (at 43591) by debbugs.gnu.org; 30 Sep 2020 08:46:01 +0000 Received: from localhost ([127.0.0.1]:58085 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kNXkP-0005Fq-8X for submit@debbugs.gnu.org; Wed, 30 Sep 2020 04:46:01 -0400 Received: from dd26836.kasserver.com ([85.13.145.193]:33116) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kNXkL-0005El-8G for 43591@debbugs.gnu.org; Wed, 30 Sep 2020 04:45:58 -0400 Received: from dayas.lan (80-110-126-103.cgn.dynamic.surfer.at [80.110.126.103]) by dd26836.kasserver.com (Postfix) with ESMTPSA id EB5613362FBA; Wed, 30 Sep 2020 10:45:55 +0200 (CEST) From: Danny Milosavljevic Date: Wed, 30 Sep 2020 10:45:08 +0200 Message-Id: <20200930084512.31738-2-dannym@scratchpost.org> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20200930084512.31738-1-dannym@scratchpost.org> References: <20200924141211.21649-1-dannym@scratchpost.org> <20200930084512.31738-1-dannym@scratchpost.org> MIME-Version: 1.0 Tags: patch X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: "Guix-patches" X-getmail-retrieved-from-mailbox: Patches * gnu/packages/commencement.scm (glibc-final): Catch all cases of a glibc user not requesting 64-bit offsets and then using readdir. --- gnu/packages/commencement.scm | 68 ++++++++++++++++++++++++++++++++++- 1 file changed, 67 insertions(+), 1 deletion(-) diff --git a/gnu/packages/commencement.scm b/gnu/packages/commencement.scm index e5a4caa95c..284fa65d20 100644 --- a/gnu/packages/commencement.scm +++ b/gnu/packages/commencement.scm @@ -3462,7 +3462,73 @@ exec ~a/bin/~a-~a -B~a/lib -Wl,-dynamic-linker -Wl,~a/~a \"$@\"~%" ,hurd-headers-boot0) '()) ,@(package-outputs glibc-final-with-bootstrap-bash)) - ,@(package-arguments glibc-final-with-bootstrap-bash))))) + ,@(substitute-keyword-arguments + (package-arguments glibc-final-with-bootstrap-bash) + ((#:phases phases) + `(modify-phases ,phases + (add-after 'unpack 'patch-dirent + (lambda* (#:key outputs #:allow-other-keys) + ;; Linux kernel file offsets are always 64 bits. + ;; But userspace can be built to use 32 bit offsets. + ;; + ;; "struct dirent", returned by readdir, uses d_off to store + ;; such an "offset" that it got from the Linux kernel. + ;; In the case of ext4 that "offset" is actually a 64 bit + ;; hash value. + ;; + ;; Therefore, there are cases where such an offset that it got + ;; from the Linux kernel does not fit in the "struct dirent" + ;; field "d_off". + ;; + ;; If the guest system's glibc is 32 bit AND uses 32 bit + ;; file offsets it is going to be very confused. + ;; It does check whether d_off fits into the structure + ;; it gives back to the user--and it doesn't fit. Hence readdir + ;; fails, with errno == EOVERFLOW (which is undocumented and thus + ;; an API error). + ;; This manifests itself in simple directory reads not working + ;; anymore in parts of cmake, for example. + ;; + ;; This manifested in Guix when building stuff for + ;; ARMHF on a x86_64 build host using QEMU transparent emulation. + ;; + ;; There is a very simple and complete way to avoid this problem: + ;; Just always use 64 bit offsets in user space programs (also + ;; on 32 bit machines). The Linux kernel does that already + ;; anyway. + ;; + ;; Note: We might want to avoid using 64 bit when bootstrapping + ;; using mescc (since mescc doesn't directly support 64 bit + ;; values)--but then bootstrapping has to be done on a + ;; file system other than ext4, or on ext4 with the feature + ;; "dir_index" disabled. + ;; + ;; The change below does not affect 64 bit users. + ;; + ;; See . + (let ((port (open-file "dirent/dirent.h" "a"))) + (display " +#ifndef _LIBC +#if __SIZEOF_LONG__ < 8 +#ifndef __USE_FILE_OFFSET64 +#if defined(_FILE_OFFSET_BITS) && _FILE_OFFSET_BITS == 32 +#warning \"Using -D_FILE_OFFSET_BITS=32 and using readdir is a bad idea, see \" +#else +#undef readdir +#define readdir @READDIR_WITHOUT_FILE_OFFSET64_IS_A_REALLY_BAD_IDEA@ +#endif +#endif +#endif +#endif +" port) + (close-port port)) + ;; This file includes and thus checks sanity already. + ;; TODO: Check dirent/scandir-tail.c, dirent/scandir64-tail.c. + (substitute* "posix/glob.c" + (("(#[ ]*define[ ][ ]*readdir)") " +#undef readdir +#define readdir")) + #t))))))))) (define/system-dependent gcc-boot0-wrapped ;; Make the cross-tools GCC-BOOT0 and BINUTILS-BOOT0 available under the From patchwork Wed Sep 30 08:45:12 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Danny Milosavljevic X-Patchwork-Id: 24384 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id A858927BBE9; Wed, 30 Sep 2020 09:47:05 +0100 (BST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-2.9 required=5.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_MSPIKE_H4,RCVD_IN_MSPIKE_WL autolearn=unavailable autolearn_force=no version=3.4.2 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTPS id 7334927BBE8 for ; Wed, 30 Sep 2020 09:47:05 +0100 (BST) Received: from localhost ([::1]:35918 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1kNXlQ-0005fv-Md for patchwork@mira.cbaines.net; Wed, 30 Sep 2020 04:47:04 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:53188) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1kNXkS-0005B2-Hd for guix-patches@gnu.org; Wed, 30 Sep 2020 04:46:04 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:46546) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1kNXkS-0008Ed-8t for guix-patches@gnu.org; Wed, 30 Sep 2020 04:46:04 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1kNXkS-0005H4-7W for guix-patches@gnu.org; Wed, 30 Sep 2020 04:46:04 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#43591] [PATCH core-updates v2 5/5] gnu: rhash: Explicity declare the _FILE_OFFSET_BITS we want. Resent-From: Danny Milosavljevic Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Wed, 30 Sep 2020 08:46:04 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 43591 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 43591@debbugs.gnu.org Cc: Danny Milosavljevic Received: via spool by 43591-submit@debbugs.gnu.org id=B43591.160145556220228 (code B ref 43591); Wed, 30 Sep 2020 08:46:04 +0000 Received: (at 43591) by debbugs.gnu.org; 30 Sep 2020 08:46:02 +0000 Received: from localhost ([127.0.0.1]:58087 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kNXkP-0005G3-O1 for submit@debbugs.gnu.org; Wed, 30 Sep 2020 04:46:02 -0400 Received: from dd26836.kasserver.com ([85.13.145.193]:33130) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1kNXkM-0005Ey-9G for 43591@debbugs.gnu.org; Wed, 30 Sep 2020 04:45:58 -0400 Received: from dayas.lan (80-110-126-103.cgn.dynamic.surfer.at [80.110.126.103]) by dd26836.kasserver.com (Postfix) with ESMTPSA id EF1093363599; Wed, 30 Sep 2020 10:45:56 +0200 (CEST) From: Danny Milosavljevic Date: Wed, 30 Sep 2020 10:45:12 +0200 Message-Id: <20200930084512.31738-6-dannym@scratchpost.org> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20200930084512.31738-1-dannym@scratchpost.org> References: <20200924141211.21649-1-dannym@scratchpost.org> <20200930084512.31738-1-dannym@scratchpost.org> MIME-Version: 1.0 Tags: patch X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: "Guix-patches" X-getmail-retrieved-from-mailbox: Patches * gnu/packages/crypto.scm (rhash)[arguments]<#:make-flags>: Explicity declare the _FILE_OFFSET_BITS we want. --- gnu/packages/crypto.scm | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/gnu/packages/crypto.scm b/gnu/packages/crypto.scm index 028c140185..a10dd62e8b 100644 --- a/gnu/packages/crypto.scm +++ b/gnu/packages/crypto.scm @@ -845,8 +845,13 @@ BLAKE.") "/bin/" ,target "-gcc")) '()))) #:make-flags - ;; The binaries in /bin need some help finding librhash.so.0. - (list (string-append "LDFLAGS=-Wl,-rpath=" %output "/lib")) + (list ;; This package uses a configure script that is not from GNU + ;; autotools; it doesn't handle the environment variable + ;; CFLAGS (or for that matter the configure option). + ;; Therefore, directly pass it to make. + "CFLAGS=-D_FILE_OFFSET_BITS=64" + ;; The binaries in /bin need some help finding librhash.so.0. + (string-append "LDFLAGS=-Wl,-rpath=" %output "/lib")) #:test-target "test" ; ‘make check’ just checks the sources #:phases (modify-phases %standard-phases