diff mbox series

[bug#57788] gnu: rust: Disable debug info to reduce build requirements

Message ID 874jxahlpi.fsf@netris.org
State New
Headers show
Series [bug#57788] gnu: rust: Disable debug info to reduce build requirements | expand

Checks

Context Check Description
cbaines/applying patch fail View Laminar job
cbaines/issue success View issue

Commit Message

Mark H Weaver Sept. 13, 2022, 11:56 p.m. UTC 
The following patch, which I've tested only on my private branch of
Guix, dramatically reduces the amount of memory and time required to
build Rust.

With this patch applied, I'm able to build Rust on a Thinkpad X200 with
only 4 GB of RAM using both cores.  Without it, I'm unable to build Rust
at all, even with parallel builds disabled, running in a bare text-only
terminal, and with every nonessential daemon stopped.  When I tried, a
single compiler process used more than 9 GB of virtual memory with heavy
thrashing before the OOM killer finally put it out of its misery.

Moreover, with this patch the builds are dramatically faster than I've
previously observed.  In my earlier bootstraps of Rust (up to
rust-1.51), it took about 5 hours to build each compiler on my machine.
Now it takes only about 2 hours per compiler, except for the initial
bootstrap (1.54) which takes about 4 hours.

The following patch is meant to be applied to the 'staging' branch, but
I've only tested it on my own private branch, which is based on very old
'master' (April 2021) and is thus quite different by now.

I offer it here in case it might be of interest, although it hasn't been
tested on contemporary Guix.  I'm not subscribed to this mailing list,
so please make sure to CC me on any replies you'd like me to see.

     Regards,
       Mark

Comments

Efraim Flashner Oct. 7, 2022, 10:36 a.m. UTC | #1 
On Tue, Sep 13, 2022 at 07:56:46PM -0400, Mark H Weaver wrote:
> The following patch, which I've tested only on my private branch of
> Guix, dramatically reduces the amount of memory and time required to
> build Rust.
> 
> With this patch applied, I'm able to build Rust on a Thinkpad X200 with
> only 4 GB of RAM using both cores.  Without it, I'm unable to build Rust
> at all, even with parallel builds disabled, running in a bare text-only
> terminal, and with every nonessential daemon stopped.  When I tried, a
> single compiler process used more than 9 GB of virtual memory with heavy
> thrashing before the OOM killer finally put it out of its misery.

Your x200 is an x86_64? How much RAM+swap did you end up using with this
patch? I was going to point you to a patch upstream in mrustc but I
can't find it now, where they disabled debug builds by default, but I
can't find it now.

> Moreover, with this patch the builds are dramatically faster than I've
> previously observed.  In my earlier bootstraps of Rust (up to
> rust-1.51), it took about 5 hours to build each compiler on my machine.
> Now it takes only about 2 hours per compiler, except for the initial
> bootstrap (1.54) which takes about 4 hours.

I tested it out on my machine, it brought the build time down from 101
minutes to 71 minutes. Not the same improvement, but still noticeable.

> The following patch is meant to be applied to the 'staging' branch, but
> I've only tested it on my own private branch, which is based on very old
> 'master' (April 2021) and is thus quite different by now.
> 
> I offer it here in case it might be of interest, although it hasn't been
> tested on contemporary Guix.  I'm not subscribed to this mailing list,
> so please make sure to CC me on any replies you'd like me to see.
> 
>      Regards,
>        Mark

I'm building out to librsvg on staging right now with this patch
applied, so hopefully we'll see that it's good.

I'm assuming that setting CARGO_PROFILE_DEV_DEBUG to false won't cause
any problems.
Efraim Flashner Oct. 9, 2022, 12:38 p.m. UTC | #2 
On Fri, Oct 07, 2022 at 01:36:08PM +0300, Efraim Flashner wrote:
> On Tue, Sep 13, 2022 at 07:56:46PM -0400, Mark H Weaver wrote:
> 
> > Moreover, with this patch the builds are dramatically faster than I've
> > previously observed.  In my earlier bootstraps of Rust (up to
> > rust-1.51), it took about 5 hours to build each compiler on my machine.
> > Now it takes only about 2 hours per compiler, except for the initial
> > bootstrap (1.54) which takes about 4 hours.
> 
> I tested it out on my machine, it brought the build time down from 101
> minutes to 71 minutes. Not the same improvement, but still noticeable.

To follow up, it brought the time to build rust-bootstrap aka 1.54, on
riscv64-linux down from ~56 hours to 13 hours.
diff mbox series

Patch

From 74a004295dc7bb6bdb20172aaf4778435bdf5832 Mon Sep 17 00:00:00 2001
From: Mark H Weaver <mhw@netris.org>
Date: Tue, 13 Sep 2022 19:28:03 -0400
Subject: [PATCH] gnu: rust: Disable debug info to reduce build memory
 requirements.

This dramatically reduces the memory required to build Rust, and also
speeds up the build.

* gnu/packages/patches/mrustc-disable-debug.patch: New file
* gnu/local.mk (dist_patch_DATA): Add it.
* gnu/packages/rust.scm (%mrustc-source): Apply patch.
(rust-bootstrap, rust-1.55)[arguments]: In 'build' phase, set
CARGO_PROFILE_DEV_DEBUG environment variable to "false".
---
 gnu/local.mk                                  |   1 +
 .../patches/mrustc-disable-debug.patch        | 110 ++++++++++++++++++
 gnu/packages/rust.scm                         |   7 +-
 3 files changed, 117 insertions(+), 1 deletion(-)
 create mode 100644 gnu/packages/patches/mrustc-disable-debug.patch

diff --git a/gnu/local.mk b/gnu/local.mk
index cc96b77e03..a7f81ceda5 100644
--- a/gnu/local.mk
+++ b/gnu/local.mk
@@ -1521,6 +1521,7 @@  dist_patch_DATA =						\
   %D%/packages/patches/monero-use-system-miniupnpc.patch			\
   %D%/packages/patches/mosaicatcher-unbundle-htslib.patch	\
   %D%/packages/patches/mrrescue-support-love-11.patch		\
+  %D%/packages/patches/mrustc-disable-debug.patch		\
   %D%/packages/patches/mrustc-riscv64-support.patch		\
   %D%/packages/patches/mtools-mformat-uninitialized.patch	\
   %D%/packages/patches/mupen64plus-ui-console-notice.patch	\
diff --git a/gnu/packages/patches/mrustc-disable-debug.patch b/gnu/packages/patches/mrustc-disable-debug.patch
new file mode 100644
index 0000000000..6458cecb31
--- /dev/null
+++ b/gnu/packages/patches/mrustc-disable-debug.patch
@@ -0,0 +1,110 @@ 
+diff -ru mrustc-0.10-orig/Makefile mrustc-0.10/Makefile
+--- mrustc-0.10-orig/Makefile	2022-09-13 07:49:26.299978699 -0400
++++ mrustc-0.10/Makefile	2022-09-13 07:52:19.243970927 -0400
+@@ -38,9 +38,9 @@
+ # - Disable deleting intermediate files
+ .SECONDARY:
+ 
+-LINKFLAGS := -g
++LINKFLAGS := 
+ LIBS := -lz
+-CXXFLAGS := -g -Wall
++CXXFLAGS := -Wall
+ CXXFLAGS += -std=c++14
+ #CXXFLAGS += -Wextra
+ CXXFLAGS += -O2
+diff -ru mrustc-0.10-orig/tools/common/Makefile mrustc-0.10/tools/common/Makefile
+--- mrustc-0.10-orig/tools/common/Makefile	2022-09-13 07:49:26.335978698 -0400
++++ mrustc-0.10/tools/common/Makefile	2022-09-13 07:50:58.243974567 -0400
+@@ -11,7 +11,7 @@
+ BIN := ../../bin/common_lib.a
+ OBJS = toml.o path.o debug.o
+ 
+-CXXFLAGS := -Wall -std=c++14 -g -O2
++CXXFLAGS := -Wall -std=c++14 -O2
+ 
+ CXXFLAGS += $(CXXFLAGS_EXTRA)
+ 
+diff -ru mrustc-0.10-orig/tools/dump_hirfile/Makefile mrustc-0.10/tools/dump_hirfile/Makefile
+--- mrustc-0.10-orig/tools/dump_hirfile/Makefile	2022-09-13 07:49:26.335978698 -0400
++++ mrustc-0.10/tools/dump_hirfile/Makefile	2022-09-13 07:51:02.723974366 -0400
+@@ -15,7 +15,7 @@
+ OBJS := main.o
+ 
+ LINKFLAGS := -g -lpthread -lz
+-CXXFLAGS := -Wall -std=c++14 -g -O2
++CXXFLAGS := -Wall -std=c++14 -O2
+ CXXFLAGS += -I ../common -I ../../src -I ../../src/include
+ 
+ CXXFLAGS += $(CXXFLAGS_EXTRA)
+diff -ru mrustc-0.10-orig/tools/minicargo/build.cpp mrustc-0.10/tools/minicargo/build.cpp
+--- mrustc-0.10-orig/tools/minicargo/build.cpp	2022-09-13 07:49:26.335978698 -0400
++++ mrustc-0.10/tools/minicargo/build.cpp	2022-09-13 07:47:19.567984395 -0400
+@@ -956,7 +956,7 @@
+             }
+         }
+     }
+-    if( true /*this->enable_debug*/ ) {
++    if( false /*this->enable_debug*/ ) {
+         args.push_back("-g");
+         args.push_back("--cfg"); args.push_back("debug_assertions");
+     }
+@@ -1106,7 +1106,7 @@
+     args.push_back("--crate-type"); args.push_back("bin");
+     args.push_back("-o"); args.push_back(outfile);
+     args.push_back("-L"); args.push_back(this->get_output_dir(true).str()); // NOTE: Forces `is_for_host` to true here.
+-    if( true )
++    if( false )
+     {
+         args.push_back("-g");
+     }
+diff -ru mrustc-0.10-orig/tools/minicargo/Makefile mrustc-0.10/tools/minicargo/Makefile
+--- mrustc-0.10-orig/tools/minicargo/Makefile	2022-09-13 07:49:26.335978698 -0400
++++ mrustc-0.10/tools/minicargo/Makefile	2022-09-13 07:51:07.011974173 -0400
+@@ -16,7 +16,7 @@
+ OBJS := main.o build.o manifest.o repository.o cfg.o
+ 
+ LINKFLAGS := -g -lpthread
+-CXXFLAGS := -Wall -std=c++14 -g -O2
++CXXFLAGS := -Wall -std=c++14 -O2
+ CXXFLAGS += -I ../common
+ 
+ CXXFLAGS += $(CXXFLAGS_EXTRA)
+diff -ru mrustc-0.10-orig/tools/mir_opt_test/Makefile mrustc-0.10/tools/mir_opt_test/Makefile
+--- mrustc-0.10-orig/tools/mir_opt_test/Makefile	2022-09-13 07:49:26.339978698 -0400
++++ mrustc-0.10/tools/mir_opt_test/Makefile	2022-09-13 07:51:10.555974014 -0400
+@@ -15,7 +15,7 @@
+ LIBS := ../../bin/mrustc.a ../../bin/common_lib.a
+ 
+ LINKFLAGS := -g -lpthread -lz
+-CXXFLAGS := -Wall -std=c++14 -g -O2
++CXXFLAGS := -Wall -std=c++14 -O2
+ CXXFLAGS += -I ../common -I ../../src/include -I ../../src -I .
+ CXXFLAGS += -Wno-misleading-indentation	# Gets REALLY confused by the TU_ARM macro
+ 
+diff -ru mrustc-0.10-orig/tools/standalone_miri/Makefile mrustc-0.10/tools/standalone_miri/Makefile
+--- mrustc-0.10-orig/tools/standalone_miri/Makefile	2022-09-13 07:49:26.339978698 -0400
++++ mrustc-0.10/tools/standalone_miri/Makefile	2022-09-13 07:51:14.731973826 -0400
+@@ -15,7 +15,7 @@
+ OBJS += miri.o miri_extern.o miri_intrinsic.o
+ 
+ LINKFLAGS := -g -lpthread
+-CXXFLAGS := -Wall -std=c++14 -g -O2
++CXXFLAGS := -Wall -std=c++14 -O2
+ CXXFLAGS += -I ../common -I ../../src/include -I .
+ CXXFLAGS += -Wno-misleading-indentation	# Gets REALLY confused by the TU_ARM macro
+ 
+diff -ru mrustc-0.10-orig/tools/testrunner/Makefile mrustc-0.10/tools/testrunner/Makefile
+--- mrustc-0.10-orig/tools/testrunner/Makefile	2022-09-13 07:49:26.339978698 -0400
++++ mrustc-0.10/tools/testrunner/Makefile	2022-09-13 07:51:24.819973373 -0400
+@@ -5,8 +5,8 @@
+ BIN := ../../bin/testrunner
+ OBJS := main.o path.o
+ 
+-LINKFLAGS := -g
+-CXXFLAGS := -Wall -std=c++14 -g -O2
++LINKFLAGS := 
++CXXFLAGS := -Wall -std=c++14 -O2
+ 
+ CXXFLAGS += $(CXXFLAGS_EXTRA)
+ LINKFLAGS += $(LINKFLAGS_EXTRA)
diff --git a/gnu/packages/rust.scm b/gnu/packages/rust.scm
index 25295a283e..7d1b17ad5f 100644
--- a/gnu/packages/rust.scm
+++ b/gnu/packages/rust.scm
@@ -139,7 +139,8 @@  (define %mrustc-source
       (sha256
        (base32
         "0f7kh4n2663sn0z3xib8gzw0s97qpvwag40g2vs3bfjlrbpgi9z0"))
-      (patches (search-patches "mrustc-riscv64-support.patch")))))
+      (patches (search-patches "mrustc-riscv64-support.patch"
+                               "mrustc-disable-debug.patch")))))
 
 ;;; Rust 1.54 is special in that it is built with mrustc, which shortens the
 ;;; bootstrap path.
@@ -297,6 +298,8 @@  (define rust-bootstrap
                ;; to set the level of parallelism.
                (setenv "PARLEVEL" (number->string job-count))
                (setenv "CARGO_BUILD_JOBS" (number->string job-count))
+               ;; reduce memory requirements during the build
+               (setenv "CARGO_PROFILE_DEV_DEBUG" "false")
                (display "Building mrustc...\n")
                (apply invoke "make" make-flags)
 
@@ -453,6 +456,8 @@  (define rust-1.55
          (replace 'build
            ;; The standard library source location moved in this release.
            (lambda* (#:key parallel-build? #:allow-other-keys)
+             ;; reduce memory requirements during the build
+             (setenv "CARGO_PROFILE_DEV_DEBUG" "false")
              (let ((job-spec (string-append
                               "-j" (if parallel-build?
                                        (number->string (parallel-job-count))
-- 
2.37.1