mbox

[bug#41871,0/7] IcedTea: Build (efficiently) on aarch64-linux

Message ID 20200615151856.12283-1-simon@simonsouth.net
Headers show

Message

Simon South June 15, 2020, 3:18 p.m. UTC 
This patch series applies a number of changes to the icedtea-6, -7 and -8
packages that allow them to build (efficiently) on aarch64-linux systems.

It assumes the patches in issues 41748 and 41648 have already been applied, in
that order.

With these three sets of patches I've been able to complete the entire Java
bootstrap process on both AArch64 and x86_64 and have successfully compiled
and run a Java application on AArch64 using OpenJDK 14.

The changes in this series

- Ensure the correct number of parallel build jobs is used. The
  "--with-parallel-jobs" option to IcedTea's configure script enables a
  parallel build but by default, the script uses its own heuristic to decide
  how many jobs to run simultaneously. This can produce a poor result: On my
  dual-core, hyperthreaded x86_64 machine, IcedTea 6 picks five jobs (anything
  above two is inefficient); on my six-core AArch64 machine, IcedTea 7 picks
  only two (on a machine where engaging every core is important!). In any
  case, there is no guarantee the figure the script chooses will match the
  number of jobs requested by the user.

  These patches address this by ensuring the "--with-parallel-jobs" parameter
  is passed to configure along with a parameter explicitly specifying the
  correct number of jobs to use.

- Remove an obsolete and architecture-dependent patch. The "gcc-segfault"
  patches yield a broken JIT on AArch64 because they embed a constant value,
  11, that is architecture-dependent and apparently valid only for
  x86_64. This number represents the length of an array whose size is
  determined by the HotSpot build process and output in code it generates (at
  src/share/vm/adlc/output_h.cpp:893).

  Presumably the patches could be fixed, but I've been unable to reproduce the
  problem they're meant to solve and I suspect it was only ever an issue with
  gcc 5, meaning the patches are now obsolete. (The move to gcc 7 occurred
  about three months after they were added.)

  My changes remove these patches, including (for completeness and
  consistency) the patch to IcedTea 6 which doesn't actually interfere with
  the AArch64 build as it doesn't produce a JIT for that platform.

- Allow all three IcedTea packages to build on aarch64-linux. For IcedTea 6,
  this means applying a backport of a patch to JDK 9 (see
  http://openjdk.java.net/jeps/237) that extends the support for AArch64 in
  HotSpot's shared code and allows the portable Zero VM to be built. For
  IcedTea 7, it means removing an unneeded C++ template that causes the build
  to fail when using gcc 7 and its default support for only the C++98
  standard.

  IcedTea 8 (and subsequent versions of OpenJDK) support AArch64 and gcc 7
  out-of-the-box and require no specific changes.

--
Simon South
simon@simonsouth.net


Simon South (7):
  gnu: icedtea-6: Build in parallel using correct number of jobs.
  gnu: icedtea-6: Remove obsolete, architecture-dependent patch.
  gnu: icedtea-6: Fix build on aarch64-linux.
  gnu: icedtea-7: Build in parallel using correct number of jobs.
  gnu: icedtea-7: Fix build on aarch64-linux.
  gnu: icedtea-8: Build in parallel using correct number of jobs.
  gnu: icedtea-8: Fix build on aarch64-linux.

 gnu/local.mk                                  |    4 +-
 gnu/packages/java.scm                         |   32 +-
 ...tea-6-extend-hotspot-aarch64-support.patch | 1831 +++++++++++++++++
 ...ea-6-hotspot-gcc-segfault-workaround.patch |   42 -
 .../icedtea-7-hotspot-aarch64-use-c++98.patch |   33 +
 ...ea-7-hotspot-gcc-segfault-workaround.patch |   45 -
 6 files changed, 1879 insertions(+), 108 deletions(-)
 create mode 100644 gnu/packages/patches/icedtea-6-extend-hotspot-aarch64-support.patch
 delete mode 100644 gnu/packages/patches/icedtea-6-hotspot-gcc-segfault-workaround.patch
 create mode 100644 gnu/packages/patches/icedtea-7-hotspot-aarch64-use-c++98.patch
 delete mode 100644 gnu/packages/patches/icedtea-7-hotspot-gcc-segfault-workaround.patch

Comments

Efraim Flashner Oct. 28, 2020, 8:51 a.m. UTC | #1 
On Mon, Sep 14, 2020 at 03:24:15PM +0200, Ricardo Wurmus wrote:
> 
> Hi Simon,
> 
> my apologies for the very long delay!
> 
> > This patch series applies a number of changes to the icedtea-6, -7 and -8
> > packages that allow them to build (efficiently) on aarch64-linux systems.
> >
> > It assumes the patches in issues 41748 and 41648 have already been applied, in
> > that order.
> >
> > With these three sets of patches I've been able to complete the entire Java
> > bootstrap process on both AArch64 and x86_64 and have successfully compiled
> > and run a Java application on AArch64 using OpenJDK 14.
> 
> This looks really great!  I have yet to read the patches in 41748 and
> 41648, but these patches here look good to me.
> 
> Thank you very much for working on this!
> 
> -- 
> Ricardo
> 

I had this on my TODO list for a while and I seem to have forgotten
about it. Since at least one of the patches was applied to staging I'm
testing them on staging now. Assuming no problems on x86_64 and aarch64
I'll go ahead and push them.
Simon South Oct. 28, 2020, 2:35 p.m. UTC | #2 
Efraim Flashner <efraim@flashner.co.il> writes:
> On Mon, Sep 14, 2020 at 03:24:15PM +0200, Ricardo Wurmus wrote:
>> This looks really great!  I have yet to read the patches in 41748 and
>> 41648, but these patches here look good to me...
>
> I had this on my TODO list for a while and I seem to have forgotten
> about it. Since at least one of the patches was applied to staging I'm
> testing them on staging now. Assuming no problems on x86_64 and aarch64
> I'll go ahead and push them.

Wonderful. Thank you both for looking at these changes.

If they're able to be applied in time, it'll be possible to list
"Support for Java applications on aarch64-linux" among the new features
in Guix 1.2.0.
Efraim Flashner Oct. 29, 2020, 1:15 p.m. UTC | #3 
On Wed, Oct 28, 2020 at 10:35:08AM -0400, Simon South wrote:
> Efraim Flashner <efraim@flashner.co.il> writes:
> > On Mon, Sep 14, 2020 at 03:24:15PM +0200, Ricardo Wurmus wrote:
> >> This looks really great!  I have yet to read the patches in 41748 and
> >> 41648, but these patches here look good to me...
> >
> > I had this on my TODO list for a while and I seem to have forgotten
> > about it. Since at least one of the patches was applied to staging I'm
> > testing them on staging now. Assuming no problems on x86_64 and aarch64
> > I'll go ahead and push them.
> 
> Wonderful. Thank you both for looking at these changes.
> 
> If they're able to be applied in time, it'll be possible to list
> "Support for Java applications on aarch64-linux" among the new features
> in Guix 1.2.0.
> 

All three versions of icedtea built with no problems on x86_64.
icedtea-2 failed the first time through on aarch64. I'm sending it
through again.

<..snip..>
#   org.w3c.dom.events
#   org.w3c.dom.bootstrap
#   org.w3c.dom.ls
#   org.xml.sax
#   org.xml.sax.ext
#   org.xml.sax.helpers
/tmp/guix-build-icedtea-2.6.13.drv-0/icedtea-2.6.13/bootstrap/jdk1.6.0/bin/java -XX:-PrintVMOptions -XX:+UnlockDiagnosticVMOptions -XX:-LogVMOutput -Xmx1536m -
Xms512m -XX:PermSize=32m -XX:MaxPermSize=160m "-Xbootclasspath/p:/tmp/guix-build-icedtea-2.6.13.drv-0/icedtea-2.6.13/openjdk.build/langtools/dist/bootstrap/lib/javadoc.jar:/tmp/guix-build-icedtea-2.6.13.drv-0/icedtea-2.6.13/openjdk.build/langtools/dist/bootstrap/lib/javac.jar:/tmp/guix-build-icedtea-2.6.13.drv-0/icedtea-2.6.13/openjdk.build/langtools/dist/bootstrap/lib/doclets.jar" -jar /tmp/guix-build-icedtea-2.6.13.drv-0/icedtea-2.6.13/openjdk.build/langtools/dist/bootstrap/lib/javadoc.jar -bootclasspath "/tmp/guix-build-icedtea-2.6.13.drv-0/icedtea-2.6.13/openjdk.build/classes"  -d /tmp/guix-build-icedtea-2.6.13.drv-0/icedtea-2.6.13/openjdk.build/docs/api \
  @/tmp/guix-build-icedtea-2.6.13.drv-0/icedtea-2.6.13/openjdk.build/tmp/docs/doctmp/coredocs.options @/tmp/guix-build-icedtea-2.6.13.drv-0/icedtea-2.6.13/openjdk.build/tmp/docs/doctmp/coredocs.packages
../../src/share/classes/java/awt/color/ICC_Profile.java:1073: warning - Tag @see: missing '#': "activateDeferredProfile()"
../../src/share/classes/java/text/NumberFormat.java:305: warning - @param argument "toAppendTo" is not a parameter name.
../../src/share/classes/java/text/NumberFormat.java:305: warning - @param argument "pos" is not a parameter name.
../../src/share/classes/java/util/Calendar.java:1720: warning - Tag @see: can't find setInternallySetState(int) in java.util.Calendar
../../src/share/classes/java/util/Currency.java:700: warning - @throws tag has no arguments.
../../src/share/classes/java/util/concurrent/ArrayBlockingQueue.java:867: warning - Tag @see: can't find floor(double) in java.util.concurrent.ArrayBlockingQueue
../../src/share/classes/javax/swing/plaf/nimbus/NimbusStyle.java:854: warning - @return tag has no arguments.
../../src/share/classes/javax/swing/plaf/nimbus/NimbusStyle.java:926: warning - @return tag has no arguments.
/tmp/guix-build-icedtea-2.6.13.drv-0/icedtea-2.6.13/openjdk.build/impsrc/javax/xml/bind/JAXBContext.java:262: warning - Tag @see: reference not found: S 7.4.1 "Named Packages" in Java Language Specification</a>
make[4]: *** [Makefile:325: /tmp/guix-build-icedtea-2.6.13.drv-0/icedtea-2.6.13/openjdk.build/docs/api/index.html] Error 143
make[4]: Leaving directory '/tmp/guix-build-icedtea-2.6.13.drv-0/icedtea-2.6.13/openjdk/jdk/make/docs'
make[3]: *** [Makefile:263: docs] Error 1
make[3]: Leaving directory '/tmp/guix-build-icedtea-2.6.13.drv-0/icedtea-2.6.13/openjdk/jdk/make'
make[2]: *** [make/jdk-rules.gmk:93: jdk-build] Error 2
make[2]: Leaving directory '/tmp/guix-build-icedtea-2.6.13.drv-0/icedtea-2.6.13/openjdk'
make[1]: *** [Makefile:251: build_product_image] Error 2
make[1]: Leaving directory '/tmp/guix-build-icedtea-2.6.13.drv-0/icedtea-2.6.13/openjdk'
make: *** [Makefile:2463: stamps/icedtea.stamp] Error 2
command "make" "-j" "2" failed with status 2
builder for `/gnu/store/n9p75jv705bdf0983xrbw5142fw1fp7z-icedtea-2.6.13.drv' failed with exit code 1
@ build-failed /gnu/store/n9p75jv705bdf0983xrbw5142fw1fp7z-icedtea-2.6.13.drv - 1 builder for `/gnu/store/n9p75jv705bdf0983xrbw5142fw1fp7z-icedtea-2.6.13.drv' failed with exit code 1
derivation '/gnu/store/n9p75jv705bdf0983xrbw5142fw1fp7z-icedtea-2.6.13.drv' offloaded to 'pine64' failed: build of `/gnu/store/n9p75jv705bdf0983xrbw5142fw1fp7z-icedtea-2.6.13.drv' failed
build of /gnu/store/n9p75jv705bdf0983xrbw5142fw1fp7z-icedtea-2.6.13.drv failed
View build log at '/var/log/guix/drvs/n9/p75jv705bdf0983xrbw5142fw1fp7z-icedtea-2.6.13.drv.bz2'.
cannot build derivation `/gnu/store/ci079q2b5max3j0vvia94q0ccbqxwh0q-icedtea-3.7.0.drv': 1 dependencies couldn't be built
guix build: error: build of `/gnu/store/ci079q2b5max3j0vvia94q0ccbqxwh0q-icedtea-3.7.0.drv' failed

real    1581m16.100s
user    2m9.646s
sys     0m16.733s
Efraim Flashner Nov. 2, 2020, 9:54 a.m. UTC | #4 
On Wed, Oct 28, 2020 at 10:35:08AM -0400, Simon South wrote:
> Efraim Flashner <efraim@flashner.co.il> writes:
> > On Mon, Sep 14, 2020 at 03:24:15PM +0200, Ricardo Wurmus wrote:
> >> This looks really great!  I have yet to read the patches in 41748 and
> >> 41648, but these patches here look good to me...
> >
> > I had this on my TODO list for a while and I seem to have forgotten
> > about it. Since at least one of the patches was applied to staging I'm
> > testing them on staging now. Assuming no problems on x86_64 and aarch64
> > I'll go ahead and push them.
> 
> Wonderful. Thank you both for looking at these changes.
> 
> If they're able to be applied in time, it'll be possible to list
> "Support for Java applications on aarch64-linux" among the new features
> in Guix 1.2.0.
> 

I'm still running into trouble building icedtea@2 on aarch64 on staging
so I've locally applied the patches on master and I'll test it out
there.


The error message, in case anyone wants to try to work at it
specifically:

/tmp/guix-build-icedtea-2.6.13.drv-0/icedtea-2.6.13/bootstrap/jdk1.6.0/bin/java -XX:-PrintVMOptions -XX:+UnlockDiagnosticVMOptions -XX:-LogVMOutput -Xmx512m -Xms512m -XX:PermSize=32m -XX:MaxPermSize=160m -Xbootclasspath/p:/tmp/guix-build-icedtea-2.6.13.drv-0/icedtea-2.6.13/openjdk.build-boot/langtools/dist/bootstrap/lib/javac.jar -jar /tmp/guix-build-icedtea-2.6.13.drv-0/icedtea-2.6.13/openjdk.build-boot/langtools/dist/bootstrap/lib/javac.jar -g -Xlint:-path -source 6 -target 6 -encoding ascii -Xbootclasspath:/tmp/guix-build-icedtea-2.6.13.drv-0/icedtea-2.6.13/openjdk.build-boot/classes -sourcepath /tmp/guix-build-icedtea-2.6.13.drv-0/icedtea-2.6.13/generated.build:../../../src/solaris/classes:../../../src/share/classes -d /tmp/guix-build-icedtea-2.6.13.drv-0/icedtea-2.6.13/openjdk.build-boot/classes @/tmp/guix-build-icedtea-2.6.13.drv-0/icedtea-2.6.13/openjdk.build-boot/tmp/java/java.lang/java/.classes.list.filtered
../../../src/share/classes/java/lang/invoke/MethodHandle.java:463: error: MethodHandle API building requires -target 7 runtimes or better; current is -target 1.6
    public final native @PolymorphicSignature Object invokeExact(Object... args) throws Throwable;
                                                     ^
../../../src/share/classes/java/lang/invoke/MethodHandle.java:498: error: MethodHandle API building requires -target 7 runtimes or better; current is -target 1.6
    public final native @PolymorphicSignature Object invoke(Object... args) throws Throwable;
                                                     ^
../../../src/share/classes/java/lang/invoke/MethodHandle.java:515: error: MethodHandle API building requires -target 7 runtimes or better; current is -target 1.6
    /*non-public*/ final native @PolymorphicSignature Object invokeBasic(Object... args) throws Throwable;
                                                             ^
../../../src/share/classes/java/lang/invoke/MethodHandle.java:517: error: MethodHandle API building requires -target 7 runtimes or better; current is -target 1.6
    /*non-public*/ static native @PolymorphicSignature Object linkToVirtual(Object... args) throws Throwable;
                                                              ^
../../../src/share/classes/java/lang/invoke/MethodHandle.java:524: error: MethodHandle API building requires -target 7 runtimes or better; current is -target 1.6
    /*non-public*/ static native @PolymorphicSignature Object linkToStatic(Object... args) throws Throwable;
                                                              ^
../../../src/share/classes/java/lang/invoke/MethodHandle.java:531: error: MethodHandle API building requires -target 7 runtimes or better; current is -target 1.6
    /*non-public*/ static native @PolymorphicSignature Object linkToSpecial(Object... args) throws Throwable;
                                                              ^
../../../src/share/classes/java/lang/invoke/MethodHandle.java:538: error: MethodHandle API building requires -target 7 runtimes or better; current is -target 1.6
    /*non-public*/ static native @PolymorphicSignature Object linkToInterface(Object... args) throws Throwable;
                                                              ^
Note: Some input files use or override a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
Note: Some input files use unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.
7 errors
make[5]: *** [/tmp/guix-build-icedtea-2.6.13.drv-0/icedtea-2.6.13/openjdk-boot/jdk/make/common/Rules.gmk:254: .compile.classlist] Error 1
make[5]: Leaving directory '/tmp/guix-build-icedtea-2.6.13.drv-0/icedtea-2.6.13/openjdk-boot/jdk/make/java/java'
make[4]: *** [Makefile:63: all] Error 1
make[4]: Leaving directory '/tmp/guix-build-icedtea-2.6.13.drv-0/icedtea-2.6.13/openjdk-boot/jdk/make/java'
make[3]: *** [Makefile:253: all] Error 1
make[3]: Leaving directory '/tmp/guix-build-icedtea-2.6.13.drv-0/icedtea-2.6.13/openjdk-boot/jdk/make'
make[2]: *** [make/jdk-rules.gmk:93: jdk-build] Error 2
make[2]: Leaving directory '/tmp/guix-build-icedtea-2.6.13.drv-0/icedtea-2.6.13/openjdk-boot'
make[1]: *** [Makefile:251: build_product_image] Error 2
make[1]: Leaving directory '/tmp/guix-build-icedtea-2.6.13.drv-0/icedtea-2.6.13/openjdk-boot'
make: *** [Makefile:2741: stamps/icedtea-boot.stamp] Error 2
command "make" "-j" "2" failed with status 2
builder for `/gnu/store/n9p75jv705bdf0983xrbw5142fw1fp7z-icedtea-2.6.13.drv' failed with exit code 1
@ build-failed /gnu/store/n9p75jv705bdf0983xrbw5142fw1fp7z-icedtea-2.6.13.drv - 1 builder for `/gnu/store/n9p75jv705bdf0983xrbw5142fw1fp7z-icedtea-2.6.13.drv' failed with exit code 1
derivation '/gnu/store/n9p75jv705bdf0983xrbw5142fw1fp7z-icedtea-2.6.13.drv' offloaded to 'pine64' failed: build of `/gnu/store/n9p75jv705bdf0983xrbw5142fw1fp7z-icedtea-2.6.13.drv' failed
build of /gnu/store/n9p75jv705bdf0983xrbw5142fw1fp7z-icedtea-2.6.13.drv failed
Simon South Nov. 2, 2020, 12:35 p.m. UTC | #5 
Efraim Flashner <efraim@flashner.co.il> writes:
> I'm still running into trouble building icedtea@2 on aarch64 on staging
> so I've locally applied the patches on master and I'll test it out
> there.

I'll look into this today as well.

I certainly did not see these build errors earlier so I wonder what
could have changed.
Simon South Nov. 3, 2020, 12:59 p.m. UTC | #6 
Efraim Flashner <efraim@flashner.co.il> writes:
> I'm still running into trouble building icedtea@2 on aarch64 on
> staging...

This completed successfully overnight on my ROCK64. Efraim, would you be
willing to share with me (off-list) your log file from the failed build
so I can try to find what's different?

My build was from the staging branch (commit 353bdae32f72) with my seven
commits added.
Efraim Flashner Nov. 3, 2020, 1:24 p.m. UTC | #7 
On Tue, Nov 03, 2020 at 07:59:02AM -0500, Simon South wrote:
> Efraim Flashner <efraim@flashner.co.il> writes:
> > I'm still running into trouble building icedtea@2 on aarch64 on
> > staging...
> 
> This completed successfully overnight on my ROCK64. Efraim, would you be
> willing to share with me (off-list) your log file from the failed build
> so I can try to find what's different?
> 
> My build was from the staging branch (commit 353bdae32f72) with my seven
> commits added.
> 

It turns out my log file was truncated during the unpack phase (I
probably started and canceled the build) so I'm rebuilding it. I've also
gone and cloned guix on my odroid-c2 to see if it works there on
staging. It's possible that my pine64 is acting up.
Simon South Nov. 3, 2020, 1:41 p.m. UTC | #8 
Efraim Flashner <efraim@flashner.co.il> writes:
> It's possible that my pine64 is acting up.

I meant to mention in my original email: My build was using a derivation
with a hash prefix different from yours, and my build log doesn't show
"MethodHandle.java" being built at all. So I suspect there may be
something different in your machine's environment.
Efraim Flashner Nov. 4, 2020, 10:58 a.m. UTC | #9 
On Tue, Nov 03, 2020 at 08:41:21AM -0500, Simon South wrote:
> Efraim Flashner <efraim@flashner.co.il> writes:
> > It's possible that my pine64 is acting up.
> 
> I meant to mention in my original email: My build was using a derivation
> with a hash prefix different from yours, and my build log doesn't show
> "MethodHandle.java" being built at all. So I suspect there may be
> something different in your machine's environment.
> 

Not sure what it is about my pine64, could be that I'm currently running
it without swap. In any event, all three versions of icedtea built just
fine on my odroid-c2 so I'm going to go ahead and push the commits to
staging.

I built icedtea@3 a second time, under 200 minutes with guix-daemon
limited to 2 cores.