[bug#55998] gnu: Add cctools.

* gnu/packages/darwin.scm: New file.
* gnu/local.mk (GNU_SYSTEM_MODULES): Add it.
* gnu/packages/darwin.scm (cctools): New variable.
---

These tools are used, for example, by Nix [1] and conda-forge [2].

I've used only one small part of this package so far: in [3], I use
`install_name_tool` somewhat like `patchelf` in the process of building the
libgit2 shared library using Guix for distribution as a Racket package. (I
plan to write up for the mailing list what worked well with that approach and
what maybe could work better.)

 -Philip

[1]: https://github.com/NixOS/nixpkgs/blob/master/pkgs/os-specific/darwin/cctools/port.nix
[2]: https://conda-forge.org/blog/posts/2020-10-29-macos-arm64/
[3]: https://github.com/LiberalArtist/native-libgit2-pkgs/tree/build-scripts

 gnu/local.mk            |  1 +
 gnu/packages/darwin.scm | 85 +++++++++++++++++++++++++++++++++++++++++
 2 files changed, 86 insertions(+)
 create mode 100644 gnu/packages/darwin.scm


base-commit: 8a04ac4b2f5d356719d896536dabc95a9520c938

Philip McGrath schreef op wo 15-06-2022 om 13:15 [-0400]:
> I've used only one small part of this package so far: in [3],
> 

Nitpick: in libgit2-for-racket.scm:

                ;; it could be provenance.json, but neither
                ;; Racket nor Guix has a pretty-printer

If you use the guile-json library, you can pass ‘#:pretty #true’ to
pretty-print the json.

Greetings,
Maxime.

Philip McGrath schreef op wo 15-06-2022 om 13:15 [-0400]:
> +      (license license:apsl2))))

Seems to have a problematic cause:

‘13.6 Dispute Resolution. Any litigation or other dispute resolution
between You and Apple relating to this License shall take place in the
Northern District of California, and You and Apple hereby consent to
the personal jurisdiction of, and venue in, the state and federal
courts within that District with respect to this License. The
application of the United Nations Convention on Contracts for the
International Sale of Goods is expressly excluded.’

To me it seems a bit much to require of users _outside_ the US to have
to subject theirselves to the US whenever Apple feels like litigating.

(Is such an unilateral requirement even legal?)

Greetings,
Maxime.

Philip McGrath schreef op wo 15-06-2022 om 13:15 [-0400]:
> +      (synopsis "Darwin's @code{cctools} and @code{ld64}")
> +      ;; Confusingly enough, the program is called ld64, but the command is
> +      ;; just ld (with no symlink), so @command{ld64} would be wrong.
> +      (description
> +       "Darwin's @code{cctools} are a set of tools somewhat similar in purpose
> +to GNU Binutils, but for Mach-O files targeting Darwin.  The suite includes
> +@command{install_name_tool}, @command{libtool}, and other specialized tools in
> +addition to standard utilities like @command{ld} and @command{as}.  This
> +package provides portable versions of the tools.")
> +      (license license:apsl2))))

How can this work? We don't have any (cross-compiled) Darwin libc
libraries to let it link against. Is this a draft patch?

Greetings,
Maxime.

Philip McGrath schreef op wo 15-06-2022 om 13:15 [-0400]:
> +                ;; FIXME: This is a very ugly way to make
> +                ;;     #include <linux/limits.h>
> +                ;; work---what is a better way?


Searching for <linux/limit.h> in the guix git checkout, I found:

   ;; Glibc's <limits.h> refers to <linux/limit.h>, for instance, so glibc
   ;; users should automatically pull Linux headers as well.  On GNU/Hurd,
   ;; libc provides <hurd.h>, which includes a bunch of Hurd and Mach headers,
   ;; so both should be propagated.
   (propagated-inputs
    (if (hurd-target?)
        `(("hurd-core-headers" ,hurd-core-headers))
        `(("kernel-headers" ,linux-libre-headers))))

(The hurd bit is not relevant here, and in this case I'd assume that no
propagation is required.)

Greetings,
Maxime.

Philip McGrath schreef op wo 15-06-2022 om 13:15 [-0400]:
> +      (native-inputs (list clang-toolchain))

Why?  Would the standard GCC compiler suffice?

Greetings,
Maxime.

On 6/15/22 14:45, Maxime Devos wrote:
> Philip McGrath schreef op wo 15-06-2022 om 13:15 [-0400]:
>> +      (license license:apsl2))))
> 
> Seems to have a problematic cause:
> 
> ‘13.6 Dispute Resolution. Any litigation or other dispute resolution
> between You and Apple relating to this License shall take place in the
> Northern District of California, and You and Apple hereby consent to
> the personal jurisdiction of, and venue in, the state and federal
> courts within that District with respect to this License. The
> application of the United Nations Convention on Contracts for the
> International Sale of Goods is expressly excluded.’
> 
> To me it seems a bit much to require of users _outside_ the US to have
> to subject theirselves to the US whenever Apple feels like litigating.
> 
> (Is such an unilateral requirement even legal?)
> 

I agree that the choice-of-law language is less than friendly to users.

The FSF has issued an opinion [1] that the APSL 2.0 is a free software 
license: they say that "Apple's lawyers worked with the FSF to produce a 
license that would qualify" (after problems with earlier versions of the 
license). They "recommend you do not release new software using this 
license; but it is ok to use and improve software which other people 
release under this license."

IIUC, (guix licenses) only defines FSDG-compatible licenses.

Certainly there are broader community governance questions implicated, 
but I don't think this patch needs to resolve them.

-Philip

[1]: https://www.gnu.org/philosophy/apsl.html

On 6/15/22 14:53, Maxime Devos wrote:
> Philip McGrath schreef op wo 15-06-2022 om 13:15 [-0400]:
>> +      (synopsis "Darwin's @code{cctools} and @code{ld64}")
>> +      ;; Confusingly enough, the program is called ld64, but the command is
>> +      ;; just ld (with no symlink), so @command{ld64} would be wrong.
>> +      (description
>> +       "Darwin's @code{cctools} are a set of tools somewhat similar in purpose
>> +to GNU Binutils, but for Mach-O files targeting Darwin.  The suite includes
>> +@command{install_name_tool}, @command{libtool}, and other specialized tools in
>> +addition to standard utilities like @command{ld} and @command{as}.  This
>> +package provides portable versions of the tools.")
>> +      (license license:apsl2))))
> 
> How can this work? We don't have any (cross-compiled) Darwin libc
> libraries to let it link against. Is this a draft patch?
> 

Fortunately, we don't need a Darwin libc: tools like `install_name_tool` 
run on GNU/Linux, but work with Darwin binaries, somewhat like 
`patchelf` can work on a cross-compiled binary. From another point of 
view, it's a bit like some parts of MinGW.

This patch is not enough for a Darwin cross-compilation toolchain, 
though I believe it would play a role analogous to GNU Binutils in such 
a toolchain.

Still, several of the tools are useful (albeit niche) on their own, 
which is why I sent this patch now. A whole group of tools supports 
inspecting Mach-O binaries, for example.

-Philip

On 6/15/22 14:56, Maxime Devos wrote:
> Philip McGrath schreef op wo 15-06-2022 om 13:15 [-0400]:
>> +      (native-inputs (list clang-toolchain))
> 
> Why?  Would the standard GCC compiler suffice?
> 

Unfortunately, these tools are tightly coupled to Clang/LLVM and don't 
support GCC. For example, `otool` is a wrapper around `llvm-objdump`.

-Philip

Philip McGrath schreef op wo 15-06-2022 om 15:21 [-0400]:
> Unfortunately, these tools are tightly coupled to Clang/LLVM and don't 
> support GCC. For example, `otool` is a wrapper around `llvm-objdump`.

In that case, it needs to be in `inputs`, not `native-inputs`, such
that cross-compiling this cross-compiler can work.

FWIW, you could compile the wrapper `otool` with GCC and at the same
time let `otool` use `llvm-objdump`.

Greetings,
Maxime.

On 6/15/22 14:55, Maxime Devos wrote:
> Philip McGrath schreef op wo 15-06-2022 om 13:15 [-0400]:
>> +                ;; FIXME: This is a very ugly way to make
>> +                ;;     #include <linux/limits.h>
>> +                ;; work---what is a better way?
> 
> 
> Searching for <linux/limit.h> in the guix git checkout, I found:
> 
>     ;; Glibc's <limits.h> refers to <linux/limit.h>, for instance, so glibc
>     ;; users should automatically pull Linux headers as well.  On GNU/Hurd,
>     ;; libc provides <hurd.h>, which includes a bunch of Hurd and Mach headers,
>     ;; so both should be propagated.
>     (propagated-inputs
>      (if (hurd-target?)
>          `(("hurd-core-headers" ,hurd-core-headers))
>          `(("kernel-headers" ,linux-libre-headers))))
> 
> (The hurd bit is not relevant here, and in this case I'd assume that no
> propagation is required.)
> 

I saw that, too, and I don't understand why <linux/limits.h> (it's 
plural) isn't found automatically, especially when the neighboring 
<limits.h> is found: with both clang-toolchain and gcc-toolchain, 
include/linux is a symlink into linux-libre-headers.

-Philip

Philip McGrath schreef op wo 15-06-2022 om 15:06 [-0400]:
> I agree that the choice-of-law language is less than friendly to users.
> 
> The FSF has issued an opinion [1] that the APSL 2.0 is a free software 
> license: they say that "Apple's lawyers worked with the FSF to produce a 
> license that would qualify" (after problems with earlier versions of the 
> license)

I am not contesting that FSF considers APSL 2.0 to be a free software
license.  In fact, I looked at that web page to look at why FSF
considers it to be a free software license.  But I didn't find any
answer about the ‘dispute resolution’ clause.  So it seems to me that
FSF overlooked that particular issue, considered it acceptable because
of the US being based in the US, or considered it acceptable due to
some other (unknown) reason.

In case of the FSF overlooking things: mistakes can and should be
corrected (this is a free software distro!).  In case of US-centrism:
err, no.  In case of an unknwon reason: reason is unknown.

The point is being free, not being stamped as free by the FSF.

> IIUC, (guix licenses) only defines FSDG-compatible licenses.

Apparently, it doesn't, given the presence of the APSL 2.0, though
that's a bug.

> Certainly there are broader community governance questions
> implicated, but I don't think this patch needs to resolve them.

I did not ask anything about community governance?

Greetings,
Maxime.

Philip McGrath schreef op wo 15-06-2022 om 13:15 [-0400]:
> +      (source
> +       (origin
> +         (method git-fetch)
> +         (uri (git-reference
> +               (url "https://github.com/tpoechtrager/cctools-port")
> +               (commit commit)))
> +         (sha256

This contains generated files (look for 'configure' and 'Makefile.in'.
Per standard Guix policy, things need to be built from source.  The
autotools are no exception, see
<https://www.mail-archive.com/guix-devel@gnu.org/msg61160.html>.
They need to be removed, e.g. with delete-file and find-files in a
snippet.

Greetings,
Maxime.

Philip McGrath schreef op wo 15-06-2022 om 15:19 [-0400]:
> Still, several of the tools are useful (albeit niche) on their own, 
> which is why I sent this patch now. A whole group of tools supports 
> inspecting Mach-O binaries, for example.

Ok, but the package description is implying a linker is available,
which is not the case, due to the lack of a Darwin libc in Guix.

Greetings,
Maxime.

Philip McGrath schreef op wo 15-06-2022 om 13:15 [-0400]:
> +      (license license:apsl2))))

It's also BSD-4:

https://github.com/tpoechtrager/cctools-port/blob/04663295d0425abfac90a42440a7ec02d7155fea/cctools/gprof/gprof.c#L27

It's also bundling GNUstep code:

https://github.com/tpoechtrager/cctools-port/tree/master/cctools/libobjc2

Greetings,
Maxime.

Philip McGrath schreef op wo 15-06-2022 om 13:15 [-0400]:
> +      (license license:apsl2))))

There's some Expat licensed code as well:

https://github.com/tpoechtrager/cctools-port/tree/master/cctools/libobjc2

Please investigate the rest as well.

Greetings,
Maxime.

Philip McGrath schreef op wo 15-06-2022 om 13:15 [-0400]:
> +      (license license:apsl2))))


There's also the GDB General public license:

https://github.com/tpoechtrager/cctools-port/blob/master/cctools/include/gnu/symseg.h

and BSD-3:

https://github.com/tpoechtrager/cctools-port/blob/master/cctools/include/xar/xar.h

and additional term:

 * This file contains Original Code and/or Modifications of Original
Code
 * as defined in and that are subject to the Apple Public Source
License
 * Version 2.0 (the 'License'). You may not use this file except in
 * compliance with the License. The rights granted to you under the
License
 * may not be used to create, or enable the creation or redistribution
of,
 * unlawful or unlicensed copies of an Apple operating system, or to
 * circumvent, violate, or enable the circumvention or violation of,
any
 * terms of an Apple operating system software license agreement.

(in partiular, a reference to an unspecified ‘Apple operating system software
license agreement’)

Greetings,
Maxime.

On 6/15/22 15:35, Maxime Devos wrote:
> Philip McGrath schreef op wo 15-06-2022 om 15:06 [-0400]:
>> I agree that the choice-of-law language is less than friendly to users.
>>
>> The FSF has issued an opinion [1] that the APSL 2.0 is a free software
>> license: they say that "Apple's lawyers worked with the FSF to produce a
>> license that would qualify" (after problems with earlier versions of the
>> license)
> 
> I am not contesting that FSF considers APSL 2.0 to be a free software
> license.  In fact, I looked at that web page to look at why FSF
> considers it to be a free software license.  But I didn't find any
> answer about the ‘dispute resolution’ clause.  So it seems to me that
> FSF overlooked that particular issue, considered it acceptable because
> of the US being based in the US, or considered it acceptable due to
> some other (unknown) reason.
> 
> In case of the FSF overlooking things: mistakes can and should be
> corrected (this is a free software distro!).  In case of US-centrism:
> err, no.  In case of an unknwon reason: reason is unknown.
> 

According to 
<https://www.gnu.org/philosophy/free-sw.html#legal-details>, "It is 
acceptable for a free license to specify which jurisdiction's law 
applies, or where litigation must be done, or both."

That paragraph was apparently added in version 1.129, in 2012, but the 
note says that "this was always our policy": 
<https://web.cvs.savannah.gnu.org/viewvc/www/www/philosophy/free-sw.html?r1=1.128&r2=1.129>

So it is not a matter of something being overlooked. Some other FSF-free 
licenses include similar provisions, which generally seem to make the 
license in question not GPL-compatible. For example: 
<https://directory.fsf.org/wiki/License:YPL-1.1>.

> The point is being free, not being stamped as free by the FSF.
> 
>> IIUC, (guix licenses) only defines FSDG-compatible licenses.
> 
> Apparently, it doesn't, given the presence of the APSL 2.0, though
> that's a bug.
> 
>> Certainly there are broader community governance questions
>> implicated, but I don't think this patch needs to resolve them.
> 
> I did not ask anything about community governance?
> 

I meant "community governance" broadly to include questions like, "Who 
decides what 'free' means?" Since I basically agree with statements like 
<https://guix.gnu.org/blog/2019/joint-statement-on-the-gnu-project/>, I 
think there are troubling questions about the FSF's role and how such 
decisions ought to be made in the future. Still, IIUC Guix's current 
policy is 
<https://www.gnu.org/distros/free-system-distribution-guidelines.html>, 
which links to <https://www.gnu.org/philosophy/free-sw.html> for its 
definition of "free license". Bugs are one thing, but this seems to be 
an explicitly allowed under the existing policy, and I don't think this 
patch is the right place to debate substantive changes to Guix's policy.

-Philip

Philip McGrath schreef op wo 15-06-2022 om 17:00 [-0400]:
> According to 
> <https://www.gnu.org/philosophy/free-sw.html#legal-details>, "It is 
> acceptable for a free license to specify which jurisdiction's law 
> applies, or where litigation must be done, or both."

What

> an explicitly allowed under the existing policy, and I don't think
> this patch is the right place to debate substantive changes to Guix's
> policy.

Maybe move it to guix-devel, and depending on the conclusion, continue
with this patch / drop it?

> I meant "community governance" broadly to include questions like,
> "Who decides what 'free' means?"

Ok, makes sense to me.

Greetings,
Maxime.

On Wed Jun 15, 2022 at 10:00 PM BST, Philip McGrath wrote:
> According to 
> <https://www.gnu.org/philosophy/free-sw.html#legal-details>, "It is 
> acceptable for a free license to specify which jurisdiction's law 
> applies, or where litigation must be done, or both."

Although I don't know anything about licensing, wouldn't this mean a
company could, for example, publish software under a license that states
that disputes must be resolved in some theoretical country that bans all
free sharing of software (even if the owner wants to share it), then sue
somebody who's modified the ostensibly free software for copyright
infringement, winning because the country forbids any software sharing,
and still have the license count as free?

I'm probably misunderstanding this; surely there wouldn't be such a
gaping hole in the FSF's free license definition?

On Wednesday, June 15, 2022 4:17:07 PM EDT Maxime Devos wrote:
> Philip McGrath schreef op wo 15-06-2022 om 13:15 [-0400]:
> > +      (license license:apsl2))))
> 
> It's also BSD-4:
> 
> https://github.com/tpoechtrager/cctools-port/blob/04663295d0425abfac90a42440
> a7ec02d7155fea/cctools/gprof/gprof.c#L27
> 

My understanding is that Guix's practice is to list the overall license or 
licenses of a package, not every permissive license that might apply to 
particular files.

While that file uses the original license header, it's effectively BSD-3-Clause, 
because the University of California retroactively deleted the advertising 
clause in 1999: see <https://www.gnu.org/licenses/bsd.html> and <ftp://
ftp.cs.berkeley.edu/pub/4bsd/README.Impt.License.Change>.

> It's also bundling GNUstep code:
> 
> https://github.com/tpoechtrager/cctools-port/tree/master/cctools/libobjc2
> 

libobjc2 is Expat-licensed—I will look into whether it can be unbundled, but 
it's not a license issue.

-Philip

On Wednesday, June 15, 2022 4:23:27 PM EDT Maxime Devos wrote:
> Philip McGrath schreef op wo 15-06-2022 om 13:15 [-0400]:
> > +      (license license:apsl2))))
> 
> There's also the GDB General public license:
> 
> https://github.com/tpoechtrager/cctools-port/blob/master/cctools/include/gnu
> /symseg.h
> 

This file is dead code: I will delete it in a snippet and send a patch 
upstream.

> and BSD-3:
> 
> https://github.com/tpoechtrager/cctools-port/blob/master/cctools/include/xar
> /xar.h
> 
> and additional term:
> 
>  * This file contains Original Code and/or Modifications of Original
> Code
>  * as defined in and that are subject to the Apple Public Source
> License
>  * Version 2.0 (the 'License'). You may not use this file except in
>  * compliance with the License. The rights granted to you under the
> License
>  * may not be used to create, or enable the creation or redistribution
> of,
>  * unlawful or unlicensed copies of an Apple operating system, or to
>  * circumvent, violate, or enable the circumvention or violation of,
> any
>  * terms of an Apple operating system software license agreement.
> 
> (in partiular, a reference to an unspecified ‘Apple operating system
> software license agreement’)
> 

I'm not a lawyer, but I read that language as, "The rights granted to you 
under the License may not be used to [do things that are unlawful]." Having  
rights under one license to some program does not make it legal for you to 
violate a completely different license that applies to some other program.

I'd suggest discussing any remaining license issues at <https://lists.gnu.org/
archive/html/guix-devel/2022-06/msg00235.html>, though.

-Philip

Philip McGrath schreef op do 16-06-2022 om 18:29 [-0400]:
> My understanding is that Guix's practice is to list the overall
> license or licenses of a package, not every permissive license that
> might apply to particular files.

Apparently not fully consistent between packages and packagers?
What I tend to do and recommend, is list all _relevant_ licenses,
without doing an interpretation of what would be the overall license.

FWIW, there was past discussion:

* https://lists.gnu.org/archive/html/guix-devel/2021-05/msg00183.html
* https://lists.gnu.org/archive/html/guix-devel/2021-05/msg00186.html

that was somewhere in between the two endpoints.

Greetings,
Maxime.

On Wednesday, June 15, 2022 4:04:21 PM EDT Maxime Devos wrote:
> Philip McGrath schreef op wo 15-06-2022 om 13:15 [-0400]:
> > +      (source
> > +       (origin
> > +         (method git-fetch)
> > +         (uri (git-reference
> > +               (url "https://github.com/tpoechtrager/cctools-port")
> > +               (commit commit)))
> > +         (sha256
> 
> This contains generated files (look for 'configure' and 'Makefile.in'.
> Per standard Guix policy, things need to be built from source.  The
> autotools are no exception, see
> <https://www.mail-archive.com/guix-devel@gnu.org/msg61160.html>.
> They need to be removed, e.g. with delete-file and find-files in a
> snippet.
> 

Doing this had the unexpected side-effect of fixing the problem finding <linux/
limits.h>!

A v2 is coming soon.

-Philip

On Wednesday, June 15, 2022 3:23:48 PM EDT Maxime Devos wrote:
> Philip McGrath schreef op wo 15-06-2022 om 15:21 [-0400]:
> > Unfortunately, these tools are tightly coupled to Clang/LLVM and don't
> > support GCC. For example, `otool` is a wrapper around `llvm-objdump`.
> 
> In that case, it needs to be in `inputs`, not `native-inputs`, such
> that cross-compiling this cross-compiler can work.
> 
> FWIW, you could compile the wrapper `otool` with GCC and at the same
> time let `otool` use `llvm-objdump`.
> 

If that were the only issue, it might be possible, but this package just 
pervasively does not work with GCC. You will not even get past `./configure` 
without Clang.

Nix seemed to only have the LLVM toolchain as a native input, but I've added 
it to `inputs` as well.

-Philip

On Wednesday, June 15, 2022 4:07:47 PM EDT Maxime Devos wrote:
> Philip McGrath schreef op wo 15-06-2022 om 15:19 [-0400]:
> > Still, several of the tools are useful (albeit niche) on their own,
> > which is why I sent this patch now. A whole group of tools supports
> > inspecting Mach-O binaries, for example.
> 
> Ok, but the package description is implying a linker is available,
> which is not the case, due to the lack of a Darwin libc in Guix.
> 

Concretely, there is an `ld`, and it runs at least enough to print out a usage 
note.

I can't say for certain *why* it works, though the way LLVM doesn't need a 
different binary for each target and the somewhat unusual way linking to libc 
and other system libraries works on Darwin. Though does a linker really need a 
libc? In principle, it could link things that don't link to libc, or even link 
libc itself.

I haven't tried actually linking anything with this package's `ld`, but, even 
if it turns out to not be fully functional yet, this is the package that 
contains Darwin's `ld`.

I did tweak the description in v2 to replace the reference to `libtool`, 
because Darwin's `libtool` is something completely unrelated than the program 
in the Guix package called `libtool`.

-Philip

Hi,

Maxime Devos <maximedevos@telenet.be> skribis:

> Philip McGrath schreef op wo 15-06-2022 om 13:15 [-0400]:
>> +      (source
>> +       (origin
>> +         (method git-fetch)
>> +         (uri (git-reference
>> +               (url "https://github.com/tpoechtrager/cctools-port")
>> +               (commit commit)))
>> +         (sha256
>
> This contains generated files (look for 'configure' and 'Makefile.in'.
> Per standard Guix policy, things need to be built from source.  The
> autotools are no exception, see
> <https://www.mail-archive.com/guix-devel@gnu.org/msg61160.html>.
> They need to be removed, e.g. with delete-file and find-files in a
> snippet.

To be clear, I think this is best described as “Guix policy-to-be”—we
have yet to write this down and actually implement it consistently.

Ludo’.