mbox series

[bug#53765,00/17] Remove limitations on clojure-tools

Message ID 87y22r32hi.fsf@reilysiegel.com
Headers show
Series Remove limitations on clojure-tools | expand

Message

Reily Siegel Feb. 4, 2022, 12:22 a.m. UTC
Due to a lack of ability to package some of clojure-tools's
dependencies, certain functionality was disabled for its initial
packaging. Now that limitations in both the clojure-build-system and
limitations building some dependencies have been overcome, the full
functionality of the clojure-tools package can be restored.

Reily Siegel (17):
  gnu: Add clojure-data-priority-map.
  gnu: Add clojure-data-json.
  gnu: Add java-eclipse-jetty-client.
  gnu: Add clojure-core-cache.
  gnu: Add clojure-core-memoize.
  gnu: clojure-data-xml: Exclude failing tests.
  gnu: Add clojure-http-kit.
  gnu: Add clojure-tools-analyzer.
  gnu: Add clojure-tools-analyzer-jvm.
  gnu: Add clojure-tools-logging.
  gnu: Add clojure-core-async.
  gnu: Add clojure-com-cognitect-http-client.
  gnu: Add clojure-com-cognitect-aws-api.
  gnu: Add clojure-com-cognitect-aws-endpoints.
  gnu: Add clojure-com-cognitect-aws-s3.
  gnu: clojure-tools-deps-alpha: Fix issues with S3 transporter.
  gnu: clojure-tools: Fix logging warning messages.

 gnu/packages/clojure.scm | 497 ++++++++++++++++++++++++++++++++++++---
 gnu/packages/web.scm     |  26 ++
 2 files changed, 495 insertions(+), 28 deletions(-)

Comments

Reily Siegel April 15, 2022, 1:55 p.m. UTC | #1
Hello,

I just wanted to check in and make sure you saw the version 3 patches I
sent in, having resolved the licensing issue.
M April 15, 2022, 6:47 p.m. UTC | #2
Reily Siegel schreef op vr 15-04-2022 om 15:55 [+0200]:
> Hello,
> 
> I just wanted to check in and make sure you saw the version 3 patches I
> sent in, having resolved the licensing issue.

I've received them but I didn't have much time.  I haven't looked a the
new patches but it seems that the only potential issue remaining is ...

> It looks like these files are generated from JSON definitions files
> released as part of the official AWS javascript sdk, which are
> available here [1] under the ASL2.0.
>
> However, it looks like based on this issue on the aws-api github [2],
> the code that generates the EDN files from the JSON definitions is
> not open source, only the resulting output EDN files. 

... do these EDN output files count as source?  They are licensed as
ASL2.0, which is an open source license, but that does not necessarily
make it free software, as they could perhaps be considered binaries
instead of source.

Well, the (original?) source (the JSON files) seems to be free
software, but if there is no free compiler for the source (and worse,
there is only a single compiler and it can only be used by a single
entity (cognitect)), wouldn't that make the software, from a practical
perspective, non-free?

gnu.org words this in more detail at
<https://www.gnu.org/philosophy/when-free-depends-on-nonfree.html>.
FWIW, this seems a similar situation as Debian's ‘contrib’ archive.

Greetings,
Maxime.
M April 15, 2022, 7:03 p.m. UTC | #3
Maxime Devos schreef op vr 15-04-2022 om 20:47 [+0200]:
> [com.cognitect.aws.* things]

I found the following messages:

  <https://lists.debian.org/debian-clojure/2021/06/msg00013.html>
  <https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=891136#27>

Apparently the aws things aren't very useful and they can be patched
out?

Greetings,
Maxime
Reily Siegel April 16, 2022, 5:39 p.m. UTC | #4
Maxime Devos <maximedevos@telenet.be> writes:

> Apparently the aws things aren't very useful and they can be patched
> out?

That is exactly what I did with the first patch I sent for clojure-tools,
which was accepted. However, this causes the tools to print warnings on
startup, as well as removing the feature of accessing S3 repositories. I
suppose these warnings could be patched out, given that the work to
packages the dependencies has already been done, I see no reason to not
resolve the issue, given that the links you sent me to Debian seem to
believe all of these dependencies fit the requirements of free software.

Elana Hashman <ehashman@debian.org> writes:

> These deps are all open source as far as I can tell, but given that it's
> a lot of work to enable commercial/proprietary development I personally
> do not want to spend volunteer time on it. I would not block someone
> from volunteering to work on this, particularly with a commercial
> sponsor, but I don't think it'd be a great use of time for e.g. a GSoC
> student.

To be clear, I am not working on these patches on behalf of a commercial
sponsor, but because it didn't feel right to me to package the library
with significant functionality missing, especially not in the long term.

Maxime Devos <maximedevos@telenet.be> writes:

> Well, the (original?) source (the JSON files) seems to be free software,
> but if there is no free compiler for the source (and worse, there is
> only a single compiler and it can only be used by a single entity
> (cognitect)), wouldn't that make the software, from a practical
> perspective, non-free?

It is my personal opinion that this is not a compilation, but a
translation. All this "compiler" is doing, as far as I can tell, is
reading the JSON source, and outputting an equivalent source in EDN
format. This seems completely fine to me, as nothing then prevents the
user of this library from editing the EDN source, which is equivalent to
the JSON source. In fact, since this is a Clojure library, the EDN
source is probably the preferred form for editing for people who are
likely to use it.

> gnu.org words this in more detail at
> <https://www.gnu.org/philosophy/when-free-depends-on-nonfree.html>.

This seems to focus on instances when some important software freedom is
lost. I can't see anything that would give guidance on what to do when
all freedoms are preserved, as they seem to be here.
M April 16, 2022, 7:16 p.m. UTC | #5
Reily Siegel schreef op za 16-04-2022 om 19:39 [+0200]:
> It is my personal opinion that this is not a compilation, but a
> translation. All this "compiler" is doing, as far as I can tell, is
> reading the JSON source, and outputting an equivalent source in EDN
> format.

Translation (between human languages) is hard.  Translation between
file formats can be hard to, e.g. for complicated documents, I often
see errors introduced by Libreoffice (odt) <-> Microsoft (docx)
conversion.  

> This seems completely fine to me, as nothing then prevents the
> user of this library from editing the EDN source, which is equivalent to
> the JSON source. In fact, since this is a Clojure library, the EDN
> source is probably the preferred form for editing for people who are
> likely to use it.

These EDN files could be modified, so non-Cognitect people could
perhaps, for every new API thing or modified API that Amazon
introduces, manually search for the relevant part in the JSON files and
transcribe them to EDN, do some tests to make sure they didn't make any
errors during the transcribing process.  Likewise for updating the
documentation parts.

However, that looks like a lot of pointless work and a power indynamic,
that could be trivially be resolved by Cognitect.

> This seems to focus on instances when some important software freedom
> is lost. I can't see anything that would give guidance on what to do
> when all freedoms are preserved, as they seem to be here.

Theoretically, the freedoms seem to be preserved here.  However, at
least currently, Cognitect is making exercising some of these freedoms
pointlessly difficult.

TBC, I'm not saying that Cognitect must make things easy, rather the
making things hard (by keeping the compiler, or translator, secret) is
what I consider the problem here.

Greetings,
Maxime.
M April 16, 2022, 7:32 p.m. UTC | #6
Reily Siegel schreef op vr 15-04-2022 om 15:55 [+0200]:
> Hello,
> 
> I just wanted to check in and make sure you saw the version 3 patches I
> sent in, having resolved the licensing issue.

I looked at the new jar that was uploaded
(https://mvnrepository.com/artifact/com.cognitect.aws/s3/822.2.1109.0)
and it looks like there is still a problem.  The APL states:

      (d) If the Work includes a "NOTICE" text file as part of its
          distribution, then any Derivative Works that You distribute
          must include a readable copy of the attribution notices
          contained within such NOTICE file, excluding [...]

and <https://github.com/aws/aws-sdk-js> has a NOTICE.txt.  However, it
is missing from the com.cognitect.aws.s3 jar.

Cognitect also modified the files (by the JSON -> EDN compilation), and
APL says:

  You must cause any modified files to carry prominent notices stating
  that You changed the files; and

though perhaps, if JSON -> EDN is sufficiently lossless, this could be
considered not a modification, I don't know.

Greetings,
Maxime.
Reily Siegel April 16, 2022, 7:35 p.m. UTC | #7
Maxime Devos <maximedevos@telenet.be> writes:

> Theoretically, the freedoms seem to be preserved here.  However, at
> least currently, Cognitect is making exercising some of these freedoms
> pointlessly difficult.
>
> TBC, I'm not saying that Cognitect must make things easy, rather the
> making things hard (by keeping the compiler, or translator, secret) is
> what I consider the problem here.

Per this issue [1] on the aws-api GitHub, Cognitect has stated that the
translation code (they call it a generator) is not open source when
asked by a community member to view the source. As for why this is, I
couldn't guess, it seems to be a relatively trivial piece of code. Is
this issue blocking for the inclusion of the package?

[1]: https://github.com/cognitect-labs/aws-api/issues/116
M April 16, 2022, 7:45 p.m. UTC | #8
Reily Siegel schreef op za 16-04-2022 om 21:35 [+0200]:
> Maxime Devos <maximedevos@telenet.be> writes:
> 
> > Theoretically, the freedoms seem to be preserved here.  However, at
> > least currently, Cognitect is making exercising some of these freedoms
> > pointlessly difficult.
> > 
> > TBC, I'm not saying that Cognitect must make things easy, rather the
> > making things hard (by keeping the compiler, or translator, secret) is
> > what I consider the problem here.
> 
> Per this issue [1] on the aws-api GitHub, Cognitect has stated that the
> translation code (they call it a generator) is not open source when
> asked by a community member to view the source. As for why this is, I
> couldn't guess, it seems to be a relatively trivial piece of code.

If it's a trivial piece of code, it would be trivial for Cognitect to
make it free software without any business loss or something.

> Is this issue blocking for the inclusion of the package?
> 
> [1]: https://github.com/cognitect-labs/aws-api/issues/116

From my POV, yes.  Also see the new mail about another license issue (a
missing NOTICE.txt).

Greetings,
Maxime.
M April 16, 2022, 7:45 p.m. UTC | #9
Reily Siegel schreef op za 16-04-2022 om 19:39 [+0200]:
> To be clear, I am not working on these patches on behalf of a commercial
> sponsor, but because it didn't feel right to me to package the library
> with significant functionality missing, especially not in the long term.

TBC, I have no problems with commercial sponsers and commerce in
general.

While there might be some functionality missing, it appears to be only
useful for being a client of Amazon.  I don't see any loss at not
helping Amazon, which reportedly practices reportedly include bad
worker conditions, tax avoidance and many other things.

Greetings,
Maxime.
Reily Siegel April 16, 2022, 8:11 p.m. UTC | #10
Maxime Devos <maximedevos@telenet.be> writes:

> I looked at the new jar that was uploaded
> (https://mvnrepository.com/artifact/com.cognitect.aws/s3/822.2.1109.0)
> and it looks like there is still a problem.  The APL states:
>
>       (d) If the Work includes a "NOTICE" text file as part of its
>           distribution, then any Derivative Works that You distribute
>           must include a readable copy of the attribution notices
>           contained within such NOTICE file, excluding [...]
>
> and <https://github.com/aws/aws-sdk-js> has a NOTICE.txt.  However, it
> is missing from the com.cognitect.aws.s3 jar.
>
> Cognitect also modified the files (by the JSON -> EDN compilation), and
> APL says:
>
>   You must cause any modified files to carry prominent notices stating
>   that You changed the files; and
>
> though perhaps, if JSON -> EDN is sufficiently lossless, this could be
> considered not a modification, I don't know.
>
> Greetings,
> Maxime.

I have made Cognitect aware of this here:
https://github.com/cognitect-labs/aws-api/issues/208
Reily Siegel April 16, 2022, 8:15 p.m. UTC | #11
Maxime Devos <maximedevos@telenet.be> writes:

>> Is this issue blocking for the inclusion of the package?
>> 
>> [1]: https://github.com/cognitect-labs/aws-api/issues/116
>
> From my POV, yes.

In this case, would you like me to submit an alternative patch that
keeps these features patched out, but silences the warnings printed that
S3 classes are not available?
M April 16, 2022, 8:47 p.m. UTC | #12
Reily Siegel schreef op za 16-04-2022 om 22:15 [+0200]:
> Maxime Devos <maximedevos@telenet.be> writes:
> 
> > > Is this issue blocking for the inclusion of the package?
> > > 
> > > [1]: https://github.com/cognitect-labs/aws-api/issues/116
> > 
> > From my POV, yes.
> 
> In this case, would you like me to submit an alternative patch that
> keeps these features patched out, but silences the warnings printed that
> S3 classes are not available?

When do these warnings happen?  Only if the user asked to do some S3
things, or also when no S3 things are done at all?  If it's the former,
I don't see a reason to patch out the warnings, though if it's the
latter I agree.

I haven't looked at the v3 patches in detail yet, but if the
com-cognitect-aws-s3 package is not part of the v4, then I expect the
v4 to be good.

FWIW, I don't think I'll have time to look again anytime soonish.

Greetings,
Maxime
Reily Siegel April 16, 2022, 9:14 p.m. UTC | #13
Maxime Devos <maximedevos@telenet.be> writes:

> Reily Siegel schreef op za 16-04-2022 om 22:15 [+0200]:
>> Maxime Devos <maximedevos@telenet.be> writes:
>> 
>> > > Is this issue blocking for the inclusion of the package?
>> > > 
>> > > [1]: https://github.com/cognitect-labs/aws-api/issues/116
>> > 
>> > From my POV, yes.
>> 
>> In this case, would you like me to submit an alternative patch that
>> keeps these features patched out, but silences the warnings printed that
>> S3 classes are not available?
>
> When do these warnings happen?  Only if the user asked to do some S3
> things, or also when no S3 things are done at all?  If it's the former,
> I don't see a reason to patch out the warnings, though if it's the
> latter I agree.

These warnings happen every time the tool is run, here is what it looks
like:

"Warning: failed to load the S3TransporterFactory class"

It is very trivial to patch out.

> I haven't looked at the v3 patches in detail yet, but if the
> com-cognitect-aws-s3 package is not part of the v4, then I expect the
> v4 to be good.
>
> FWIW, I don't think I'll have time to look again anytime soonish.

If I simply patch out this one feature, the resulting patchset will be
much simpler. Most of this patch is adding dependencies of aws-api. If
that can be patched out, the only change needed over the existing
clojure-tools package is to remove the one line responsible for printing
this warning from the file, should be doable with an easy substitution.