diff mbox series

[bug#48999] import: hackage: Accept local source for package.

Message ID a7ff1db35e4294c161b871d47a8908af12d1efd3.1623577110.git.public@yoctocell.xyz
State New
Headers show
Series [bug#48999] import: hackage: Accept local source for package. | expand

Checks

Context Check Description
cbaines/comparison success View comparision
cbaines/git branch success View Git branch
cbaines/applying patch success View Laminar job
cbaines/issue success View issue

Commit Message

Xinglu Chen June 13, 2021, 9:46 a.m. UTC 
When developing a Haskell package it is often useful to have a Guix package
definition for that package, previously one would have to write that package
definition by hand, and if the .cabal file changed one would manually update
the Guix package definition.

This commit allows one to specify a custom source for their package, meaning
that one could programatically generate a Guix package definition for their
local Haskell package.  If the .cabal file changes, the generated package
definition will also change accordingly.  One could for instance write the
following in a guix.scm file:

  (define-values (ghc-haskeme deps)
    (call-with-input-file "haskeme.cabal"
      (lambda (port)
        (hackage->guix-package
         "haskeme"
         #:port port
         #:source (local-file "." "haskeme-checkout"
                              #:recursive? #t
                              #:select? hg-predicate)))))

  ghc-haskeme

Invoking ‘guix build -f guix.scm’ would then always build an up-to-date
version of the package.

* guix/import/hackage.scm (hackage-module->sexp): Add optional keyword
argument ‘source’
(hackage->guix-package): Likewise.
* tests/hackage.scm (eval-test-with-cabal): Likewise.
("hackage->guix-package local source"): New test.
---
Without this I would have to manually write my Guix package definition,
and manually update the ‘inputs’ field if I added a new dependency in my
.cabal file.  This patch offers something similar to pkgs.callCabal2nix
(I could find any link to the docs, classic Nix) in Nixpkgs where i can
just write the following

--8<---------------cut here---------------start------------->8---
let
  haskellPackages = pkgs.haskell.packages.${compiler}.override {
    overrides = hpNew: hpOld: {
      haskeme = hpNew.callCabal2nix "haskeme" ./. { };
    };
  };
in
{
  haskeme = haskellPackages.haskeme;
}
--8<---------------cut here---------------end--------------->8---

to get a Nix package definition of my Haskell package.

 guix/import/hackage.scm | 43 ++++++++++++++++++++++++++---------------
 tests/hackage.scm       | 28 +++++++++++++++++++++++++--
 2 files changed, 53 insertions(+), 18 deletions(-)


base-commit: acb858881901aa28499f83f40d3e04d6e4749453

Comments

Ludovic Courtès June 29, 2021, 9:44 a.m. UTC | #1 
Hi,

Xinglu Chen <public@yoctocell.xyz> skribis:

> When developing a Haskell package it is often useful to have a Guix package
> definition for that package, previously one would have to write that package
> definition by hand, and if the .cabal file changed one would manually update
> the Guix package definition.
>
> This commit allows one to specify a custom source for their package, meaning
> that one could programatically generate a Guix package definition for their
> local Haskell package.  If the .cabal file changes, the generated package
> definition will also change accordingly.  One could for instance write the
> following in a guix.scm file:
>
>   (define-values (ghc-haskeme deps)
>     (call-with-input-file "haskeme.cabal"
>       (lambda (port)
>         (hackage->guix-package
>          "haskeme"
>          #:port port
>          #:source (local-file "." "haskeme-checkout"
>                               #:recursive? #t
>                               #:select? hg-predicate)))))
>
>   ghc-haskeme
>
> Invoking ‘guix build -f guix.scm’ would then always build an up-to-date
> version of the package.
>
> * guix/import/hackage.scm (hackage-module->sexp): Add optional keyword
> argument ‘source’
> (hackage->guix-package): Likewise.
> * tests/hackage.scm (eval-test-with-cabal): Likewise.
> ("hackage->guix-package local source"): New test.

Looks like a nice improvement.

What I don’t get is that this functionality doesn’t seem to be available
from the CLI, which the patch doesn’t change.  Or am I missing
something?

Thanks,
Ludo’.
Xinglu Chen June 29, 2021, 7 p.m. UTC | #2 
On Tue, Jun 29 2021, Ludovic Courtès wrote:

> Hi,
>
> Xinglu Chen <public@yoctocell.xyz> skribis:
>
>> When developing a Haskell package it is often useful to have a Guix package
>> definition for that package, previously one would have to write that package
>> definition by hand, and if the .cabal file changed one would manually update
>> the Guix package definition.
>>
>> This commit allows one to specify a custom source for their package, meaning
>> that one could programatically generate a Guix package definition for their
>> local Haskell package.  If the .cabal file changes, the generated package
>> definition will also change accordingly.  One could for instance write the
>> following in a guix.scm file:
>>
>>   (define-values (ghc-haskeme deps)
>>     (call-with-input-file "haskeme.cabal"
>>       (lambda (port)
>>         (hackage->guix-package
>>          "haskeme"
>>          #:port port
>>          #:source (local-file "." "haskeme-checkout"
>>                               #:recursive? #t
>>                               #:select? hg-predicate)))))
>>
>>   ghc-haskeme
>>
>> Invoking ‘guix build -f guix.scm’ would then always build an up-to-date
>> version of the package.
>>
>> * guix/import/hackage.scm (hackage-module->sexp): Add optional keyword
>> argument ‘source’
>> (hackage->guix-package): Likewise.
>> * tests/hackage.scm (eval-test-with-cabal): Likewise.
>> ("hackage->guix-package local source"): New test.
>
> Looks like a nice improvement.
>
> What I don’t get is that this functionality doesn’t seem to be available
> from the CLI, which the patch doesn’t change.  Or am I missing
> something?

No, I don’t think this functionality is available from the CLI since
‘hackage->guix-package’ is not called with the #:source keyword
argument.  Once all the other importers (or those where it makes sense
to do this) get this functionality,  it would be a good idea to add an
option for reading a .cabal/setup.py/whatever file and generating a
package definition from that.
Ludovic Courtès June 30, 2021, 9:17 a.m. UTC | #3 
Hi,

Xinglu Chen <public@yoctocell.xyz> skribis:

> On Tue, Jun 29 2021, Ludovic Courtès wrote:
>
>> Hi,
>>
>> Xinglu Chen <public@yoctocell.xyz> skribis:
>>
>>> When developing a Haskell package it is often useful to have a Guix package
>>> definition for that package, previously one would have to write that package
>>> definition by hand, and if the .cabal file changed one would manually update
>>> the Guix package definition.
>>>
>>> This commit allows one to specify a custom source for their package, meaning
>>> that one could programatically generate a Guix package definition for their
>>> local Haskell package.  If the .cabal file changes, the generated package
>>> definition will also change accordingly.  One could for instance write the
>>> following in a guix.scm file:
>>>
>>>   (define-values (ghc-haskeme deps)
>>>     (call-with-input-file "haskeme.cabal"
>>>       (lambda (port)
>>>         (hackage->guix-package
>>>          "haskeme"
>>>          #:port port
>>>          #:source (local-file "." "haskeme-checkout"
>>>                               #:recursive? #t
>>>                               #:select? hg-predicate)))))
>>>
>>>   ghc-haskeme
>>>
>>> Invoking ‘guix build -f guix.scm’ would then always build an up-to-date
>>> version of the package.
>>>
>>> * guix/import/hackage.scm (hackage-module->sexp): Add optional keyword
>>> argument ‘source’
>>> (hackage->guix-package): Likewise.
>>> * tests/hackage.scm (eval-test-with-cabal): Likewise.
>>> ("hackage->guix-package local source"): New test.
>>
>> Looks like a nice improvement.
>>
>> What I don’t get is that this functionality doesn’t seem to be available
>> from the CLI, which the patch doesn’t change.  Or am I missing
>> something?
>
> No, I don’t think this functionality is available from the CLI since
> ‘hackage->guix-package’ is not called with the #:source keyword
> argument.

IOW, this functionality is not accessible, unless you use the Scheme API
as in the example above, right?

> Once all the other importers (or those where it makes sense to do
> this) get this functionality, it would be a good idea to add an option
> for reading a .cabal/setup.py/whatever file and generating a package
> definition from that.

‘guix import hackage’ could support it even if other importers don’t
have equivalent functionality, no?

Thanks,
Ludo’.
Simon Tournier June 30, 2021, 2:10 p.m. UTC | #4 
Hi,

On Wed, 30 Jun 2021 at 11:17, Ludovic Courtès <ludo@gnu.org> wrote:
>>> Xinglu Chen <public@yoctocell.xyz> skribis:

>>>>   (define-values (ghc-haskeme deps)
>>>>     (call-with-input-file "haskeme.cabal"
>>>>       (lambda (port)
>>>>         (hackage->guix-package
>>>>          "haskeme"
>>>>          #:port port
>>>>          #:source (local-file "." "haskeme-checkout"
>>>>                               #:recursive? #t
>>>>                               #:select? hg-predicate)))))
>>>>
>>>>   ghc-haskeme
>>>>
>>>> Invoking ‘guix build -f guix.scm’ would then always build an up-to-date
>>>> version of the package.

[...]

>>> What I don’t get is that this functionality doesn’t seem to be available
>>> from the CLI, which the patch doesn’t change.  Or am I missing
>>> something?
>>
>> No, I don’t think this functionality is available from the CLI since
>> ‘hackage->guix-package’ is not called with the #:source keyword
>> argument.
>
> IOW, this functionality is not accessible, unless you use the Scheme API
> as in the example above, right?
>
>> Once all the other importers (or those where it makes sense to do
>> this) get this functionality, it would be a good idea to add an option
>> for reading a .cabal/setup.py/whatever file and generating a package
>> definition from that.
>
> ‘guix import hackage’ could support it even if other importers don’t
> have equivalent functionality, no?

From my understanding, there are 2 levels:

 1. simplify the Scheme snippet for developing local “foreign” package,
 i.e., “guix build -f guix.scm” with a simpler ’guix.scm’ file.

 2. an option to import local package,
 e.g., “guix import hackage --path=.”

and this for (almost) all the importers.


Cheers,
simon
Xinglu Chen July 4, 2021, 10:43 a.m. UTC | #5 
On Wed, Jun 30 2021, Ludovic Courtès wrote:

> Hi,
>
> Xinglu Chen <public@yoctocell.xyz> skribis:
>
>> On Tue, Jun 29 2021, Ludovic Courtès wrote:
>>
>>> Hi,
>>>
>>> Xinglu Chen <public@yoctocell.xyz> skribis:
>>>
>>>> When developing a Haskell package it is often useful to have a Guix package
>>>> definition for that package, previously one would have to write that package
>>>> definition by hand, and if the .cabal file changed one would manually update
>>>> the Guix package definition.
>>>>
>>>> This commit allows one to specify a custom source for their package, meaning
>>>> that one could programatically generate a Guix package definition for their
>>>> local Haskell package.  If the .cabal file changes, the generated package
>>>> definition will also change accordingly.  One could for instance write the
>>>> following in a guix.scm file:
>>>>
>>>>   (define-values (ghc-haskeme deps)
>>>>     (call-with-input-file "haskeme.cabal"
>>>>       (lambda (port)
>>>>         (hackage->guix-package
>>>>          "haskeme"
>>>>          #:port port
>>>>          #:source (local-file "." "haskeme-checkout"
>>>>                               #:recursive? #t
>>>>                               #:select? hg-predicate)))))
>>>>
>>>>   ghc-haskeme
>>>>
>>>> Invoking ‘guix build -f guix.scm’ would then always build an up-to-date
>>>> version of the package.
>>>>
>>>> * guix/import/hackage.scm (hackage-module->sexp): Add optional keyword
>>>> argument ‘source’
>>>> (hackage->guix-package): Likewise.
>>>> * tests/hackage.scm (eval-test-with-cabal): Likewise.
>>>> ("hackage->guix-package local source"): New test.
>>>
>>> Looks like a nice improvement.
>>>
>>> What I don’t get is that this functionality doesn’t seem to be available
>>> from the CLI, which the patch doesn’t change.  Or am I missing
>>> something?
>>
>> No, I don’t think this functionality is available from the CLI since
>> ‘hackage->guix-package’ is not called with the #:source keyword
>> argument.
>
> IOW, this functionality is not accessible, unless you use the Scheme API
> as in the example above, right?

Yes.

>> Once all the other importers (or those where it makes sense to do
>> this) get this functionality, it would be a good idea to add an option
>> for reading a .cabal/setup.py/whatever file and generating a package
>> definition from that.
>
> ‘guix import hackage’ could support it even if other importers don’t
> have equivalent functionality, no?

Sure, that could work too.
Xinglu Chen July 4, 2021, 11:53 a.m. UTC | #6 
Changes since v1:

* Add ‘--path’ command line option for importing the package from the
  local filesystem.

* Add ‘git-repository?’ and ‘hg-repository?’ procedures for correctly
  setting the #:select? keyword (‘git-predicate’ or ‘hg-predicate’) for
  ‘local-file’.  Other VCS don’t have a ‘-predicate’ procedure, so I
  didn’t bother adding those.

Patch [1/3] hasn’t changed since v1.

Xinglu Chen (3):
  import: hackage: Accept local source for package.
  import: utils: Add predicates for checking VCS repositories.
  scripts: import: hackage: Add option to import package from local
    filesystem.

 guix/import/hackage.scm         | 43 +++++++++++++++++++++------------
 guix/import/utils.scm           | 15 ++++++++++++
 guix/scripts/import/hackage.scm | 43 +++++++++++++++++++++++++++------
 tests/hackage.scm               | 28 +++++++++++++++++++--
 4 files changed, 103 insertions(+), 26 deletions(-)


base-commit: b65af6ed9120234cf655e8e76317558cfbd02477
diff mbox series

Patch

diff --git a/guix/import/hackage.scm b/guix/import/hackage.scm
index f94a1e7087..326ab92365 100644
--- a/guix/import/hackage.scm
+++ b/guix/import/hackage.scm
@@ -227,10 +227,13 @@  package being processed and is used to filter references to itself."
           dependencies))
 
 (define* (hackage-module->sexp cabal cabal-hash
-                               #:key (include-test-dependencies? #t))
+                               #:key
+                               (include-test-dependencies? #t)
+                               (source #f))
   "Return the `package' S-expression for a Cabal package.  CABAL is the
-representation of a Cabal file as produced by 'read-cabal'.  CABAL-HASH is
-the hash of the Cabal file."
+representation of a Cabal file as produced by 'read-cabal'.  CABAL-HASH is the
+hash of the Cabal file.  If SOURCE is specified, it will be used as the source
+for the package."
 
   (define name
     (cabal-package-name cabal))
@@ -294,20 +297,24 @@  the hash of the Cabal file."
       (() '())
       (args `((arguments (,'quasiquote ,args))))))
 
-  (let ((tarball (with-store store
-                   (download-to-store store source-url))))
+  (let ((tarball (if source
+                     #f
+                     (with-store store
+                       (download-to-store store source-url)))))
     (values
      `(package
         (name ,(hackage-name->package-name name))
         (version ,version)
-        (source (origin
-                  (method url-fetch)
-                  (uri (string-append ,@(factorize-uri source-url version)))
-                  (sha256
-                   (base32
-                    ,(if tarball
-                         (bytevector->nix-base32-string (file-sha256 tarball))
-                         "failed to download tar archive")))))
+        (source ,(if source
+                     source
+                     `(origin
+                        (method url-fetch)
+                        (uri (string-append ,@(factorize-uri source-url version)))
+                        (sha256
+                         (base32
+                          ,(if tarball
+                               (bytevector->nix-base32-string (file-sha256 tarball))
+                               "failed to download tar archive"))))))
         (build-system haskell-build-system)
         ,@(maybe-inputs 'inputs dependencies)
         ,@(maybe-inputs 'native-inputs native-dependencies)
@@ -321,10 +328,12 @@  the hash of the Cabal file."
 (define* (hackage->guix-package package-name #:key
                                 (include-test-dependencies? #t)
                                 (port #f)
+                                (source #f)
                                 (cabal-environment '()))
   "Fetch the Cabal file for PACKAGE-NAME from hackage.haskell.org, or, if the
-called with keyword parameter PORT, from PORT.  Return the `package'
-S-expression corresponding to that package, or #f on failure.
+called with keyword parameter PORT, from PORT.  If SOURCE is specified, use it
+as the source for the package instead of trying to fetch a tarball.  Return
+the `package' S-expression corresponding to that package, or #f on failure.
 CABAL-ENVIRONMENT is an alist defining the environment in which the Cabal
 conditionals are evaluated.  The accepted keys are: \"os\", \"arch\", \"impl\"
 and the name of a flag.  The value associated with a flag has to be either the
@@ -338,7 +347,9 @@  respectively."
                     (hackage-fetch-and-hash package-name))))
     (and=> cabal-meta (compose (cut hackage-module->sexp <> cabal-hash
                                     #:include-test-dependencies?
-                                    include-test-dependencies?)
+                                    include-test-dependencies?
+                                    #:source
+                                    source)
                                (cut eval-cabal <> cabal-environment)))))
 
 (define hackage->guix-package/m                   ;memoized variant
diff --git a/tests/hackage.scm b/tests/hackage.scm
index 66a13d9881..54590dcece 100644
--- a/tests/hackage.scm
+++ b/tests/hackage.scm
@@ -22,6 +22,7 @@ 
   #:use-module (guix import cabal)
   #:use-module (guix import hackage)
   #:use-module (guix tests)
+  #:use-module (guix gexp)
   #:use-module (srfi srfi-64)
   #:use-module (ice-9 match))
 
@@ -186,9 +187,28 @@  library
     ('description (? string?))
     ('license 'license:bsd-3)))
 
-(define* (eval-test-with-cabal test-cabal matcher #:key (cabal-environment '()))
+(define-package-matcher match-ghc-foo-local-source
+  ('package
+    ('name "ghc-foo")
+    ('version "1.0.0")
+    ('source
+     (? file-like?))
+    ('build-system 'haskell-build-system)
+    ('inputs
+     ('quasiquote
+      (("ghc-http" ('unquote 'ghc-http)))))
+    ('home-page "http://test.org")
+    ('synopsis (? string?))
+    ('description (? string?))
+    ('license 'license:bsd-3)))
+
+(define* (eval-test-with-cabal test-cabal matcher
+                               #:key (cabal-environment '()) (source #f))
   (define port (open-input-string test-cabal))
-  (matcher (hackage->guix-package "foo" #:port port #:cabal-environment cabal-environment)))
+  (matcher (hackage->guix-package "foo"
+                                  #:port port
+                                  #:cabal-environment cabal-environment
+                                  #:source source)))
 
 (test-assert "hackage->guix-package test 1"
   (eval-test-with-cabal test-cabal-1 match-ghc-foo))
@@ -208,6 +228,10 @@  library
   (eval-test-with-cabal test-cabal-5 match-ghc-foo
                         #:cabal-environment '(("impl" . "ghc-7.8"))))
 
+(test-assert "hackage->guix-package local source"
+  (eval-test-with-cabal test-cabal-1 match-ghc-foo-local-source
+                        #:source (plain-file "dummy source" "source")))
+
 (define-package-matcher match-ghc-foo-6
   ('package
     ('name "ghc-foo")