[bug#57280,0/3] Add documentation-files argument to emacs build system.

On 2022-08-31 12:07, Liliana Marie Prikler wrote:

> Am Mittwoch, dem 31.08.2022 um 12:36 +0300 schrieb Andrew Tropin:
>> > I think if we want to go this more generic route, we'd have to
>> > redesign this a little.  For instance, (build-texinfo-
>> > documentation) should take
>> > regular expressions as remaining arguments.  
>> 
>> What can be a good place (module) for such build phases?
> I was thinking (guix build utils).  Of course, one could introduce a
> new module, but doing that would come with even more downsides in terms
> of UX (or PX if we're pedantic).

Ok, I prepared more generic version of the patch.
Picked (guix build emacs-utils) for now, it's done to avoid huge
rebuilds, while testing, later we can move it to (guix build utils).

Also, temporary added pandoc and texinfo to native-inputs for
emacs-build-system, otherwise I would need to update inputs for almost
every emacs-* package.  Need to figure out what to do with this.

>
>> Attaching the latest version of the documentation-files patch I have
> Looking at this patch, perhaps we'd also have to allow customizing
> command line options.  Also, as for installing, I think this should be
> handled by the install phase, which already has includes
> "^[^/]*\\.info$" and "^doc/.*\\.info$" by default.  Thus, you only need
> to build documentation before the install phase.

That's right, but in the new iteration (v3) of build-documentation phase
I use find-root-library-file, which expects to be executed when elpa
directory is already available, so I can't do it before install phase
and need to manually install info files.

Also, current build phases order is a little confusing, a lot of builds
happens after install phase, directly in output directory:

`set-SOURCE-DATE-EPOCH'
`set-paths'
`install-locale'
`unpack'
`expand-load-path'
`patch-usr-bin-file'
`patch-source-shebangs'
`patch-generated-file-shebangs'
`check'
`install'
`make-autoloads'
`enable-autoloads-compilation'
`patch-el-files'
`ensure-package-description'
`build'
`validate-compiled-autoloads'
`build-documentation'
`move-doc'
`patch-shebangs'
`strip'
`validate-runpath'
`validate-documentation-location'
`delete-info-dir-file'
`patch-dot-desktop-files'
`make-dynamic-linker-cache'
`install-license-files'
`reset-gzip-timestamps'
`compress-documentation'

What if instead of install phase we will use
create-tmp-lisp-and-documentation-directories phase (or something more
meaningful) to make a temporary directory, where we will build all the
stuff and after that, at the end of the build process will install
everything from this temporary directory to the store?  This way
emacs-build-system will become more usual and easier to understand and
predict.

Hi,

Am Freitag, dem 02.09.2022 um 17:02 +0300 schrieb Andrew Tropin:
> Picked (guix build emacs-utils) for now, it's done to avoid huge
> rebuilds, while testing, later we can move it to (guix build utils).
> 
> Also, temporary added pandoc and texinfo to native-inputs for
> emacs-build-system, otherwise I would need to update inputs for
> almost every emacs-* package.  Need to figure out what to do with
> this.
I still don't like that we need pandoc and texinfo as implicit inputs.
There ought to be a better solution than this.

> > 
> > > Attaching the latest version of the documentation-files patch I
> > > have
> > Looking at this patch, perhaps we'd also have to allow customizing
> > command line options.  Also, as for installing, I think this should
> > be
> > handled by the install phase, which already has includes
> > "^[^/]*\\.info$" and "^doc/.*\\.info$" by default.  Thus, you only
> > need
> > to build documentation before the install phase.
> 
> That's right, but in the new iteration (v3) of build-documentation
> phase I use find-root-library-file, which expects to be executed when
> elpa directory is already available, so I can't do it before install
> phase and need to manually install info files.
> 
> Also, current build phases order is a little confusing, a lot of
> builds happens after install phase, directly in output directory:
> 
> `set-SOURCE-DATE-EPOCH'
> `set-paths'
> `install-locale'
> `unpack'
> `expand-load-path'
> `patch-usr-bin-file'
> `patch-source-shebangs'
> `patch-generated-file-shebangs'
> `check'
> `install'
> `make-autoloads'
> `enable-autoloads-compilation'
> `patch-el-files'
> `ensure-package-description'
> `build'
> `validate-compiled-autoloads'
> `build-documentation'
> `move-doc'
> `patch-shebangs'
> `strip'
> `validate-runpath'
> `validate-documentation-location'
> `delete-info-dir-file'
> `patch-dot-desktop-files'
> `make-dynamic-linker-cache'
> `install-license-files'
> `reset-gzip-timestamps'
> `compress-documentation'
> 
> What if instead of install phase we will use create-tmp-lisp-and-
> documentation-directories phase (or something
> more meaningful) to make a temporary directory, where we will build
> all the stuff and after that, at the end of the build process will
> install everything from this temporary directory to the store?  This
> way emacs-build-system will become more usual and easier to
> understand andpredict.
I don't think we would need to do staged installations.  As for why we
don't build in the temporary directory, I do not know, I merely
inherited that code.

> +(define* (build-documentation-texinfo
> +          #:key
> +          (files '())
> +          (command '("makeinfo" "--no-split")))
> +  "Don't forget to add texinfo into list of inputs for the package."
> +  (lambda* (#:key outputs #:allow-other-keys)
> +    (for-each (lambda (f) (apply invoke (append command (list f))))
> files)))
This is hardly specific to emacs, is it?
Also, append is usually a code smell.  Can't we (apply invoke
"makeinfo" file options)?

> +(define* (convert-documentation
> +          #:key
> +          (mapping '())
> +          (command '("pandoc")))
> +  "Don't forget to add pandoc into list of inputs for the package."
> +  (lambda* (#:key outputs #:allow-other-keys)
> +    (for-each (lambda (p) (apply invoke
> +                                 (append command
> +                                         (list (car p) "-o" (cdr
> p)))))
> +              mapping)))
As above.

> +(define* (build-documentation-org
> +          #:key
> +          (files '()))
This one is emacs-specific due to emacs-batch-script and can remain
there.

> +    (add-after 'validate-compiled-autoloads 'move-doc move-doc)
> +    (add-before 'move-doc 'build-documentation
> build-documentation)))
I don't think that we'll have to add this phase once we've added the
helpers.  And as previously discussed, we'd have to build the
documentation before install.

Cheers

On 2022-09-02 16:52, Liliana Marie Prikler wrote:

> Hi,
>
> Am Freitag, dem 02.09.2022 um 17:02 +0300 schrieb Andrew Tropin:
>> Picked (guix build emacs-utils) for now, it's done to avoid huge
>> rebuilds, while testing, later we can move it to (guix build utils).
>> 
>> Also, temporary added pandoc and texinfo to native-inputs for
>> emacs-build-system, otherwise I would need to update inputs for
>> almost every emacs-* package.  Need to figure out what to do with
>> this.
> I still don't like that we need pandoc and texinfo as implicit inputs.
> There ought to be a better solution than this.
>
>> > 
>> > > Attaching the latest version of the documentation-files patch I
>> > > have
>> > Looking at this patch, perhaps we'd also have to allow customizing
>> > command line options.  Also, as for installing, I think this should
>> > be
>> > handled by the install phase, which already has includes
>> > "^[^/]*\\.info$" and "^doc/.*\\.info$" by default.  Thus, you only
>> > need
>> > to build documentation before the install phase.
>> 
>> That's right, but in the new iteration (v3) of build-documentation
>> phase I use find-root-library-file, which expects to be executed when
>> elpa directory is already available, so I can't do it before install
>> phase and need to manually install info files.
>> 
>> Also, current build phases order is a little confusing, a lot of
>> builds happens after install phase, directly in output directory:
>> 
>> `set-SOURCE-DATE-EPOCH'
>> `set-paths'
>> `install-locale'
>> `unpack'
>> `expand-load-path'
>> `patch-usr-bin-file'
>> `patch-source-shebangs'
>> `patch-generated-file-shebangs'
>> `check'
>> `install'
>> `make-autoloads'
>> `enable-autoloads-compilation'
>> `patch-el-files'
>> `ensure-package-description'
>> `build'
>> `validate-compiled-autoloads'
>> `build-documentation'
>> `move-doc'
>> `patch-shebangs'
>> `strip'
>> `validate-runpath'
>> `validate-documentation-location'
>> `delete-info-dir-file'
>> `patch-dot-desktop-files'
>> `make-dynamic-linker-cache'
>> `install-license-files'
>> `reset-gzip-timestamps'
>> `compress-documentation'
>> 
>> What if instead of install phase we will use create-tmp-lisp-and-
>> documentation-directories phase (or something
>> more meaningful) to make a temporary directory, where we will build
>> all the stuff and after that, at the end of the build process will
>> install everything from this temporary directory to the store?  This
>> way emacs-build-system will become more usual and easier to
>> understand andpredict.
> I don't think we would need to do staged installations.  As for why we
> don't build in the temporary directory, I do not know, I merely
> inherited that code.
>
>> +(define* (build-documentation-texinfo
>> +          #:key
>> +          (files '())
>> +          (command '("makeinfo" "--no-split")))
>> +  "Don't forget to add texinfo into list of inputs for the package."
>> +  (lambda* (#:key outputs #:allow-other-keys)
>> +    (for-each (lambda (f) (apply invoke (append command (list f))))
>> files)))
> This is hardly specific to emacs, is it?
> Also, append is usually a code smell.  Can't we (apply invoke
> "makeinfo" file options)?
>
>> +(define* (convert-documentation
>> +          #:key
>> +          (mapping '())
>> +          (command '("pandoc")))
>> +  "Don't forget to add pandoc into list of inputs for the package."
>> +  (lambda* (#:key outputs #:allow-other-keys)
>> +    (for-each (lambda (p) (apply invoke
>> +                                 (append command
>> +                                         (list (car p) "-o" (cdr
>> p)))))
>> +              mapping)))
> As above.
>
>> +(define* (build-documentation-org
>> +          #:key
>> +          (files '()))
> This one is emacs-specific due to emacs-batch-script and can remain
> there.
>
>> +    (add-after 'validate-compiled-autoloads 'move-doc move-doc)
>> +    (add-before 'move-doc 'build-documentation
>> build-documentation)))
> I don't think that we'll have to add this phase once we've added the
> helpers.  And as previously discussed, we'd have to build the
> documentation before install.
>
> Cheers

I think we can consider this patch series as an thought experiment.

Manually adding documentation build phases works good enough and
probably there is no need to do something more generic like this.

Closing the ticket without merging.

Liliana, thank you for you time!