[bug#63139,python,02/20] gnu: Add python-pyproject-hooks.

* gnu/packages/python-build.scm (python-pyproject-hooks): New variable.
---
 gnu/packages/python-build.scm | 41 +++++++++++++++++++++++++++++++++++
 1 file changed, 41 insertions(+)

--
2.39.2

Hi,

> +               (replace 'build
> +                 (lambda _
> +                   (invoke "python" "-m" "build" "--wheel" "--no-isolation" ".")))
> +               (replace 'install
> +                 (lambda _
> +                   (let ((whl (car (find-files "dist" "\\.whl$"))))
> +                     (invoke "pip" "--no-cache-dir" "--no-input"
> +                             "install" "--no-deps" "--prefix" #$output whl)))))))

why is this necessary? As far as I see pyproject-hooks wants to be built
using flit_core, which we have a fully bootstrapped version of. Same
for pypa-build, which – as far as I can see – also builds using flit_core.

Cheers,
Lars

Hi Lars,

On Mon, May 01, 2023 at 08:58 AM, Lars-Dominik Braun wrote:

> Hi,
>
>> +               (replace 'build
>> +                 (lambda _
>> +                   (invoke "python" "-m" "build" "--wheel" "--no-isolation" ".")))
>> +               (replace 'install
>> +                 (lambda _
>> +                   (let ((whl (car (find-files "dist" "\\.whl$"))))
>> +                     (invoke "pip" "--no-cache-dir" "--no-input"
>> +                             "install" "--no-deps" "--prefix" #$output whl)))))))
>
> why is this necessary? As far as I see pyproject-hooks wants to be built
> using flit_core, which we have a fully bootstrapped version of. Same
> for pypa-build, which – as far as I can see – also builds using flit_core.
>

This gets to what I wasn't sure about, what exactly needs to be
bootstrapped and what depends on what. From what I remember and
thought, the newer python-pypa-build needs python-pyproject-hooks.
Which itself would be built with pyproject-build-system, creating a
circular dependency. But, I don't know anything about flit_core and
that system. So you are saying flit_core is used to build each of
these without a cycle then?

Can you layout what the new structure would be for updating
python-pypa-build with python-pyproject-hooks then? (And then
deprecate pep517.)

Thanks!
John

Hi,

> From what I remember and
> thought, the newer python-pypa-build needs python-pyproject-hooks.
> Which itself would be built with pyproject-build-system, creating a
> circular dependency.

ah, yes, you’re right. For some reason the current
pyproject-build-system differs from my original proposal and bundles
python-pypa-build in python-toolchain (even though it is not used). As
we can see from this example doing so is a very very bad idea (because
it leads to cycles).

> But, I don't know anything about flit_core and
> that system. So you are saying flit_core is used to build each of
> these without a cycle then?
> Can you layout what the new structure would be for updating
> python-pypa-build with python-pyproject-hooks then? (And then
> deprecate pep517.)

In theory that would be possible if python-toolchain did not depend on
pypa-build. Looking at the respective pyproject.toml files flit-core
is entirely self-contained and can be built using itself. Then you can
build pyproject-hooks, packaging and tomli using flit-core. And then
pypa-build can be built using flit-core and all of the previous packages
as dependencies. poetry-core is also self-contained and can be built
without any dependencies (and the full poetry should never be required
to build any project, but obviously packages in Guix use it for that
purpose -.-). But none of this is really necessary, because – again
– pypa-build is not used by pyproject-build-system.

So I propose the following:

- Create a python-team branch.
- Use python-sans-pip-wrapper as default-python in pyproject-build-system,
  like I originally proposed and add python-setuptools/python-wheel
  where necessary.
- Implement backend-path in pyproject-build-system, so we can bootstrap
  flit-core/poetry-core/… properly.
- Expose ensure-no-mtimes-pre-1980 (#63044).
- Finally fix python-yubikey-manager.

Anything else that’s broken right now?

Cheers,
Lars

Hi Lars

Apologies for the delayed response (meant to earlier), top posting, and the formatting. The reason for all three is that I'm currently away from a proper computer for a few weeks but I'll be back at the beginning of June and can start hacking away then. Of course, happy to be beat to the job or have anything to build off of.

I'm not sure of any other deep python breakages, but definitely something to look at for the python branch. Although I'm not python expert, especially at packaging, I should probably add myself to the team when I'm back as well.

Thanks for the details and information, that's very helpful and sounds like a roadmap to have a clean and updated toolchain here.

John
-------- Original Message --------
On May 8, 2023, 3:44 PM, Lars-Dominik Braun wrote:

> Hi, > From what I remember and > thought, the newer python-pypa-build needs python-pyproject-hooks. > Which itself would be built with pyproject-build-system, creating a > circular dependency. ah, yes, you’re right. For some reason the current pyproject-build-system differs from my original proposal and bundles python-pypa-build in python-toolchain (even though it is not used). As we can see from this example doing so is a very very bad idea (because it leads to cycles). > But, I don't know anything about flit_core and > that system. So you are saying flit_core is used to build each of > these without a cycle then? > Can you layout what the new structure would be for updating > python-pypa-build with python-pyproject-hooks then? (And then > deprecate pep517.) In theory that would be possible if python-toolchain did not depend on pypa-build. Looking at the respective pyproject.toml files flit-core is entirely self-contained and can be built using itself. Then you can build pyproject-hooks, packaging and tomli using flit-core. And then pypa-build can be built using flit-core and all of the previous packages as dependencies. poetry-core is also self-contained and can be built without any dependencies (and the full poetry should never be required to build any project, but obviously packages in Guix use it for that purpose -.-). But none of this is really necessary, because – again – pypa-build is not used by pyproject-build-system. So I propose the following: - Create a python-team branch. - Use python-sans-pip-wrapper as default-python in pyproject-build-system, like I originally proposed and add python-setuptools/python-wheel where necessary. - Implement backend-path in pyproject-build-system, so we can bootstrap flit-core/poetry-core/… properly. - Expose ensure-no-mtimes-pre-1980 (#63044). - Finally fix python-yubikey-manager. Anything else that’s broken right now? Cheers, Lars

Hi John,

> Apologies for the delayed response (meant to earlier), top posting, and the formatting. The reason for all three is that I'm currently away from a proper computer for a few weeks but I'll be back at the beginning of June and can start hacking away then. Of course, happy to be beat to the job or have anything to build off of.

no worries. I also won’t have any time to look into this
for a few weeks. My (rather dirty) python-team branch is here:
https://github.com/PromyLOPh/guix/commits/python-team
Perhaps it helps you to fix this issue.

Cheers,
Lars