mbox series

[bug#65119,0/8] Sharing service code between Home and System

Message ID cover.1691355218.git.ludo@gnu.org
Headers show
Series Sharing service code between Home and System | expand

Message

Ludovic Courtès Aug. 6, 2023, 9:04 p.m. UTC
Hello Guix!

Some services make as much sense as a System and as a Home service.
Two examples that came up recently are Syncthing and dicod:

  https://issues.guix.gnu.org/63758
  https://issues.guix.gnu.org/62401

In these cases, we’d rather avoid code duplication and have a common
code base for both the System and the Home service.

This patch lets us map System services to Home and uses the new
mechanism for three services: mcron, dicod, and syncthing.

Feedback welcome!

Ludo’.

Ludovic Courtès (8):
  services: dicod: Remove Shepherd < 0.9.0 compatibility layer.
  services: dicod: Pre-build the GCIDE index.
  services: syncthing: Use 'match-record'.
  services: Define 'for-home'.
  home: services: Support mapping of System services to Home services.
  home: services: mcron: Define as a mapping of the system service.
  home: services: Add dicod.
  home: services: Add Syncthing.

 doc/guix.texi                   | 96 ++++++++++++++++++++++++++++++++-
 gnu/home/services.scm           | 69 +++++++++++++++++++++++-
 gnu/home/services/dict.scm      | 32 +++++++++++
 gnu/home/services/mcron.scm     | 96 ++++-----------------------------
 gnu/home/services/shepherd.scm  |  4 +-
 gnu/home/services/syncthing.scm | 30 +++++++++++
 gnu/local.mk                    |  2 +
 gnu/services.scm                | 25 ++++++++-
 gnu/services/dict.scm           | 84 ++++++++++++++---------------
 gnu/services/mcron.scm          | 80 ++++++++++++---------------
 gnu/services/syncthing.scm      | 56 ++++++++++---------
 11 files changed, 372 insertions(+), 202 deletions(-)
 create mode 100644 gnu/home/services/dict.scm
 create mode 100644 gnu/home/services/syncthing.scm


base-commit: 0ab46ef3f9719f03d9b191a16e5aa91619e95451

Comments

宋文武 Aug. 13, 2023, 5:28 a.m. UTC | #1
Ludovic Courtès <ludo@gnu.org> writes:

> [...]
>
> This patch lets us map System services to Home and uses the new
> mechanism for three services: mcron, dicod, and syncthing.
>
>   services: dicod: Remove Shepherd < 0.9.0 compatibility layer.
>   services: dicod: Pre-build the GCIDE index.
>   services: syncthing: Use 'match-record'.
>   services: Define 'for-home'.
>   home: services: Support mapping of System services to Home services.
>   home: services: mcron: Define as a mapping of the system service.
>   home: services: Add dicod.
>   home: services: Add Syncthing.

I didn't test them, but those changes look good to me!
Ludovic Courtès Aug. 20, 2023, 9:23 p.m. UTC | #2
Hi,

宋文武 <iyzsong@envs.net> skribis:

> Ludovic Courtès <ludo@gnu.org> writes:
>
>> [...]
>>
>> This patch lets us map System services to Home and uses the new
>> mechanism for three services: mcron, dicod, and syncthing.
>>
>>   services: dicod: Remove Shepherd < 0.9.0 compatibility layer.
>>   services: dicod: Pre-build the GCIDE index.
>>   services: syncthing: Use 'match-record'.
>>   services: Define 'for-home'.
>>   home: services: Support mapping of System services to Home services.
>>   home: services: mcron: Define as a mapping of the system service.
>>   home: services: Add dicod.
>>   home: services: Add Syncthing.
>
> I didn't test them, but those changes look good to me!

Pushed as 7605c01fccb20b387f5a0d98d6b81074b3039e23, thanks for taking a
look!

Ludo’.
Andrew Tropin Aug. 21, 2023, 1:43 p.m. UTC | #3
On 2023-08-13 13:28, 宋文武 wrote:

> Ludovic Courtès <ludo@gnu.org> writes:
>
>> [...]
>>
>> This patch lets us map System services to Home and uses the new
>> mechanism for three services: mcron, dicod, and syncthing.
>>
>>   services: dicod: Remove Shepherd < 0.9.0 compatibility layer.
>>   services: dicod: Pre-build the GCIDE index.
>>   services: syncthing: Use 'match-record'.
>>   services: Define 'for-home'.
>>   home: services: Support mapping of System services to Home services.
>>   home: services: mcron: Define as a mapping of the system service.
>>   home: services: Add dicod.
>>   home: services: Add Syncthing.
>
> I didn't test them, but those changes look good to me!

Hi Ludovic, 宋文武!

Sorry for comming late to the party, I saw this message only a week ago
and didn't have time to make an extensive reply yet, so I will share my
quick thought on the most problematic part and maybe later will
formulate others thoughts in more details.

define-service-type-mapping looks imperative and potentially very
problematic.  Collecting those values in unknown order and applying this
implicit transformation making a good room for foot shooting.  Imagine
someone would like to use his own (let's say) shepherd home service
implementation and will add this to one of the source files of their
channel:

--8<---------------cut here---------------start------------->8---
(define-service-type-mapping
  shepherd-root-service-type => my-home-shepherd-service-type)
--8<---------------cut here---------------end--------------->8---

What happens if somebody will use his channel just for getting some
package?  Very likely it would break the build or in the worst case it
will build with unexpected service implementation under the hood.

I had [1][2] and still have concerns about macros and records
composability and reusability.  I personally don't like excessive usage
of them in general.  By adding more macros, already quite complex guix
services mechanism becomes even more harder to learn, inspect, reason
about and work with.  In addition to that it has a major technical issue
mentioned above.  I'm strongly against this change and would suggest to
revert it.

I hope it doesn't sound rude and I'm really thankful for your work on
this, but I just think it's not the right solution, at least yet, in its
current form.

[1]: https://yhetil.org/guix-devel/878rvp1deg.fsf@trop.in/T/#u
[2]: https://yhetil.org/87ild88kax.fsf@trop.in
Ludovic Courtès Aug. 22, 2023, 4:25 p.m. UTC | #4
Hi Andrew,

Andrew Tropin <andrew@trop.in> skribis:

> Sorry for comming late to the party, I saw this message only a week ago
> and didn't have time to make an extensive reply yet, so I will share my
> quick thought on the most problematic part and maybe later will
> formulate others thoughts in more details.
>
> define-service-type-mapping looks imperative and potentially very
> problematic.  Collecting those values in unknown order and applying this
> implicit transformation making a good room for foot shooting.  Imagine
> someone would like to use his own (let's say) shepherd home service
> implementation and will add this to one of the source files of their
> channel:
>
> (define-service-type-mapping
>   shepherd-root-service-type => my-home-shepherd-service-type)
>
> What happens if somebody will use his channel just for getting some
> package?  Very likely it would break the build or in the worst case it
> will build with unexpected service implementation under the hood.

Yes, this is always possible.  Though it’s not very different from:

  (set! home-shepherd-service-type …)

Maybe the unintended effect is more likely to happen unwillingly though,
maybe.

Do you have other solutions in mind, be it for this specific issue or
for system/home service mapping in general?

> I had [1][2] and still have concerns about macros and records
> composability and reusability.  I personally don't like excessive usage
> of them in general.  By adding more macros, already quite complex guix
> services mechanism becomes even more harder to learn, inspect, reason
> about and work with.  In addition to that it has a major technical issue
> mentioned above.  I'm strongly against this change and would suggest to
> revert it.
>
> I hope it doesn't sound rude and I'm really thankful for your work on
> this, but I just think it's not the right solution, at least yet, in its
> current form.

It does sound a bit rude.  :-)  I would have loved to get any feedback
from you while we were discussing this in the course of reviewing the
Syncthing and Dicod patches a couple of months ago (which I believe you
were Cc’d on, as member of the Home team).

I won’t argue about macros and records, it’s off-topic: macros and
records are part of the Schemer’s toolbox, we try and use them wisely,
and Guix code has always used macros and records.  We can discuss
whether a specific macro or record type is suitable, of course, but
general statements about them are unhelpful.

Was it ‘for-home’ that triggered your comment?

The patches do not introduce any new record type IIRC; what triggered
your comment regarding records?

I’m all for making changes to improve on this patch series.  I’m against
reverting the patch series: the conditions for reverting are not met
(info "(guix) Commit Access").

Thanks for your feedback,
Ludo’.
Andrew Tropin Aug. 25, 2023, 6:28 a.m. UTC | #5
On 2023-08-22 18:25, Ludovic Courtès wrote:

> Hi Andrew,
>
> Andrew Tropin <andrew@trop.in> skribis:
>
>> Sorry for comming late to the party, I saw this message only a week ago
>> and didn't have time to make an extensive reply yet, so I will share my
>> quick thought on the most problematic part and maybe later will
>> formulate others thoughts in more details.
>>
>> define-service-type-mapping looks imperative and potentially very
>> problematic.  Collecting those values in unknown order and applying this
>> implicit transformation making a good room for foot shooting.  Imagine
>> someone would like to use his own (let's say) shepherd home service
>> implementation and will add this to one of the source files of their
>> channel:
>>
>> (define-service-type-mapping
>>   shepherd-root-service-type => my-home-shepherd-service-type)
>>
>> What happens if somebody will use his channel just for getting some
>> package?  Very likely it would break the build or in the worst case it
>> will build with unexpected service implementation under the hood.
>
> Yes, this is always possible.  Though it’s not very different from:
>
>   (set! home-shepherd-service-type …)

Yes, and and this is exactly what this solution does and in addition to
that it hides it under the sweet define-service-type-mapping interface.
`set!` explicitly communicates the danger of usage,
define-service-type-mapping does not.


1. We introduce a global state here and infect potentially all the
modules from all channels with it + mask it with nice interface. => Now
the result of evaluation depends on the order source files are read.
Every channel can break valid user's configurations with perfectly legal
public guix API.  We make reloading of the modules and REPL-driven
development in general a huge pain in the lower back.


2. The service extension mechanism is already quite complex to understand
on its own, in addition to that the devirsity of record creation macros
/ DSLs (define-record-type, define-record-type*, define-configuration)
doesn't make the situation better.  We introduce one more DSL on top of
couple existing. => Learning curve raises even higher. Inspecting,
navigating and debugging such code becomes harder. It prevents future
extension with for-container or for-development-environment.  It saves
couple of lines and avoids some minor repetions at the moment, but not
sure that it's the right way to reuse the stuff between home and system
services.

>
> Maybe the unintended effect is more likely to happen unwillingly though,
> maybe.
>
> Do you have other solutions in mind, be it for this specific issue or
> for system/home service mapping in general?
>
>> I had [1][2] and still have concerns about macros and records
>> composability and reusability.  I personally don't like excessive usage
>> of them in general.  By adding more macros, already quite complex guix
>> services mechanism becomes even more harder to learn, inspect, reason
>> about and work with.  In addition to that it has a major technical issue
>> mentioned above.  I'm strongly against this change and would suggest to
>> revert it.
>>
>> I hope it doesn't sound rude and I'm really thankful for your work on
>> this, but I just think it's not the right solution, at least yet, in its
>> current form.
>
> It does sound a bit rude.  :-)  I would have loved to get any feedback
> from you while we were discussing this in the course of reviewing the
> Syncthing and Dicod patches a couple of months ago (which I believe you
> were Cc’d on, as member of the Home team).
>
> I won’t argue about macros and records, it’s off-topic: macros and
> records are part of the Schemer’s toolbox, we try and use them wisely,
> and Guix code has always used macros and records.  We can discuss
> whether a specific macro or record type is suitable, of course, but
> general statements about them are unhelpful.
>

Actually, it's quite on-topic IMO and it seems as a consequence of the
abandoned discussions I linked earlier, but it's a long story and we can
keep it aside for now, because despite the reasons lead to this solution
it has fundamental problems on its own we better to focus on.  Maybe
later I'll elaborate on my thought process in more details.

> Was it ‘for-home’ that triggered your comment?
>
> The patches do not introduce any new record type IIRC; what triggered
> your comment regarding records?
>
> I’m all for making changes to improve on this patch series.  I’m against
> reverting the patch series: the conditions for reverting are not met
> (info "(guix) Commit Access").
>
> Thanks for your feedback,
> Ludo’.

Please treat it with adjustment to my not very proficient level of
english and lacking ability to express nuances.  It's not a personal
critique in any way, it's my roughly formulated concerns on 1. technical
aspects of the solution and 2. possible consequences for the future of
guix and its user-facing API.
Andrew Tropin Sept. 8, 2023, 12:42 p.m. UTC | #6
On 2023-08-25 10:28, Andrew Tropin wrote:

> On 2023-08-22 18:25, Ludovic Courtès wrote:
>
>> Hi Andrew,
>>
>> Andrew Tropin <andrew@trop.in> skribis:
>>
>>> Sorry for comming late to the party, I saw this message only a week ago
>>> and didn't have time to make an extensive reply yet, so I will share my
>>> quick thought on the most problematic part and maybe later will
>>> formulate others thoughts in more details.
>>>
>>> define-service-type-mapping looks imperative and potentially very
>>> problematic.  Collecting those values in unknown order and applying this
>>> implicit transformation making a good room for foot shooting.  Imagine
>>> someone would like to use his own (let's say) shepherd home service
>>> implementation and will add this to one of the source files of their
>>> channel:
>>>
>>> (define-service-type-mapping
>>>   shepherd-root-service-type => my-home-shepherd-service-type)
>>>
>>> What happens if somebody will use his channel just for getting some
>>> package?  Very likely it would break the build or in the worst case it
>>> will build with unexpected service implementation under the hood.
>>
>> Yes, this is always possible.  Though it’s not very different from:
>>
>>   (set! home-shepherd-service-type …)
>
> Yes, and and this is exactly what this solution does and in addition to
> that it hides it under the sweet define-service-type-mapping interface.
> `set!` explicitly communicates the danger of usage,
> define-service-type-mapping does not.
>
>
> 1. We introduce a global state here and infect potentially all the
> modules from all channels with it + mask it with nice interface. =>

To support my previous suggestion to revert this patch series I will
document the example of a consequence:

https://git.savannah.gnu.org/cgit/guix.git/commit/?h=cf6abf50dbbbd95fef465ab4bb3b608843ac47e1
https://issues.guix.gnu.org/65510

It highlights this:

> Now the result of evaluation depends on the order source files are
> read.
>>> Collecting those values in unknown order and applying this
>>> implicit transformation making a good room for foot shooting.

to say it differently this change introduced the implicit dependency on
the fact of the evaluation of the form, which is not explicitly
referenced anywhere.

> Every channel can break valid user's configurations with perfectly
> legal public guix API.  We make reloading of the modules and
> REPL-driven development in general a huge pain in the lower back.
>
> 2. The service extension mechanism is already quite complex to understand
> on its own, in addition to that the devirsity of record creation macros
> / DSLs (define-record-type, define-record-type*, define-configuration)
> doesn't make the situation better.  We introduce one more DSL on top of
> couple existing. => Learning curve raises even higher. Inspecting,
> navigating and debugging such code becomes harder. It prevents future
> extension with for-container or for-development-environment.  It saves
> couple of lines and avoids some minor repetions at the moment, but not
> sure that it's the right way to reuse the stuff between home and system
> services.
>
>>
>> Maybe the unintended effect is more likely to happen unwillingly though,
>> maybe.
>>
>> Do you have other solutions in mind, be it for this specific issue or
>> for system/home service mapping in general?
>>
>>> I had [1][2] and still have concerns about macros and records
>>> composability and reusability.  I personally don't like excessive usage
>>> of them in general.  By adding more macros, already quite complex guix
>>> services mechanism becomes even more harder to learn, inspect, reason
>>> about and work with.  In addition to that it has a major technical issue
>>> mentioned above.  I'm strongly against this change and would suggest to
>>> revert it.
>>>
>>> I hope it doesn't sound rude and I'm really thankful for your work on
>>> this, but I just think it's not the right solution, at least yet, in its
>>> current form.
>>
>> It does sound a bit rude.  :-)  I would have loved to get any feedback
>> from you while we were discussing this in the course of reviewing the
>> Syncthing and Dicod patches a couple of months ago (which I believe you
>> were Cc’d on, as member of the Home team).
>>
>> I won’t argue about macros and records, it’s off-topic: macros and
>> records are part of the Schemer’s toolbox, we try and use them wisely,
>> and Guix code has always used macros and records.  We can discuss
>> whether a specific macro or record type is suitable, of course, but
>> general statements about them are unhelpful.
>>
>
> Actually, it's quite on-topic IMO and it seems as a consequence of the
> abandoned discussions I linked earlier, but it's a long story and we can
> keep it aside for now, because despite the reasons lead to this solution
> it has fundamental problems on its own we better to focus on.  Maybe
> later I'll elaborate on my thought process in more details.
>
>> Was it ‘for-home’ that triggered your comment?
>>
>> The patches do not introduce any new record type IIRC; what triggered
>> your comment regarding records?
>>
>> I’m all for making changes to improve on this patch series.  I’m against
>> reverting the patch series: the conditions for reverting are not met
>> (info "(guix) Commit Access").
>>
>> Thanks for your feedback,
>> Ludo’.
>
> Please treat it with adjustment to my not very proficient level of
> english and lacking ability to express nuances.  It's not a personal
> critique in any way, it's my roughly formulated concerns on 1. technical
> aspects of the solution and 2. possible consequences for the future of
> guix and its user-facing API.
Ludovic Courtès Sept. 8, 2023, 10:18 p.m. UTC | #7
Hi Andrew,

Andrew Tropin <andrew@trop.in> skribis:

> Yes, and and this is exactly what this solution does and in addition to
> that it hides it under the sweet define-service-type-mapping interface.
> `set!` explicitly communicates the danger of usage,
> define-service-type-mapping does not.
>
>
> 1. We introduce a global state here and infect potentially all the
> modules from all channels with it + mask it with nice interface. => Now
> the result of evaluation depends on the order source files are read.
> Every channel can break valid user's configurations with perfectly legal
> public guix API.  We make reloading of the modules and REPL-driven
> development in general a huge pain in the lower back.
>
>
> 2. The service extension mechanism is already quite complex to understand
> on its own, in addition to that the devirsity of record creation macros
> / DSLs (define-record-type, define-record-type*, define-configuration)
> doesn't make the situation better.  We introduce one more DSL on top of
> couple existing. => Learning curve raises even higher. Inspecting,
> navigating and debugging such code becomes harder. It prevents future
> extension with for-container or for-development-environment.  It saves
> couple of lines and avoids some minor repetions at the moment, but not
> sure that it's the right way to reuse the stuff between home and system
> services.

I understand what you’re saying.  I don’t necessarily agree with all of
it because I believe abstraction is a fundamental part of programming;
abstractions can sometimes be wrong or misleading of course, and that’s
what we should strive to avoid.

Back to this patch series, we’ve had one concrete illustration of a
shortcoming:

  https://issues.guix.gnu.org/65510

I’m aware of this and agree it needs to be addressed.

In assessing this patch series, one should keep in mind that it solves a
longstanding issue with Guix Home (code duplication and the inability to
share service code with Guix System), one for which no other solution
was proposed AFAIK.

My own assessment, having reviewed patches adding Home services (in all
loneliness I must say) is that the outcome is positive, in spite of the
shortcoming mentioned above.

Thanks,
Ludo’.
Andrew Tropin Sept. 9, 2023, 10:42 a.m. UTC | #8
On 2023-09-09 00:18, Ludovic Courtès wrote:

> Hi Andrew,
>
> Andrew Tropin <andrew@trop.in> skribis:
>
>> Yes, and and this is exactly what this solution does and in addition to
>> that it hides it under the sweet define-service-type-mapping interface.
>> `set!` explicitly communicates the danger of usage,
>> define-service-type-mapping does not.
>>
>>
>> 1. We introduce a global state here and infect potentially all the
>> modules from all channels with it + mask it with nice interface. => Now
>> the result of evaluation depends on the order source files are read.
>> Every channel can break valid user's configurations with perfectly legal
>> public guix API.  We make reloading of the modules and REPL-driven
>> development in general a huge pain in the lower back.
>>
>>
>> 2. The service extension mechanism is already quite complex to understand
>> on its own, in addition to that the devirsity of record creation macros
>> / DSLs (define-record-type, define-record-type*, define-configuration)
>> doesn't make the situation better.  We introduce one more DSL on top of
>> couple existing. => Learning curve raises even higher. Inspecting,
>> navigating and debugging such code becomes harder. It prevents future
>> extension with for-container or for-development-environment.  It saves
>> couple of lines and avoids some minor repetions at the moment, but not
>> sure that it's the right way to reuse the stuff between home and system
>> services.
>
> I understand what you’re saying.  I don’t necessarily agree with all of
> it because I believe abstraction is a fundamental part of programming;
> abstractions can sometimes be wrong or misleading of course, and that’s
> what we should strive to avoid.
>
> Back to this patch series, we’ve had one concrete illustration of a
> shortcoming:
>
>   https://issues.guix.gnu.org/65510
>
> I’m aware of this and agree it needs to be addressed.
>
> In assessing this patch series, one should keep in mind that it solves a
> longstanding issue with Guix Home (code duplication and the inability to
> share service code with Guix System), one for which no other solution
> was proposed AFAIK.

1. One of the possible options is to use free-style configurations for
services, so we can reuse a lot of code and drastically decrease
duplication (only the service definition itself and a one configuration
record) and maintanance burden.
https://yhetil.org/guix/87ild88kax.fsf@trop.in/

We have only two maintainers and have more than the half of a hundred
services in rde, which is not that far from guix itself (258 services).



2. Another option is to rethink service extension mechanism a little bit
and maybe make it more polymorphic.
https://lists.sr.ht/~abcdw/rde-devel/%3C87sg56g97i.fsf%40trop.in%3E

(Just making a rough example of naive implementation) Instead of
referencing service by it's value, we could reference its name.  This
way we can write one service for Home and System, which always extends
'system-accounts-service-type, and for home case provide a different
implementation for it, so when we build operating system
system-accounts-service-type will add users to /etc/passwd, when we
build home-environment system-accounts-service-type will do nothing.

Adjusting the logic of the extension based on presence/absence of a
particular service can be very useful to make it possible to reuse
configuration for Home and System services directly.

However, improving service extension mechanism in such way, keeping
backward compatibility and upstreaming it is a hard and very
time-consuming task.  So in rde we just implemented a feature mechanism
on top of service extension mechanism to solve challenges mentioned
above and it works great.  It would be even greater if we could just do
the same with plain services.


My point here is: let's solve this issue on another level of
abstraction, by improving service extension mechanism, not by creating a
new level of abstraction (as we did in rde or in this patch series) to
cover the wholes in the previous level.  I would be glad to cooperate on
this and design possible improvements.

> My own assessment, having reviewed patches adding Home services (in all
> loneliness I must say) is that the outcome is positive, in spite of the
> shortcoming mentioned above.

> in all loneliness I must say

In some previous threads on guix-devel/guix-patches I mentioned, that we
will maintain home services for rde separately
https://git.sr.ht/~abcdw/rde/tree/master/item/doc/decision-log/0004-rde-flavor-guix-services.org
and thus all (most of) the home services coming to guix-patches won't be
used by rde or me personally.  Despite the fact that I would be glad to
help with reviews, I already work hard (almost fulltime) on guix, guile
and emacs ecosystem in parrallel to running from the war and probably
reviewing home services on guix-patches is less valuable use of my time
both for me and for the community.

> In assessing this patch series, one should keep in mind that it solves a
> longstanding issue with Guix Home (code duplication and the inability to
> share service code with Guix System), one for which no other solution
> was proposed AFAIK.

I think that your concern about code duplication is valid (and I'm aware
of the issue probably even longer :) ) and I would be glad to cooperate
to solve it.  However, I'm almost sure that it should be done on service
extension mechanism level.  If this make sense, we can discuss possible
solutions in more details and next practical steps.
Ludovic Courtès Sept. 13, 2023, 6:06 p.m. UTC | #9
Hi,

Andrew Tropin <andrew@trop.in> skribis:

> In some previous threads on guix-devel/guix-patches I mentioned, that we
> will maintain home services for rde separately
> https://git.sr.ht/~abcdw/rde/tree/master/item/doc/decision-log/0004-rde-flavor-guix-services.org
> and thus all (most of) the home services coming to guix-patches won't be
> used by rde or me personally.  Despite the fact that I would be glad to
> help with reviews, I already work hard (almost fulltime) on guix, guile
> and emacs ecosystem in parrallel to running from the war and probably
> reviewing home services on guix-patches is less valuable use of my time
> both for me and for the community.

Then, for clarity and to avoid misunderstandings, perhaps you should
remove yourself from the Home team?

I’m disappointed that rde/Guix cooperation is a one-way street.

Ludo’.
Ludovic Courtès Sept. 13, 2023, 7:55 p.m. UTC | #10
Andrew Tropin <andrew@trop.in> skribis:

> 1. One of the possible options is to use free-style configurations for
> services, so we can reuse a lot of code and drastically decrease
> duplication (only the service definition itself and a one configuration
> record) and maintanance burden.

We’ve had this discussion before; in the case of key/value-style
configuration, I’m against “free-style configuration” (nginx
configuration is a bit different).

With free-style configuration (I assume you have alists/sexps in mind),
you might end up generating invalid config files, or get obscure error
messages, or whatever; I’ve used Gnus for decades (!), any Gnus user
will know what I mean.  This is Scheme, not Emacs Lisp/Common Lisp.

[...]

> 2. Another option is to rethink service extension mechanism a little bit
> and maybe make it more polymorphic.
> https://lists.sr.ht/~abcdw/rde-devel/%3C87sg56g97i.fsf%40trop.in%3E

I very much support experimentation in this area; I’m sure there’s room
for improvement (like Julien’s extension points, which you referred to
in the message above).

I’m not sure how that relates to factorizing Home/System services though.

[...]

> My point here is: let's solve this issue on another level of
> abstraction, by improving service extension mechanism, not by creating a
> new level of abstraction (as we did in rde or in this patch series) to
> cover the wholes in the previous level.

Do you have ideas on how you’d change service extension to support
Home/System factorization?

> I would be glad to cooperate on this and design possible improvements.

I interpret this sentence as you suggesting that you’re outside the
project—even though my perception is that you’ve been part of it for
some time, even before you got commit rights.  So do feel at Home!
(Pun intended. :-))

Ludo’.
Andrew Tropin Sept. 17, 2023, 5:28 a.m. UTC | #11
On 2023-09-13 20:06, Ludovic Courtès wrote:

> Hi,
>
> Andrew Tropin <andrew@trop.in> skribis:
>
>> In some previous threads on guix-devel/guix-patches I mentioned, that we
>> will maintain home services for rde separately
>> https://git.sr.ht/~abcdw/rde/tree/master/item/doc/decision-log/0004-rde-flavor-guix-services.org
>> and thus all (most of) the home services coming to guix-patches won't be
>> used by rde or me personally.  Despite the fact that I would be glad to
>> help with reviews, I already work hard (almost fulltime) on guix, guile
>> and emacs ecosystem in parrallel to running from the war and probably
>> reviewing home services on guix-patches is less valuable use of my time
>> both for me and for the community.
>
> I’m disappointed that rde/Guix cooperation is a one-way street.

That's probably a wrong statement.  rde creates, integrates and
innovates a bunch of good tools, tests and often upstreams them;
contributes back a lot of changes and packages, popularizes and
encriches guix and neighbour ecosystems and helps to finish stale
projects/issues in guix mailing lists, ranging from Wayland, Emacs,
Pipewire, iwd related to Tree-sitter, Guix Home and couple other
non-trivial thing.

rde definetly benifit from all the great work done in Guix, but it's
kinda offensive to say that it's one-way street.

> Then, for clarity and to avoid misunderstandings, perhaps you should
> remove yourself from the Home team?
>

I guess this one I've already sent to you an year ago is still true:
--8<---------------cut here---------------start------------->8---
    However, I perfectly understand the reasons for Guix System style
    services and respect your descion and desire to stick with them.  I
    mentioned that I'll maintain already existing rde style services in rde
    and it won't make sense for me to upstream most of them from rde to
    Guix, because it will mean almost complete rewrite for them.

    And this is another reason I don't participate in development of home
    services, because it will mean double or even triple work for me,
    however I keep an eye on patches you send, in most cases someone jumps
    in earlier with review, so I don't bother you with my comments. But
    development of Guix Home itself and the rest of Guix is a part of my
    interests.
--8<---------------cut here---------------end--------------->8---

I do want to get CCed on general Guix Home topics, I'm ok with getting
CCed on other guix home patches and threads outside of my
interests/workload capabilities.

If you think that confusion is severe, we can split the team into Guix
Home Core and Guix Home Services, however I don't think it makes a lot
of sence.
Andrew Tropin Sept. 17, 2023, 7:01 a.m. UTC | #12
On 2023-09-13 21:55, Ludovic Courtès wrote:

> Andrew Tropin <andrew@trop.in> skribis:
>
>> 1. One of the possible options is to use free-style configurations for
>> services, so we can reuse a lot of code and drastically decrease
>> duplication (only the service definition itself and a one configuration
>> record) and maintanance burden.
>
> We’ve had this discussion before; in the case of key/value-style
> configuration, I’m against “free-style configuration” (nginx
> configuration is a bit different).
>
> With free-style configuration (I assume you have alists/sexps in mind),
> you might end up generating invalid config files, or get obscure error
> messages, or whatever; I’ve used Gnus for decades (!), any Gnus user
> will know what I mean.  This is Scheme, not Emacs Lisp/Common Lisp.
>
> [...]
>
>> 2. Another option is to rethink service extension mechanism a little bit
>> and maybe make it more polymorphic.
>> https://lists.sr.ht/~abcdw/rde-devel/%3C87sg56g97i.fsf%40trop.in%3E
>
> I very much support experimentation in this area; I’m sure there’s room
> for improvement (like Julien’s extension points, which you referred to
> in the message above).
>
> I’m not sure how that relates to factorizing Home/System services though.
>
> [...]
>
>> My point here is: let's solve this issue on another level of
>> abstraction, by improving service extension mechanism, not by creating a
>> new level of abstraction (as we did in rde or in this patch series) to
>> cover the wholes in the previous level.
>
> Do you have ideas on how you’d change service extension to support
> Home/System factorization?
>

I'll write an example to demonstrate the high-level idea:

We have
--8<---------------cut here---------------start------------->8---
(extensions
 (list (service-extension account-service-type
                          (const %dicod-accounts))
       (service-extension shepherd-root-service-type
                          dicod-shepherd-service)))
--8<---------------cut here---------------end--------------->8---

We can do
--8<---------------cut here---------------start------------->8---
(home-environment
 (services
  (list (service dicod-service-type)))
 (service-mapping
  `((,shepherd-root-service-type . ,home-shepherd-service-type)
    (,account-service-type . ,ignore-service-type))))
--8<---------------cut here---------------end--------------->8---

Now you can use the same service type and configuration record in
both operating-system and home-environment and use/ignore/interpret the
configuration fields values based on the environment we are building
this thing for.

Of course service-mapping default value can be generated upfront and
stored in %guix-home-service-mapping or something like that.

This way we will get rid of almost all home- services and related
configurations.

Keep in mind that it's a raw idea, not a well-designed solution yet.

>> I would be glad to cooperate on this and design possible improvements.
>
> I interpret this sentence as you suggesting that you’re outside the
> project—even though my perception is that you’ve been part of it for
> some time, even before you got commit rights.  So do feel at Home!
> (Pun intended. :-))

My point was that rde home and system services are not affected much by
code duplication due to factorization through free-style serializers and
this problem is low priority for me at the moment.  However in long term
the proper solution would be benefitial for both guix and rde.

The reason I'm suggesting to revert this patch series is because it
feels to me as a partial solution on the wrong level of abstraction,
which cures the symptoms and also introduces shortcomings and
potentially makes it harder to introduce a proper solution in the
future.

If this problem is urgent/demanding for you, please invite me for
working or collaborating on the solution explicitly, I plan work on it
anyway, but probably not earlier than the next year.  I catched this
thread accidentially BTW, thanks to 宋文武 for CCing me.
Ludovic Courtès Sept. 17, 2023, 10:27 a.m. UTC | #13
Hi

Andrew Tropin <andrew@trop.in> skribis:

> On 2023-09-13 20:06, Ludovic Courtès wrote:
>
>> Hi,
>>
>> Andrew Tropin <andrew@trop.in> skribis:
>>
>>> In some previous threads on guix-devel/guix-patches I mentioned, that we
>>> will maintain home services for rde separately

[...]

>> I’m disappointed that rde/Guix cooperation is a one-way street.
>
> That's probably a wrong statement.

That was a reaction to “we will maintain home services for rde
separately” above; to me that statement was unambiguous.

[...]

>> Then, for clarity and to avoid misunderstandings, perhaps you should
>> remove yourself from the Home team?
>>
>
> I guess this one I've already sent to you an year ago is still true:
>
>     However, I perfectly understand the reasons for Guix System style
>     services and respect your descion and desire to stick with them.  I
>     mentioned that I'll maintain already existing rde style services in rde
>     and it won't make sense for me to upstream most of them from rde to
>     Guix, because it will mean almost complete rewrite for them.
>
>     And this is another reason I don't participate in development of home
>     services, because it will mean double or even triple work for me,
>     however I keep an eye on patches you send, in most cases someone jumps
>     in earlier with review, so I don't bother you with my comments. But
>     development of Guix Home itself and the rest of Guix is a part of my
>     interests.
>
> I do want to get CCed on general Guix Home topics, I'm ok with getting
> CCed on other guix home patches and threads outside of my
> interests/workload capabilities.
>
> If you think that confusion is severe, we can split the team into Guix
> Home Core and Guix Home Services, however I don't think it makes a lot
> of sence.

To me, team members commit to the best of their capacity to help with
patch review in the team’s scope.  To receive notifications about a
given topic, I’d suggest using email filters or similar tools.

I hope that makes sense!

Ludo’.
Ludovic Courtès Oct. 13, 2023, 4:05 p.m. UTC | #14
Hi Andrew,

Andrew Tropin <andrew@trop.in> skribis:

> We have
>
> (extensions
>  (list (service-extension account-service-type
>                           (const %dicod-accounts))
>        (service-extension shepherd-root-service-type
>                           dicod-shepherd-service)))
>
>
> We can do
>
> (home-environment
>  (services
>   (list (service dicod-service-type)))
>  (service-mapping
>   `((,shepherd-root-service-type . ,home-shepherd-service-type)
>     (,account-service-type . ,ignore-service-type))))

As a user writing the ‘home-environment’ declaration, I don’t want to
know whether a Home service is implemented by mapping a System service
to Home, or whether it’s a fully separate implementation.

To me, users shouldn’t have to specify service mappings at all; service
mapping is a tool at the disposal of service writers.

> Now you can use the same service type and configuration record in
> both operating-system and home-environment and use/ignore/interpret the
> configuration fields values based on the environment we are building
> this thing for.
>
> Of course service-mapping default value can be generated upfront and
> stored in %guix-home-service-mapping or something like that.
>
> This way we will get rid of almost all home- services and related
> configurations.
>
> Keep in mind that it's a raw idea, not a well-designed solution yet.

The good thing is that we seem to agree on the general idea of having a
way to map System services to Home.

My take is that the mapping is an implementation detail that users do
not need to be aware of.

> The reason I'm suggesting to revert this patch series is because it
> feels to me as a partial solution on the wrong level of abstraction,
> which cures the symptoms and also introduces shortcomings and
> potentially makes it harder to introduce a proper solution in the
> future.

I disagree with this assessment.

> If this problem is urgent/demanding for you, please invite me for
> working or collaborating on the solution explicitly, I plan work on it
> anyway, but probably not earlier than the next year.  I catched this
> thread accidentially BTW, thanks to 宋文武 for CCing me.

This problem has been discussed pretty much since Home was merged in
Guix¹; we’ve all had ample time to think about it and to my knowledge
this patch series is the only proposal that was ever brought.

I’ll close this bug for clarity.  I remain open to discussion on the
implementation of System -> Home service mapping.  I think further
discussion should happen in the context of incremental changes to the
implementation of that mapping mechanism, in a dedicated issue.

Thank you,
Ludo’.

¹ The cover letter at <https://issues.guix.gnu.org/65119> mentions two
  services that would have been duplicated if we didn’t have this
  mechanism.
Andrew Tropin Oct. 14, 2023, 6:03 a.m. UTC | #15
On 2023-10-13 18:05, Ludovic Courtès wrote:

> Hi Andrew,
>
> Andrew Tropin <andrew@trop.in> skribis:
>
>> We have
>>
>> (extensions
>>  (list (service-extension account-service-type
>>                           (const %dicod-accounts))
>>        (service-extension shepherd-root-service-type
>>                           dicod-shepherd-service)))
>>
>>
>> We can do
>>
>> (home-environment
>>  (services
>>   (list (service dicod-service-type)))
>>  (service-mapping
>>   `((,shepherd-root-service-type . ,home-shepherd-service-type)
>>     (,account-service-type . ,ignore-service-type))))
>
> As a user writing the ‘home-environment’ declaration, I don’t want to
> know whether a Home service is implemented by mapping a System service
> to Home, or whether it’s a fully separate implementation.
>
> To me, users shouldn’t have to specify service mappings at all; service
> mapping is a tool at the disposal of service writers.
>
>> Now you can use the same service type and configuration record in
>> both operating-system and home-environment and use/ignore/interpret the
>> configuration fields values based on the environment we are building
>> this thing for.
>>
>> Of course service-mapping default value can be generated upfront and
>> stored in %guix-home-service-mapping or something like that.
>>
>> This way we will get rid of almost all home- services and related
>> configurations.
>>
>> Keep in mind that it's a raw idea, not a well-designed solution yet.
>
> The good thing is that we seem to agree on the general idea of having a
> way to map System services to Home.
>
> My take is that the mapping is an implementation detail that users do
> not need to be aware of.
>
>> The reason I'm suggesting to revert this patch series is because it
>> feels to me as a partial solution on the wrong level of abstraction,
>> which cures the symptoms and also introduces shortcomings and
>> potentially makes it harder to introduce a proper solution in the
>> future.
>
> I disagree with this assessment.
>
>> If this problem is urgent/demanding for you, please invite me for
>> working or collaborating on the solution explicitly, I plan work on it
>> anyway, but probably not earlier than the next year.  I catched this
>> thread accidentially BTW, thanks to 宋文武 for CCing me.
>
> This problem has been discussed pretty much since Home was merged in
> Guix¹; we’ve all had ample time to think about it and to my knowledge
> this patch series is the only proposal that was ever brought.
>
> I’ll close this bug for clarity.  I remain open to discussion on the
> implementation of System -> Home service mapping.  I think further
> discussion should happen in the context of incremental changes to the
> implementation of that mapping mechanism, in a dedicated issue.

👌