diff mbox series

[bug#56867] download: Do not wrap TLS port on GnuTLS >= 3.7.7.

Message ID 87pmhgks4j.fsf@gnu.org
State Accepted
Headers show
Series [bug#56867] download: Do not wrap TLS port on GnuTLS >= 3.7.7. | expand

Checks

Context Check Description
cbaines/applying patch fail View Laminar job
cbaines/issue success View issue

Commit Message

Ludovic Courtès Aug. 4, 2022, 2:20 p.m. UTC 
Ludovic Courtès <ludo@gnu.org> skribis:

> The custom input/output port wrapping the TLS session record port would
> introduce overhead, and it would also prevent its uses in a non-blocking
> context--e.g., with Fibers.  The port close mechanism added in GnuTLS
> 3.7.7 allows us to get rid of that wrapper.
>
> * guix/build/download.scm (wrap-record-port-for-gnutls<3.7.7): New
> procedure, with code formerly in 'tls-wrap'.
> (tls-wrap): Check for 'set-session-record-port-close!' and use it when
> available; otherwise call 'wrap-record-port-for-gnutls<3.7.7'.

I synchronized Guile's copy of this code:

  317b06bf8 web: 'tls-wrap' retries handshake upon non-fatal errors.
  c01ca10b3 web: Do not wrap TLS port on GnuTLS >= 3.7.7.

I realized that’s not enough to make it possible to use non-blocking
ports though.

First, I noticed that GnuTLS doesn’t implement ‘write_wait_fd’, only
‘read_wait_fd’ (not sure how problematic that is):

--8<---------------cut here---------------start------------->8---
scheme@(guile-user)> ,use(web client)
scheme@(guile-user)> (define p (open-socket-for-uri "https://guix.gnu.org"))
scheme@(guile-user)> ((@@ (ice-9 suspendable-ports) wait-for-writable) p)
ice-9/boot-9.scm:1685:16: In procedure raise-exception:
In procedure write_wait_fd: unimplemented

Entering a new prompt.  Type `,bt' for a backtrace or `,q' to continue.
scheme@(guile-user) [1]> ,q
scheme@(guile-user)> ,use(gnutls)
scheme@(guile-user)> (gnutls-version)
$1 = "3.7.7"
scheme@(guile-user)> ((@@ (ice-9 suspendable-ports) wait-for-readable) p)
$2 = 1
--8<---------------cut here---------------end--------------->8---

Second, ‘open-socket-for-uri’ creates a blocking socket and uses that as
the backing file descriptor of the TLS session.

We’d need a way to pass flags for the ‘socket’ call made by
‘open-socket-for-uri’ so we can pass O_NONBLOCK, maybe as show below:
… which lets us do that:

--8<---------------cut here---------------start------------->8---
scheme@(guile-user)> ,use(web client)
scheme@(guile-user)> (define p (open-socket-for-uri "https://guix.gnu.org" #:flags O_NONBLOCK))
scheme@(guile-user)> (http-get "https://guix.gnu.org" #:port p)
--8<---------------cut here---------------end--------------->8---

Thoughts?

Ludo’.

Comments

Thiago Jung Bauermann Aug. 4, 2022, 2:46 p.m. UTC | #1 
Hello Ludo,

I don't have any comment/insight on what you're doing in general, except
about one of your points below:

Ludovic Courtès <ludo@gnu.org> writes:

> First, I noticed that GnuTLS doesn’t implement ‘write_wait_fd’, only
> ‘read_wait_fd’ (not sure how problematic that is):
>
> scheme@(guile-user)> ,use(web client)
> scheme@(guile-user)> (define p (open-socket-for-uri "https://guix.gnu.org"))
> scheme@(guile-user)> ((@@ (ice-9 suspendable-ports) wait-for-writable) p)
> ice-9/boot-9.scm:1685:16: In procedure raise-exception:
> In procedure write_wait_fd: unimplemented
>
> Entering a new prompt.  Type `,bt' for a backtrace or `,q' to continue.
> scheme@(guile-user) [1]> ,q
> scheme@(guile-user)> ,use(gnutls)
> scheme@(guile-user)> (gnutls-version)
> $1 = "3.7.7"
> scheme@(guile-user)> ((@@ (ice-9 suspendable-ports) wait-for-readable) p)
> $2 = 1

This occasionally causes problems when fetching substitutes, as can be
seen in bug #56005 (during substitution: write_wait_fd: unimplemented).
Ludovic Courtès Aug. 4, 2022, 4:19 p.m. UTC | #2 
Hi,

Thiago Jung Bauermann <bauermann@kolabnow.com> skribis:

> Ludovic Courtès <ludo@gnu.org> writes:
>
>> First, I noticed that GnuTLS doesn’t implement ‘write_wait_fd’, only
>> ‘read_wait_fd’ (not sure how problematic that is):
>>
>> scheme@(guile-user)> ,use(web client)
>> scheme@(guile-user)> (define p (open-socket-for-uri "https://guix.gnu.org"))
>> scheme@(guile-user)> ((@@ (ice-9 suspendable-ports) wait-for-writable) p)
>> ice-9/boot-9.scm:1685:16: In procedure raise-exception:
>> In procedure write_wait_fd: unimplemented
>>
>> Entering a new prompt.  Type `,bt' for a backtrace or `,q' to continue.
>> scheme@(guile-user) [1]> ,q
>> scheme@(guile-user)> ,use(gnutls)
>> scheme@(guile-user)> (gnutls-version)
>> $1 = "3.7.7"
>> scheme@(guile-user)> ((@@ (ice-9 suspendable-ports) wait-for-readable) p)
>> $2 = 1
>
> This occasionally causes problems when fetching substitutes, as can be
> seen in bug #56005 (during substitution: write_wait_fd: unimplemented).

Oh, I have not seen it but it’s weird: (guix scripts substitute) doesn’t
use O_NONBLOCK sockets, so I don’t get how it can hit that.  Needs
investigation…

Thanks,
Ludo’.
diff mbox series

Patch

diff --git a/module/web/client.scm b/module/web/client.scm
index a08c4203c..9273a45ad 100644
--- a/module/web/client.scm
+++ b/module/web/client.scm
@@ -320,7 +320,8 @@  host name without trailing dot."
   (read-response port))
 
 (define* (open-socket-for-uri uri-or-string
-                              #:key (verify-certificate? #t))
+                              #:key (verify-certificate? #t)
+                              (flags 0))
   "Return an open input/output port for a connection to URI-OR-STRING.
 When VERIFY-CERTIFICATE? is true, verify HTTPS server certificates."
   (define uri
@@ -373,10 +374,18 @@  When VERIFY-CERTIFICATE? is true, verify HTTPS server certificates."
     (when (and https? (current-https-proxy))
       (setup-http-tunnel s uri))
 
-    (if https?
-        (tls-wrap s (uri-host uri)
-                  #:verify-certificate? verify-certificate?)
-        s)))
+    (let ((port (if https?
+                    (tls-wrap s (uri-host uri)
+                              #:verify-certificate? verify-certificate?)
+                    s)))
+      (unless (zero? flags)
+        ;; FLAGS might contain O_NONBLOCK.  Thus, set it as a last step
+        ;; because 'handshake' otherwise throws an exception for
+        ;; GNUTLS_E_AGAIN.
+        (let ((initial-flags (fcntl s F_GETFL)))
+          (fcntl s F_SETFL (logior initial-flags flags))))
+
+      port)))
 
 (define (extend-request r k v . additional)
   (let ((r (set-field r (request-headers)