[bug#56867] download: Do not wrap TLS port on GnuTLS >= 3.7.7.
Commit Message
Ludovic Courtès <ludo@gnu.org> skribis:
> The custom input/output port wrapping the TLS session record port would
> introduce overhead, and it would also prevent its uses in a non-blocking
> context--e.g., with Fibers. The port close mechanism added in GnuTLS
> 3.7.7 allows us to get rid of that wrapper.
>
> * guix/build/download.scm (wrap-record-port-for-gnutls<3.7.7): New
> procedure, with code formerly in 'tls-wrap'.
> (tls-wrap): Check for 'set-session-record-port-close!' and use it when
> available; otherwise call 'wrap-record-port-for-gnutls<3.7.7'.
I synchronized Guile's copy of this code:
317b06bf8 web: 'tls-wrap' retries handshake upon non-fatal errors.
c01ca10b3 web: Do not wrap TLS port on GnuTLS >= 3.7.7.
I realized that’s not enough to make it possible to use non-blocking
ports though.
First, I noticed that GnuTLS doesn’t implement ‘write_wait_fd’, only
‘read_wait_fd’ (not sure how problematic that is):
--8<---------------cut here---------------start------------->8---
scheme@(guile-user)> ,use(web client)
scheme@(guile-user)> (define p (open-socket-for-uri "https://guix.gnu.org"))
scheme@(guile-user)> ((@@ (ice-9 suspendable-ports) wait-for-writable) p)
ice-9/boot-9.scm:1685:16: In procedure raise-exception:
In procedure write_wait_fd: unimplemented
Entering a new prompt. Type `,bt' for a backtrace or `,q' to continue.
scheme@(guile-user) [1]> ,q
scheme@(guile-user)> ,use(gnutls)
scheme@(guile-user)> (gnutls-version)
$1 = "3.7.7"
scheme@(guile-user)> ((@@ (ice-9 suspendable-ports) wait-for-readable) p)
$2 = 1
--8<---------------cut here---------------end--------------->8---
Second, ‘open-socket-for-uri’ creates a blocking socket and uses that as
the backing file descriptor of the TLS session.
We’d need a way to pass flags for the ‘socket’ call made by
‘open-socket-for-uri’ so we can pass O_NONBLOCK, maybe as show below:
… which lets us do that:
--8<---------------cut here---------------start------------->8---
scheme@(guile-user)> ,use(web client)
scheme@(guile-user)> (define p (open-socket-for-uri "https://guix.gnu.org" #:flags O_NONBLOCK))
scheme@(guile-user)> (http-get "https://guix.gnu.org" #:port p)
--8<---------------cut here---------------end--------------->8---
Thoughts?
Ludo’.
Comments
Hello Ludo,
I don't have any comment/insight on what you're doing in general, except
about one of your points below:
Ludovic Courtès <ludo@gnu.org> writes:
> First, I noticed that GnuTLS doesn’t implement ‘write_wait_fd’, only
> ‘read_wait_fd’ (not sure how problematic that is):
>
> scheme@(guile-user)> ,use(web client)
> scheme@(guile-user)> (define p (open-socket-for-uri "https://guix.gnu.org"))
> scheme@(guile-user)> ((@@ (ice-9 suspendable-ports) wait-for-writable) p)
> ice-9/boot-9.scm:1685:16: In procedure raise-exception:
> In procedure write_wait_fd: unimplemented
>
> Entering a new prompt. Type `,bt' for a backtrace or `,q' to continue.
> scheme@(guile-user) [1]> ,q
> scheme@(guile-user)> ,use(gnutls)
> scheme@(guile-user)> (gnutls-version)
> $1 = "3.7.7"
> scheme@(guile-user)> ((@@ (ice-9 suspendable-ports) wait-for-readable) p)
> $2 = 1
This occasionally causes problems when fetching substitutes, as can be
seen in bug #56005 (during substitution: write_wait_fd: unimplemented).
Hi,
Thiago Jung Bauermann <bauermann@kolabnow.com> skribis:
> Ludovic Courtès <ludo@gnu.org> writes:
>
>> First, I noticed that GnuTLS doesn’t implement ‘write_wait_fd’, only
>> ‘read_wait_fd’ (not sure how problematic that is):
>>
>> scheme@(guile-user)> ,use(web client)
>> scheme@(guile-user)> (define p (open-socket-for-uri "https://guix.gnu.org"))
>> scheme@(guile-user)> ((@@ (ice-9 suspendable-ports) wait-for-writable) p)
>> ice-9/boot-9.scm:1685:16: In procedure raise-exception:
>> In procedure write_wait_fd: unimplemented
>>
>> Entering a new prompt. Type `,bt' for a backtrace or `,q' to continue.
>> scheme@(guile-user) [1]> ,q
>> scheme@(guile-user)> ,use(gnutls)
>> scheme@(guile-user)> (gnutls-version)
>> $1 = "3.7.7"
>> scheme@(guile-user)> ((@@ (ice-9 suspendable-ports) wait-for-readable) p)
>> $2 = 1
>
> This occasionally causes problems when fetching substitutes, as can be
> seen in bug #56005 (during substitution: write_wait_fd: unimplemented).
Oh, I have not seen it but it’s weird: (guix scripts substitute) doesn’t
use O_NONBLOCK sockets, so I don’t get how it can hit that. Needs
investigation…
Thanks,
Ludo’.
@@ -320,7 +320,8 @@ host name without trailing dot."
(read-response port))
(define* (open-socket-for-uri uri-or-string
- #:key (verify-certificate? #t))
+ #:key (verify-certificate? #t)
+ (flags 0))
"Return an open input/output port for a connection to URI-OR-STRING.
When VERIFY-CERTIFICATE? is true, verify HTTPS server certificates."
(define uri
@@ -373,10 +374,18 @@ When VERIFY-CERTIFICATE? is true, verify HTTPS server certificates."
(when (and https? (current-https-proxy))
(setup-http-tunnel s uri))
- (if https?
- (tls-wrap s (uri-host uri)
- #:verify-certificate? verify-certificate?)
- s)))
+ (let ((port (if https?
+ (tls-wrap s (uri-host uri)
+ #:verify-certificate? verify-certificate?)
+ s)))
+ (unless (zero? flags)
+ ;; FLAGS might contain O_NONBLOCK. Thus, set it as a last step
+ ;; because 'handshake' otherwise throws an exception for
+ ;; GNUTLS_E_AGAIN.
+ (let ((initial-flags (fcntl s F_GETFL)))
+ (fcntl s F_SETFL (logior initial-flags flags))))
+
+ port)))
(define (extend-request r k v . additional)
(let ((r (set-field r (request-headers)