diff mbox series

[bug#47288] guix: http-client: Tweak http-multiple-get error handling.

Message ID 20210321004338.31867-1-mail@cbaines.net
State Accepted
Headers show
Series [bug#47288] guix: http-client: Tweak http-multiple-get error handling. | expand

Checks

Context Check Description
cbaines/submitting builds success
cbaines/comparison success View comparision
cbaines/git branch success View Git branch
cbaines/applying patch success View Laminar job
cbaines/issue success View issue

Commit Message

Christopher Baines March 21, 2021, 12:43 a.m. UTC 
This isn't meant to change the way errors are handled, and arguably makes the
code harder to read, but it's a uninformed attempt to improve the
performance (following on from a performance regression in
205833b72c5517915a47a50dbe28e7024dc74e57).

I'm guessing something about Guile internals makes calling (loop ...) within
the catch bit less performant than avoiding this and calling (loop ...) after
the catch bit has finished. Since this happens lots, this seems to be
sufficient to make guix weather a lot slower than it was before.

Anecdotal testing of guix weather suggests this change might work.

* guix/http-client.scm (http-multiple-get): Tweak how the second catch
statement works.
---
 guix/http-client.scm | 77 +++++++++++++++++++++++---------------------
 1 file changed, 41 insertions(+), 36 deletions(-)

Comments

M March 21, 2021, 8:36 a.m. UTC | #1 
On Sun, 2021-03-21 at 00:43 +0000, Christopher Baines wrote:
> This isn't meant to change the way errors are handled, and arguably makes the
> code harder to read, but it's a uninformed attempt to improve the
> performance (following on from a performance regression in
> 205833b72c5517915a47a50dbe28e7024dc74e57).
> 
> I'm guessing something about Guile internals makes calling (loop ...) within
> the catch bit less performant than avoiding this and calling (loop ...) after
> the catch bit has finished. Since this happens lots, this seems to be
> sufficient to make guix weather a lot slower than it was before.

I took a look at the code, and it seems we did somthing like:

(let loop VARS
  (match sent
    NON-LOOPING-CASES
    (STUFF
     (catch #t
       THUNK-THAT-MIGHT-CALL-LOOP-IN-TAIL-POSITION
       SOME-HANDLER-THAT-DOES-NOT-CALL-LOOP)

A small demonstration of what could go wrong with such a construction
(run this in the Guile REPL):

(let loop ((attempts-todo 20))
  (catch 'oops
    (lambda ()
      (if (<= 0 attempts-todo)
          (loop (- attempts-todo 1))
          (throw 'oops)))
    (lambda _ (display 'too-bad!) (newline) (backtrace))))

Output:
too-bad!

Backtrace:
In ice-9/boot-9.scm:
  1731:15 19 (with-exception-handler #<procedure 7f73c7e615a0 at ice-9/boot-9.scm:1815:7 (exn)> _ # _ …)
  [The previous line repeated 17 times]
  1731:15  1 (with-exception-handler #<procedure 7f73c7e61240 at ice-9/boot-9.scm:1815:7 (exn)> _ # _ …)
In unknown file:
           0 (backtrace #<undefined>)

With this construction, we were nesting exception handlers within exception handlers
... So in hindsight it doesn't seem surprising this isn't very performant.
(THUNK-THAT-MIGHT-CALL-LOOP-IN-TAIL-POSITION itself is not called in the tail-position
of the '(let loop ...)' form.)

Hope that sheds some light on the matter,
Maxime
diff mbox series

Patch

diff --git a/guix/http-client.scm b/guix/http-client.scm
index 4b4c14ed0b..a189cceecf 100644
--- a/guix/http-client.scm
+++ b/guix/http-client.scm
@@ -219,42 +219,47 @@  returning."
              (remainder
               (connect p remainder result))))
           ((head tail ...)
-           (catch #t
-             (lambda ()
-               (let* ((resp   (read-response p))
-                      (body   (response-body-port resp))
-                      (result (proc head resp body result)))
-                 ;; The server can choose to stop responding at any time,
-                 ;; in which case we have to try again.  Check whether
-                 ;; that is the case.  Note that even upon "Connection:
-                 ;; close", we can read from BODY.
-                 (match (assq 'connection (response-headers resp))
-                   (('connection 'close)
-                    (close-port p)
-                    (connect #f                       ;try again
-                             (drop requests (+ 1 processed))
-                             result))
-                   (_
-                    (loop tail (+ 1 processed) result))))) ;keep going
-             (lambda (key . args)
-               ;; If PORT was cached and the server closed the connection
-               ;; in the meantime, we get EPIPE.  In that case, open a
-               ;; fresh connection and retry.  We might also get
-               ;; 'bad-response or a similar exception from (web response)
-               ;; later on, once we've sent the request, or a
-               ;; ERROR/INVALID-SESSION from GnuTLS.
-               (if (or (and (eq? key 'system-error)
-                            (= EPIPE (system-error-errno `(,key ,@args))))
-                       (and (eq? key 'gnutls-error)
-                            (eq? (first args) error/invalid-session))
-                       (memq key
-                             '(bad-response bad-header bad-header-component)))
-                   (begin
-                     (close-port p)
-                     (connect #f      ; try again
-                              (drop requests (+ 1 processed))
-                              result))
-                   (apply throw key args))))))))))
+           (match
+               (catch #t
+                 (lambda ()
+                   (let* ((resp   (read-response p))
+                          (body   (response-body-port resp))
+                          (result (proc head resp body result)))
+                     ;; The server can choose to stop responding at any time,
+                     ;; in which case we have to try again.  Check whether
+                     ;; that is the case.  Note that even upon "Connection:
+                     ;; close", we can read from BODY.
+                     (match (assq 'connection (response-headers resp))
+                       (('connection 'close)
+                        (close-port p)
+                        'try-again-with-new-connection)
+                       (_
+                        result))))
+                 (lambda (key . args)
+                   ;; If PORT was cached and the server closed the connection
+                   ;; in the meantime, we get EPIPE.  In that case, open a
+                   ;; fresh connection and retry.  We might also get
+                   ;; 'bad-response or a similar exception from (web response)
+                   ;; later on, once we've sent the request, or a
+                   ;; ERROR/INVALID-SESSION from GnuTLS.
+                   (if (or (and (eq? key 'system-error)
+                                (= EPIPE (system-error-errno `(,key ,@args))))
+                           (and (eq? key 'gnutls-error)
+                                (eq? (first args) error/invalid-session))1
+                                (memq key
+                                      '(bad-response
+                                        bad-header
+                                        bad-header-component)))
+                       (begin
+                         (close-port p)
+                         'try-again-with-new-connection)
+                       (apply throw key args))))
+             ('try-again-with-new-connection
+              (connect #f
+                       (drop requests (+ 1 processed))
+                       result))
+             (result
+              (loop tail (+ 1 processed) result)))))))))
 
 
 ;;;