From patchwork Thu Jan 20 13:08:43 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: M X-Patchwork-Id: 36644 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id 8703927BBEA; Thu, 20 Jan 2022 18:03:45 +0000 (GMT) X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-2.7 required=5.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,FREEMAIL_FROM,MAILING_LIST_MULTI,RCVD_IN_MSPIKE_H3, RCVD_IN_MSPIKE_WL,SPF_HELO_PASS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.6 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTPS id E570927BBE9 for ; Thu, 20 Jan 2022 18:03:44 +0000 (GMT) Received: from localhost ([::1]:43734 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nAbmh-0008C4-Ul for patchwork@mira.cbaines.net; Thu, 20 Jan 2022 13:03:43 -0500 Received: from eggs.gnu.org ([209.51.188.92]:50372) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nAXBb-00042U-51 for guix-patches@gnu.org; Thu, 20 Jan 2022 08:09:09 -0500 Received: from debbugs.gnu.org ([209.51.188.43]:35556) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1nAXBW-0004Kp-Bu for guix-patches@gnu.org; Thu, 20 Jan 2022 08:09:05 -0500 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1nAXBW-0004qL-7Q for guix-patches@gnu.org; Thu, 20 Jan 2022 08:09:02 -0500 X-Loop: help-debbugs@gnu.org Subject: [bug#53389] [PATCH 3/9] tests/minetest: Run a HTTP server instead of mocking. Resent-From: Maxime Devos Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Thu, 20 Jan 2022 13:09:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 53389 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 53389@debbugs.gnu.org Cc: ludo@gnu.org, Maxime Devos Received: via spool by 53389-submit@debbugs.gnu.org id=B53389.164268413518597 (code B ref 53389); Thu, 20 Jan 2022 13:09:02 +0000 Received: (at 53389) by debbugs.gnu.org; 20 Jan 2022 13:08:55 +0000 Received: from localhost ([127.0.0.1]:56688 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nAXBO-0004pT-3y for submit@debbugs.gnu.org; Thu, 20 Jan 2022 08:08:55 -0500 Received: from baptiste.telenet-ops.be ([195.130.132.51]:42012) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nAXBM-0004os-2W for 53389@debbugs.gnu.org; Thu, 20 Jan 2022 08:08:52 -0500 Received: from localhost.localdomain ([IPv6:2a02:1811:8c09:9d00:3c5f:2eff:feb0:ba5a]) by baptiste.telenet-ops.be with bizsmtp id l18q2600H4UW6Th0118rNH; Thu, 20 Jan 2022 14:08:51 +0100 From: Maxime Devos Date: Thu, 20 Jan 2022 13:08:43 +0000 Message-Id: <20220120130849.292178-3-maximedevos@telenet.be> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20220120130849.292178-1-maximedevos@telenet.be> References: <20220120130849.292178-1-maximedevos@telenet.be> MIME-Version: 1.0 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=telenet.be; s=r22; t=1642684131; bh=QPzeQ1OR9EZOc2aVtRHjROXo3bd0aTaZ2ZBGRw7ZSYs=; h=From:To:Cc:Subject:Date:In-Reply-To:References; b=NTjbY3JQB4RQt/A1eKbhb2VQCScm6SA/XOl4J4v4SoSl+jK3qeh90V+cZ2oSsKd7q v+TPEVntNyNjsLIStnAOCOQ82Nkdv3fZpMFJntbaZJdIJYa4Ffx379sm7/aJOKaQvs tE5M0swg7T3dzO+oAexyw7Ph95724HtnOIDi4kc2L5sXqChwajUAoG1EMo11HER7mV 3Qyq85saw7PtDRa967qcDFmyj0ttuDdhFC+CjrGVVdYBLmpzlWSamCtH+C76jGaUe2 5MuM+QNR+I0mT0dLGUOixSVV+Edd20QUOfLj89Cr2OsyQdK5MBLhUFiyBHKc/sAVBS BB7ZnfAKibyoQ== X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: "Guix-patches" X-getmail-retrieved-from-mailbox: Patches Fixes: Unfortunately, for some unknown reason (a limitation of (guix tests http) perhaps?), parallelism causes ECONNREFUSED in tests but not in the wild, so 'par-map' has to be mocked for now. * tests/minetest.scm (call-with-packages): Avoid mocking by running an actual HTTP server. * guix/import/minetest.scm (par-map): Allow mocking the Minetest importer's use of par-map without impacting anything else. Suggested-by: Ludovic Courtès --- guix/import/minetest.scm | 5 ++- tests/minetest.scm | 82 ++++++++++++++++++++++++---------------- 2 files changed, 53 insertions(+), 34 deletions(-) diff --git a/guix/import/minetest.scm b/guix/import/minetest.scm index 3b2cdcdcac..3eab5f703f 100644 --- a/guix/import/minetest.scm +++ b/guix/import/minetest.scm @@ -19,7 +19,6 @@ (define-module (guix import minetest) #:use-module (ice-9 match) #:use-module (ice-9 receive) - #:use-module (ice-9 threads) #:use-module (ice-9 hash-table) #:use-module (srfi srfi-1) #:use-module (srfi srfi-2) @@ -69,6 +68,10 @@ (define (delete-cr text) (string-delete #\cr text)) +;; Mocked by tests. +(define par-map (@ (ice-9 threads) par-map)) +(set! par-map par-map) + ;;; diff --git a/tests/minetest.scm b/tests/minetest.scm index cbb9e83889..bdd8bd0645 100644 --- a/tests/minetest.scm +++ b/tests/minetest.scm @@ -1,5 +1,5 @@ ;;; GNU Guix --- Functional package management for GNU -;;; Copyright © 2021 Maxime Devos +;;; Copyright © 2021, 2022 Maxime Devos ;;; ;;; This file is part of GNU Guix. ;;; @@ -23,6 +23,7 @@ #:use-module (guix import minetest) #:use-module (guix import utils) #:use-module (guix tests) + #:use-module (guix tests http) #:use-module (guix packages) #:use-module (guix git-download) #:use-module ((gnu packages minetest) @@ -30,6 +31,9 @@ #:use-module ((gnu packages base) #:select (hello)) #:use-module (json) + #:use-module (web request) + #:use-module (web uri) + #:use-module (web client) #:use-module (ice-9 match) #:use-module (srfi srfi-1) #:use-module (srfi srfi-26) @@ -152,7 +156,7 @@ (invalidate-memoization! minetest->guix-package) (define (scm->json-port scm) (open-input-string (scm->json-string scm))) - (define (handle-package url requested-author requested-name . rest) + (define (handle-package subresource requested-author requested-name . rest) (define relevant-argument-list (any (lambda (argument-list) (apply (lambda* (#:key (author "Author") (name "foo") @@ -164,14 +168,15 @@ argument-lists)) (when (not relevant-argument-list) (error "the package ~a/~a should be irrelevant, but ~a is fetched" - requested-author requested-name url)) - (scm->json-port - (apply (match rest - (("") make-package-json) - (("dependencies" "") make-dependencies-json) - (("releases" "") make-releases-json) - (_ (error "TODO ~a" rest))) - relevant-argument-list))) + requested-author requested-name subresource)) + (define json (apply + (match rest + (("") make-package-json) + (("dependencies" "") make-dependencies-json) + (("releases" "") make-releases-json) + (_ (error "TODO ~a" rest))) + relevant-argument-list)) + (values '() (lambda (port) (scm->json json port)))) (define (handle-mod-search sort) ;; Produce search results, sorted by SORT in descending order. (define arguments->key @@ -191,29 +196,40 @@ ("name" . ,name) ("type" . ,type)))) (define argument-list->json (cut apply arguments->json <>)) - (scm->json-port - (list->vector (filter-map argument-list->json sorted-argument-lists)))) - (mock ((guix http-client) http-fetch - (lambda* (url #:key headers) - (unless (string-prefix? "mock://api/packages/" url) - (error "the URL ~a should not be used" url)) - (define resource - (substring url (string-length "mock://api/packages/"))) - (define components (string-split resource #\/)) - (match components - ((author name . rest) - (apply handle-package url author name rest)) - (((? (cut string-prefix? "?type=mod&q=" <>) query)) - (handle-mod-search - (cond ((string-contains query "sort=score") "score") - ((string-contains query "sort=downloads") "downloads") - (#t (error "search query ~a has unknown sort key" - query))))) - (_ - (error "the URL ~a should have an author and name component" - url))))) - (parameterize ((%contentdb-api "mock://api/")) - (thunk)))) + (define json + (list->vector (filter-map argument-list->json sorted-argument-lists))) + (values '() + (lambda (port) (scm->json json port)))) + (with-http-server* + (lambda (request _) + (unless (eq? 'GET (request-method request)) + (error "wrong HTTP method")) + (define resource (uri-path (request-uri request))) + (unless (string-prefix? "/api/packages/" resource) + (error "the resource ~a should not be used" resource)) + (define subresource + (substring resource (string-length "/api/packages/"))) + (define components (string-split subresource #\/)) + (match components + ((author name . rest) + (apply handle-package subresource author name rest)) + (("") + (let ((query (uri-query (request-uri request)))) + (handle-mod-search + (cond ((string-contains query "sort=score") "score") + ((string-contains query "sort=downloads") "downloads") + (#t (error "search query ~a has unknown sort key" + query)))))) + (_ + (error "the resource ~a should have an author and name component" + resource)))) + (parameterize ((%contentdb-api + (format #f "http://localhost:~a/api/" (%http-server-port))) + (current-http-proxy #f)) + ;; XXX: for some unknown reason, parallelism causes ECONNREFUSED in + ;; tests but not in the wild. + (mock ((guix import minetest) par-map map) + (thunk))))) (define* (minetest->guix-package* #:key (author "Author") (name "foo") (sort %default-sort-key)