From patchwork Tue Apr 14 22:59:02 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Ricardo Wurmus X-Patchwork-Id: 21262 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id EE62727BBE4; Wed, 15 Apr 2020 00:00:12 +0100 (BST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-2.8 required=5.0 tests=BAYES_00,DKIM_SIGNED, MAILING_LIST_MULTI,T_DKIM_INVALID,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.2 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTP id 603E427BBE1 for ; Wed, 15 Apr 2020 00:00:12 +0100 (BST) Received: from localhost ([::1]:40054 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jOUXL-00072d-VZ for patchwork@mira.cbaines.net; Tue, 14 Apr 2020 19:00:11 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:43826) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jOUXD-000724-OX for guix-patches@gnu.org; Tue, 14 Apr 2020 19:00:04 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1jOUXC-0002QM-LF for guix-patches@gnu.org; Tue, 14 Apr 2020 19:00:03 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:51654) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1jOUXC-0002QA-Hi for guix-patches@gnu.org; Tue, 14 Apr 2020 19:00:02 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1jOUXC-00023k-Ha for guix-patches@gnu.org; Tue, 14 Apr 2020 19:00:02 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#40629] [PATCH v2 8/9] import/utils: alist->package: Ignore known inputs. References: <87y2qym5v8.fsf@elephly.net> In-Reply-To: <87y2qym5v8.fsf@elephly.net> Resent-From: Ricardo Wurmus Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Tue, 14 Apr 2020 23:00:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 40629 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: To: 40629@debbugs.gnu.org Cc: Ricardo Wurmus Received: via spool by 40629-submit@debbugs.gnu.org id=B40629.15869051567823 (code B ref 40629); Tue, 14 Apr 2020 23:00:02 +0000 Received: (at 40629) by debbugs.gnu.org; 14 Apr 2020 22:59:16 +0000 Received: from localhost ([127.0.0.1]:34964 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jOUWO-000222-Rj for submit@debbugs.gnu.org; Tue, 14 Apr 2020 18:59:16 -0400 Received: from sender4-of-o53.zoho.com ([136.143.188.53]:21350) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jOUWN-00021u-AO for 40629@debbugs.gnu.org; Tue, 14 Apr 2020 18:59:11 -0400 ARC-Seal: i=1; a=rsa-sha256; t=1586905148; cv=none; d=zohomail.com; s=zohoarc; b=KYvAdCw8oSnSxVm2lyM+AJ83tXANhlhihO/FOcGdRAEr8tZKElZkkx+84GZAImsuNm9jF2AuDFYVOf7RzyN8fMtd6rYeKwUr3o8RrHsg+LPs1oik49dNMDfUvznT/PhzfE+DnxKJynkJnZIad4KCa0JwX85n1dF02jTUQBRhTmo= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1586905148; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:MIME-Version:Message-ID:Subject:To; bh=Xthvuy7Jh//mvLeGD4LN+ErqRLjNiv3sLtTmqW8HbHE=; b=XcSadAtD9kb2VJ9cxd320qgNBz45OXaN3g82c2gHLwzaAUgIfjb5Pag4upIJzvhGZPR5FKtHXQgq6thFOvVJwhfgezQp5ZsJR6b5DN5L8sP1Q+UhQlzAGtCvhlway+lO/tNZ3TrkP5LxFPf8a9hiRnuNToQwc7rQSR+m9606cOo= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass header.i=elephly.net; spf=pass smtp.mailfrom=rekado@elephly.net; dmarc=pass header.from= header.from= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1586905148; s=zoho; d=elephly.net; i=rekado@elephly.net; h=From:To:Cc:Message-ID:Subject:Date:MIME-Version:Content-Type:Content-Transfer-Encoding; bh=Xthvuy7Jh//mvLeGD4LN+ErqRLjNiv3sLtTmqW8HbHE=; b=UOvuuJcc6dmGjGyzKgE1UqelK+B72LE9YOCvDQLySLt1k04YWuuVRWDq175sYOXw MlJa5jO1T5ODlWV1JQokTGZ0QRr+hlHmWvaL9eTfimhC9eC2ppNisdG3wTX1yz03CDu 29BwsF0/Df0QQvnahukBr4QjKD9Syl/m2Mxv0CRk= Received: from localhost (p54AD4D5F.dip0.t-ipconnect.de [84.173.77.95]) by mx.zohomail.com with SMTPS id 1586905147797662.6808580815199; Tue, 14 Apr 2020 15:59:07 -0700 (PDT) From: Ricardo Wurmus Message-ID: <20200414225903.10862-1-rekado@elephly.net> Date: Wed, 15 Apr 2020 00:59:02 +0200 X-Mailer: git-send-email 2.25.1 MIME-Version: 1.0 X-ZohoMailClient: External X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.51.188.43 X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: "Guix-patches" X-getmail-retrieved-from-mailbox: Patches * guix/import/utils.scm (alist->package): Accept optional list of known inputs, which are excluded from the specification lookup. * guix/import/print.scm (package->code)[package-lists->code]: Handle inputs which are just symbols. --- guix/import/print.scm | 2 ++ guix/import/utils.scm | 27 ++++++++++++++++----------- 2 files changed, 18 insertions(+), 11 deletions(-) diff --git a/guix/import/print.scm b/guix/import/print.scm index 08f3ec9c34..471687c0ff 100644 --- a/guix/import/print.scm +++ b/guix/import/print.scm @@ -92,6 +92,8 @@ when evaluated." (define (package-lists->code lsts) (list 'quasiquote (map (match-lambda + ((? symbol? s) + (list (symbol->string s) (list 'unquote s))) ((label pkg . out) (let ((mod (package-module-name pkg))) (cons* label diff --git a/guix/import/utils.scm b/guix/import/utils.scm index 94c8cb040b..5fb1322535 100644 --- a/guix/import/utils.scm +++ b/guix/import/utils.scm @@ -2,7 +2,7 @@ ;;; Copyright © 2012, 2013, 2018, 2019, 2020 Ludovic Courtès ;;; Copyright © 2016 Jelle Licht ;;; Copyright © 2016 David Craven -;;; Copyright © 2017, 2019 Ricardo Wurmus +;;; Copyright © 2017, 2019, 2020 Ricardo Wurmus ;;; Copyright © 2018 Oleg Pykhalov ;;; Copyright © 2019 Robert Vollmert ;;; @@ -310,7 +310,18 @@ the expected fields of an object." (uri (assoc-ref orig "uri")) (sha256 sha)))))) -(define (alist->package meta) +(define* (alist->package meta #:optional (known-inputs '())) + "Return a package value generated from the alist META. If the list of +strings KNOWN-INPUTS is provided, do not treat the mentioned inputs as +specifications to look up and replace them with plain symbols instead." + (define (process-inputs which) + (let-values (((regular known) + (lset-diff+intersection + string=? + (vector->list (or (assoc-ref meta which) #())) + known-inputs))) + (append (specs->package-lists regular) + (map string->symbol known)))) (package (name (assoc-ref meta "name")) (version (assoc-ref meta "version")) @@ -318,15 +329,9 @@ the expected fields of an object." (build-system (lookup-build-system-by-name (string->symbol (assoc-ref meta "build-system")))) - (native-inputs - (specs->package-lists - (vector->list (or (assoc-ref meta "native-inputs") '#())))) - (inputs - (specs->package-lists - (vector->list (or (assoc-ref meta "inputs") '#())))) - (propagated-inputs - (specs->package-lists - (vector->list (or (assoc-ref meta "propagated-inputs") '#())))) + (native-inputs (process-inputs "native-inputs")) + (inputs (process-inputs "inputs")) + (propagated-inputs (process-inputs "propagated-inputs")) (home-page (assoc-ref meta "home-page")) (synopsis From patchwork Tue Apr 14 22:59:03 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ricardo Wurmus X-Patchwork-Id: 21263 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id E704F27BBE4; Wed, 15 Apr 2020 00:00:13 +0100 (BST) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-2.8 required=5.0 tests=BAYES_00,DKIM_SIGNED, MAILING_LIST_MULTI,T_DKIM_INVALID,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.2 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTP id 74FF327BBE1 for ; Wed, 15 Apr 2020 00:00:13 +0100 (BST) Received: from localhost ([::1]:40056 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jOUXN-00073d-18 for patchwork@mira.cbaines.net; Tue, 14 Apr 2020 19:00:13 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:43833) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jOUXE-000727-8l for guix-patches@gnu.org; Tue, 14 Apr 2020 19:00:05 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1jOUXD-0002SZ-3D for guix-patches@gnu.org; Tue, 14 Apr 2020 19:00:04 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:51655) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1jOUXC-0002S5-Vu for guix-patches@gnu.org; Tue, 14 Apr 2020 19:00:03 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1jOUXC-00023t-VV for guix-patches@gnu.org; Tue, 14 Apr 2020 19:00:02 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#40629] [PATCH v2 9/9] import/json: json->code: Handle files with more than one definition. Resent-From: Ricardo Wurmus Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Tue, 14 Apr 2020 23:00:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 40629 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: To: 40629@debbugs.gnu.org Cc: Ricardo Wurmus Received: via spool by 40629-submit@debbugs.gnu.org id=B40629.15869051587839 (code B ref 40629); Tue, 14 Apr 2020 23:00:02 +0000 Received: (at 40629) by debbugs.gnu.org; 14 Apr 2020 22:59:18 +0000 Received: from localhost ([127.0.0.1]:34967 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jOUWU-00022M-HK for submit@debbugs.gnu.org; Tue, 14 Apr 2020 18:59:18 -0400 Received: from sender4-of-o53.zoho.com ([136.143.188.53]:21353) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jOUWS-000226-7X for 40629@debbugs.gnu.org; Tue, 14 Apr 2020 18:59:16 -0400 ARC-Seal: i=1; a=rsa-sha256; t=1586905153; cv=none; d=zohomail.com; s=zohoarc; b=LakB16ZUUthPOpnhLm3fml1mZaORqMyHJ8jpTZAB5k5PTfU3TX/5SPNndhtsuME5EWROegH3iyKDJ587JsQcey0v6JHvJRFcjh38oG7qOxxLSAwpme6XznYgXsPGXlC5aZyAlHRwRVVR+ujkuVtOa2zC69ROSAHD/yJ7moh0DO0= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1586905153; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:MIME-Version:Message-ID:References:Subject:To; bh=aYK6ll64b9PgLkoDr38p35JQJUGeF2wKQfc/BQEsnx0=; b=CIsrrijMtC+f9CxitGsb/iZgEL0UNBJCDK/aHYXSOraqT9h+VXFj2uuPwcOyuigE4+v9+cxZ2x624zaViCXuQNzn925W/Itk+swSNgPnNNcAoiQse2WBRthJ9ILWzvUg4gFCKrzmbrEY09T/LwKHLp9v0V/RuOOZ6U5X1gop5A0= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass header.i=elephly.net; spf=pass smtp.mailfrom=rekado@elephly.net; dmarc=pass header.from= header.from= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1586905153; s=zoho; d=elephly.net; i=rekado@elephly.net; h=From:To:Cc:Message-ID:Subject:Date:In-Reply-To:References:MIME-Version:Content-Transfer-Encoding:Content-Type; bh=aYK6ll64b9PgLkoDr38p35JQJUGeF2wKQfc/BQEsnx0=; b=c3/GQaFMNSBtubDfsuUdZXWiELAb2jL5i6u8g/DPR38VZdXXz2Tti0PiTbH8Gdzq lglS6x3z077hqYjTBa5WBEjjsoM8YfuhiN6ln9wex/HSHh8MVfAz4Rhee3JCBvLkU2R E9bLE95brYxMUQNGm308NceIYF4XQ6DdoBqgHIxk= Received: from localhost (p54AD4D5F.dip0.t-ipconnect.de [84.173.77.95]) by mx.zohomail.com with SMTPS id 1586905151321454.3056656523403; Tue, 14 Apr 2020 15:59:11 -0700 (PDT) From: Ricardo Wurmus Message-ID: <20200414225903.10862-2-rekado@elephly.net> Date: Wed, 15 Apr 2020 00:59:03 +0200 X-Mailer: git-send-email 2.25.1 In-Reply-To: <20200414225903.10862-1-rekado@elephly.net> References: <20200414225903.10862-1-rekado@elephly.net> MIME-Version: 1.0 X-ZohoMailClient: External X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.51.188.43 X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: "Guix-patches" X-getmail-retrieved-from-mailbox: Patches * guix/import/json.scm (json->code): Convert JSON arrays to lists of package definitions. (json->scheme-file): Write all expressions to the target file. --- guix/import/json.scm | 35 ++++++++++++++++++++++++++++++----- 1 file changed, 30 insertions(+), 5 deletions(-) diff --git a/guix/import/json.scm b/guix/import/json.scm index 8f8dbbd05d..0c98bb25b8 100644 --- a/guix/import/json.scm +++ b/guix/import/json.scm @@ -24,8 +24,11 @@ #:use-module (guix http-client) #:use-module (guix import utils) #:use-module (guix import print) + #:use-module (ice-9 match) #:use-module (ice-9 rdelim) + #:use-module (srfi srfi-1) #:use-module (srfi srfi-2) + #:use-module (srfi srfi-26) #:use-module (srfi srfi-34) #:export (json-fetch json->code @@ -50,19 +53,41 @@ the query." result))) (define (json->code file-name) - "Read FILE-NAME containing a JSON package definition and return an -S-expression, or return #F when the JSON is invalid." + "Read FILE-NAME containing one ore more JSON package definitions and return +a list of S-expressions, or return #F when the JSON is invalid." (catch 'json-invalid (lambda () (let ((json (json-string->scm (with-input-from-file file-name read-string)))) - (package->code (alist->package json)))) + (match json + (#(packages ...) + ;; To allow definitions to refer to one another, collect references + ;; to local definitions and tell alist->package to ignore them. + (second + (memq #:result + (fold + (lambda (pkg names+result) + (match names+result + ((#:names names #:result result) + (list #:names + (cons (assoc-ref pkg "name") names) + #:result + (append result + (list + (package->code (alist->package pkg names)) + (string->symbol (assoc-ref pkg "name")))))))) + (list #:names '() + #:result '()) + packages)))) + (package + (list (package->code (alist->package json)) + (string->symbol (assoc-ref json "name"))))))) (const #f))) (define (json->scheme-file file) "Convert the FILE containing a JSON package definition to a Scheme representation and return the new file name (or #F on error)." - (and-let* ((json (json->code file)) + (and-let* ((sexprs (json->code file)) (file* (let* ((tempdir (or (getenv "TMPDIR") "/tmp")) (template (string-append tempdir "/guix-XXXXXX")) (port (mkstemp! template))) @@ -74,5 +99,5 @@ representation and return the new file name (or #F on error)." (guix) ((guix licenses) #:prefix license:)) port) - (write json port))) + (for-each (cut write <> port) sexprs))) file*))