From patchwork Sun May 26 16:44:39 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: "Amar M. Singh" X-Patchwork-Id: 14085 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id 53FD017027; Sun, 26 May 2019 17:45:13 +0100 (BST) X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,T_DKIM_INVALID, URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTP id E7F7816F56 for ; Sun, 26 May 2019 17:45:11 +0100 (BST) Received: from localhost ([127.0.0.1]:57424 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hUwGi-0006te-Bo for patchwork@mira.cbaines.net; Sun, 26 May 2019 12:45:08 -0400 Received: from eggs.gnu.org ([209.51.188.92]:45735) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hUwGf-0006tP-Ss for guix-patches@gnu.org; Sun, 26 May 2019 12:45:07 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hUwGe-000580-3I for guix-patches@gnu.org; Sun, 26 May 2019 12:45:05 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:38041) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1hUwGc-00057O-Uc for guix-patches@gnu.org; Sun, 26 May 2019 12:45:03 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1hUwGc-0004Wt-KF for guix-patches@gnu.org; Sun, 26 May 2019 12:45:02 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#35627] [PATCH] Importer: Add golang References: <87lfzhjx1s.fsf@disroot.org> In-Reply-To: <87lfzhjx1s.fsf@disroot.org> Resent-From: Amar Singh Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Sun, 26 May 2019 16:45:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 35627 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 35627@debbugs.gnu.org Received: via spool by 35627-submit@debbugs.gnu.org id=B35627.155888909117364 (code B ref 35627); Sun, 26 May 2019 16:45:02 +0000 Received: (at 35627) by debbugs.gnu.org; 26 May 2019 16:44:51 +0000 Received: from localhost ([127.0.0.1]:51584 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hUwGQ-0004W0-Gy for submit@debbugs.gnu.org; Sun, 26 May 2019 12:44:51 -0400 Received: from knopi.disroot.org ([178.21.23.139]:55788) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hUwGN-0004Vp-Qx for 35627@debbugs.gnu.org; Sun, 26 May 2019 12:44:48 -0400 Received: from localhost (localhost [127.0.0.1]) by disroot.org (Postfix) with ESMTP id 7A31A32167 for <35627@debbugs.gnu.org>; Sun, 26 May 2019 18:44:46 +0200 (CEST) X-Virus-Scanned: Debian amavisd-new at disroot.org Received: from knopi.disroot.org ([127.0.0.1]) by localhost (disroot.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id jOMsBMdCY2zN for <35627@debbugs.gnu.org>; Sun, 26 May 2019 18:44:44 +0200 (CEST) From: Amar Singh DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=disroot.org; s=mail; t=1558889084; bh=UIv9qT659AlpNkSA1wjPhGLroMcISKUprrpKlyIHYXE=; h=From:To:Subject:Date; b=eoxqEOPiXElUQnvfxqy6yI0BSsWvcVnA/BHtLqLF2OjffsK3DYpn2O2WnIYzOZEul FAeTxJTVB11CO79dxsT47P2Nv1EvL2eamA+tuSz+MHsmyr0R1ATEKenXpQx7ZgyCx1 pKyU/n0iTq3weJng5GL4ul1GwTzPLQ+L7sc2WREokgXP0JWAj1i/630wzVP9CvIH55 +u3Ea57q+N+6/wtR0g6Z5gMAguOqnEQSbA+MIBQj136dDEODC0NLmtBjShW50bLU67 wDlXJ0uICgKaYx9iJBwJrw6Spd+4J4iKah17qt0K6Pp/OQzr7XLhpaW8OH4+RV5DG8 gnwAddx6obWww== Date: Sun, 26 May 2019 22:14:39 +0530 Message-ID: <85blzp6u88.fsf@disroot.org> Mime-Version: 1.0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.51.188.43 X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: "Guix-patches" X-getmail-retrieved-from-mailbox: Patches This Go importer is flaky. It's a rebased patch. From f54e0aa0ae3f553701b1253a6e8605a493e4d4ac Mon Sep 17 00:00:00 2001 From: Amar Singh Date: Tue, 30 Apr 2019 23:17:51 +0530 Subject: [PATCH] Importer: Add golang * guix/import/golang.scm. Signed-off-by: Amar Singh --- guix/import/golang.scm | 217 +++++++++++++++++++++++++++++++++++++++++ 1 file changed, 217 insertions(+) create mode 100644 guix/import/golang.scm diff --git a/guix/import/golang.scm b/guix/import/golang.scm new file mode 100644 index 0000000000..b51d496602 --- /dev/null +++ b/guix/import/golang.scm @@ -0,0 +1,217 @@ +;;; GNU Guix --- Functional package management for GNU +;;; Copyright © 2019 by Amar Singh +;;; +;;; This file is part of GNU Guix. +;;; +;;; This program is free software: you can redistribute it and/or modify +;;; it under the terms of the GNU General Public License as published by +;;; the Free Software Foundation, either version 3 of the License, or +;;; (at your option) any later version. +;;; +;;; This program is distributed in the hope that it will be useful, +;;; but WITHOUT ANY WARRANTY; without even the implied warranty of +;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +;;; GNU General Public License for more details. +;;; +;;; You should have received a copy of the GNU General Public License +;;; along with this program. If not, see . + +(define-module (guix import golang)) +(use-modules + (srfi srfi-1) ;; fold + (ice-9 rdelim) ;; read-string + (guix import github) ;; latest-release + (guix utils) ;; string-replace-substring + (guix memoization) ;; memoize network operations + (guix download) ;; download-to-store + ((guix import utils) #:prefix utils:) ;; hash + (guix packages) ;; packages + ((guix licenses) #:prefix license:) ;; licenses + (guix build-system) ;; build-system printer + (guix build-system go) ;; go-build-system + (guix store) ;; with-store + ;; (gnu packages golang) ;; inherit (simple) go package + (ice-9 textual-ports) ;; to parse readme.md + (ice-9 popen) ;; open-input-ouput-pipe + (web uri) ;; uri->string + (srfi srfi-26) ;; cut + ) + +;;; To use, simply: +;;; 1. (load "golang.scm") +;;; 2. (define go-package (make-package go-name*)) +;;; 3. (package-sexp go-package) + +;;; STATUS +;;; 1. latest-release DONE +;;; 1.b latest-commit PENDING/STALLED +;;; 2. go-name->guix-name DONE +;;; 2.b style go-github-com-user-project DONE +;;; 4. go-name->url DONE +;;; 4.b go-name->tarball DONE +;;; 5. go-name->sha256 (go-name version) DONE +;;; 6. go-name->synopsis DONE +;;; 7. go-name->description DONE +;;; 4-7.b. Memoize, network procedures DONE +;;; 6-7.b try to extract sentences. TODO +;;; 8. go-name->license TODO +;;; 9. go-name->inputs DONE +;;; 9.b. inputs alist DONE +;;; 10. package-sexp DONE +;;; 10.a origin-sexp DONE +;;; 11. Package Builds TODO + +(define-public go-name* "github.com/gohugoio/hugo") ;; for tests + +(define* (go-name->url go-name #:rest args) + (if (string-contains go-name ".") + (uri->string (string->uri (apply string-append + "https://" go-name args))) + #f)) + +(define (go-name->tarball go-name version) + (go-name->url go-name "/archive/v" + version ".tar.gz")) + +(define* (string-replace-substrings string substrings + #:optional (replacement "-")) + (if (null-list? substrings) + string + ((cut string-replace-substring <> (car substrings) replacement) + (string-replace-substrings string (cdr substrings))))) + +;;; Possible remove @@ if upstream exports the symbols +(define (go-name->guix-name go-name) + (string-append "go-" + (string-replace-substrings go-name '("." "/") "-"))) + +;;; Slow; accesses the network; memoized +(define latest-release + (memoize + (lambda (go-name) + ((@@ (guix import github) latest-released-version) + (go-name->url go-name) + (go-name->guix-name go-name))))) + +;;; Slow; downloads the url from network; memoized +(define url->store + (@@ (guix import cran) download)) + +;;; Slow; download src tarball from network, returns base32 nix-hash; +;;; memoized +(define (go-name->sha256 go-name version) + (utils:guix-hash-url (url->store (go-name->tarball go-name version)))) + +;;; Slow; network access; memoized +(define go-name->readme-string + (memoize + (lambda (go-name) + (define (go-name->readme go-name) + (go-name->url "raw.githubusercontent.com" + ;; TODO, detect the domain + (substring go-name + (string-length "github.com")) + "/master/" + "README.md")) + (call-with-input-file (url->store (go-name->readme go-name)) + read-string)))) + +;;; TODO: try to match the first sentence. +(define (go-name->synopsis go-name) + (substring (go-name->readme-string go-name) 0 100)) + +;;; TODO: try to match the the next two sentences. +(define (go-name->description go-name) + (substring (go-name->readme-string go-name) 100 300)) + +(define shell-command + (lambda* (command #:rest args) + (let* ((cmd (string-join (cons command (delete #f (delete '() args))) " ")) + (port (open-input-output-pipe cmd)) + (result (read-string port)) + (exit-code (close-pipe port))) + (and (zero? exit-code) + (string-split (string-trim-right result) #\newline))))) + +(define go-name->inputs + (lambda (go-name) + (let ((recursive-depends "-f '{{ join .Deps \"\\n\" }}'") + (direct-depends "-f '{{ join .Imports \"\\n\" }}'") + (go-command (car (shell-command "which go")))) + (shell-command go-command "list" direct-depends go-name)))) + +;;; License +(define (string->license license-string) + ((@@ (guix import cran) string->license) (string-upcase license-string))) + +;;; For inputs +(define format-inputs + (@@ (guix import cran) format-inputs)) + +(define-public (make-go-package go-name) + ;; Do the expensive operations only once; query network for latest + ;; version + (let* ((version (latest-release go-name)) + (sha256 (go-name->sha256 go-name version)) + (readme-string (go-name->readme-string go-name))) + (package + ;; (inherit go-github-com-alsm-ioprogress) + (name + (string-append "go-" go-name)) + (version version) + (source + (origin (method url-fetch) + (uri (go-name->tarball go-name version)) + (sha256 (base32 sha256)))) + (home-page + (go-name->url go-name)) + (build-system + go-build-system) + (arguments + `(#:import-path ,go-name)) + ;; TODO: make inputs into (unquote ..) form + (inputs + (format-inputs (map go-name->guix-name (go-name->inputs go-name)))) + (synopsis (go-name->synopsis go-name)) + (description (go-name->description go-name)) + ;; TODO: license + (license license:expat) + ))) + +(define (filter-newlines string) + (string-filter (lambda (x) (not (equal? x #\newline))) string)) + +(define bv->nix-base32 (@@ (guix packages) + bytevector->nix-base32-string)) + +(define (origin-sexp origin) + `(origin + (method url-fetch) + (uri ,(origin-uri origin)) + (sha256 (base32 ,(bv->nix-base32 (origin-sha256 origin)))) + (file-name ,(origin-file-name origin)) + (patches ,(origin-patches origin)) + (snippet ,(origin-snippet origin)) + (patch-flags ,(origin-patch-flags origin)) + (patch-inputs ,(origin-patch-inputs origin)) + (modules ,(origin-modules origin)) + (patch-guile ,(origin-patch-guile origin)))) + +(define (build-system-sexp build-system) + (symbol-append (build-system-name build-system) '-build-system)) + +(define-public (package-sexp package) + `(package + (name ,(package-name package)) + (version ,(package-version package)) + (source ,(origin-sexp (package-source package))) + (home-page ,(package-home-page package)) + (build-system ,(build-system-sexp (package-build-system package))) + (arguments ,(package-arguments package)) + (synopsis ,(filter-newlines (package-synopsis package))) + (description ,(filter-newlines (package-description package))) + (inputs ,(format-inputs (map car (package-inputs package)))) + (native-inputs ,(format-inputs (map car (package-native-inputs package)))) + (propagated-inputs ,(format-inputs (map car (package-propagated-inputs package)))))) + +;;; golang.scm ends here