From patchwork Thu Dec 20 02:39:54 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Eric Bavier X-Patchwork-Id: 493 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id A393C16984; Thu, 20 Dec 2018 02:42:20 +0000 (GMT) X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-6.9 required=5.0 tests=BAYES_00,MIME_QP_LONG_LINE, RCVD_IN_DNSWL_HI,T_DKIM_INVALID,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) by mira.cbaines.net (Postfix) with ESMTP id 358C716913 for ; Thu, 20 Dec 2018 02:42:20 +0000 (GMT) Received: from localhost ([::1]:35053 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gZoHz-0004Rd-M7 for patchwork@mira.cbaines.net; Wed, 19 Dec 2018 21:42:19 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:51483) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gZoGs-0003gb-Dd for guix-patches@gnu.org; Wed, 19 Dec 2018 21:41:12 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gZoGl-0001cD-WD for guix-patches@gnu.org; Wed, 19 Dec 2018 21:41:07 -0500 Received: from debbugs.gnu.org ([208.118.235.43]:51084) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1gZoGk-0001aD-35 for guix-patches@gnu.org; Wed, 19 Dec 2018 21:41:03 -0500 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1gZoGj-00066c-RR for guix-patches@gnu.org; Wed, 19 Dec 2018 21:41:01 -0500 X-Loop: help-debbugs@gnu.org Subject: [bug#33809] [PATCH] refresh: github: updates for origins using 'git-fetch'. Resent-From: ericbavier@centurylink.net Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Thu, 20 Dec 2018 02:41:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: report 33809 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 33809@debbugs.gnu.org X-Debbugs-Original-To: guix-patches@gnu.org Received: via spool by submit@debbugs.gnu.org id=B.154527361323396 (code B ref -1); Thu, 20 Dec 2018 02:41:01 +0000 Received: (at submit) by debbugs.gnu.org; 20 Dec 2018 02:40:13 +0000 Received: from localhost ([127.0.0.1]:55342 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gZoFw-00065H-N9 for submit@debbugs.gnu.org; Wed, 19 Dec 2018 21:40:13 -0500 Received: from eggs.gnu.org ([208.118.235.92]:41314) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gZoFv-000651-Df for submit@debbugs.gnu.org; Wed, 19 Dec 2018 21:40:11 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gZoFo-0000Ts-Ql for submit@debbugs.gnu.org; Wed, 19 Dec 2018 21:40:06 -0500 Received: from lists.gnu.org ([2001:4830:134:3::11]:47593) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1gZoFo-0000TO-Js for submit@debbugs.gnu.org; Wed, 19 Dec 2018 21:40:04 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:50893) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gZoFn-0002sG-D2 for guix-patches@gnu.org; Wed, 19 Dec 2018 21:40:04 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gZoFk-0000LJ-4P for guix-patches@gnu.org; Wed, 19 Dec 2018 21:40:03 -0500 Received: from mail.onyx.syn-alias.com ([206.152.134.66]:52429 helo=smtp.centurylink.net) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1gZoFj-0000IX-Rl for guix-patches@gnu.org; Wed, 19 Dec 2018 21:40:00 -0500 DKIM-Signature: v=1; a=rsa-sha1; d=centurylink.net; s=ctl201402; c=relaxed/simple; q=dns/txt; i=@centurylink.net; t=1545273598; h=From:Subject:Date:To:MIME-Version:Content-Type; bh=NvqpZtLd0WiGC2NjN4eBDb0YbI0=; b=vWUx+RuiDEkIXKVfd/Ax2rLDjMXprTPQAs53m0b/PoIBycGWJ0Rwusu66c+O5UyS QW86Oif/1hW+IzmLpQsdHuOO/DejXD+YyQ/qFkn7kRBtdLDcPiqE4pktVwehh2z4 Sn5E2Cmh+qlUvR5M4ZcEHOZaZ2Rhl9FbL4iBqA0n9xOjIILLOjoEnSvysA3x2KcV HasYVhsY+azZLyZEy0BVG//J7T1C9CnI8QnnFuoK0UwJs72TVFD0usfKeu1BpaiG vcQVbEXj5hcTzh7QTLkb4g65kt3SEXd6IcOLoz9NKSFzd7H2figKzeknMsnkpggR RI1CNuQoWWVOAfTQHJs9+w==; X_CMAE_Category: , , X-CNFS-Analysis: v=2.3 cv=dP/YZ9Rb c=1 sm=1 tr=0 a=cjJt/q+I1cGZQhdwTxhucA==:117 a=cjJt/q+I1cGZQhdwTxhucA==:17 a=KGjhK52YXX0A:10 a=IkcTkHD0fZMA:10 a=2ur7OfE09M0A:10 a=zSv6zi-ZgakA:10 a=inoG4m36AAAA:8 a=pGLkceISAAAA:8 a=mDV3o1hIAAAA:8 a=Fvvde27rAAAA:20 a=NEAV23lmAAAA:8 a=6MZGje2BAAAA:20 a=QQzXMK8EJJvuU45c4GIA:9 a=QEXdDO2ut3YA:10 a=_Qgj-RFg6mImvLaOgLPg:22 a=_FVE-zBwftR9WsbkzFJk:22 X-CM-Score: 0 X-Scanned-by: Cloudmark Authority Engine X-Authed-Username: ZXJpY2JhdmllckBjZW50dXJ5bGluay5uZXQ= Authentication-Results: smtp01.onyx.dfw.sync.lan smtp.user=ericbavier@centurylink.net; auth=pass (LOGIN) Received: from [97.116.125.252] ([97.116.125.252:45390] helo=localhost.localdomain) by smtp.centurylink.net (envelope-from ) (ecelerity 3.6.25.56547 r(Core:3.6.25.0)) with ESMTPSA (cipher=DHE-RSA-AES128-GCM-SHA256) id A3/00-25193-EF00B1C5; Wed, 19 Dec 2018 21:39:58 -0500 From: ericbavier@centurylink.net Date: Wed, 19 Dec 2018 20:39:54 -0600 Message-Id: <20181220023954.22913-1-ericbavier@centurylink.net> X-Mailer: git-send-email 2.20.1 MIME-Version: 1.0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6.x [fuzzy] X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6.x X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 208.118.235.43 X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Eric Bavier Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: "Guix-patches" X-getmail-retrieved-from-mailbox: Patches From: Eric Bavier * guix/import/github.scm (updated-github-url): Respond with the repository url for the 'git-fetch' fetch method. (github-package?): Simplify boolean expression. (github-repository, github-user-slash-repository): Strip trailing ".git" from project if present. (latest-release): Recognize a 'git-reference'. --- Hello Guix, With our increasing adoption of git checkouts from github, for reproducibility reasons, I thought it would be nice if our github updater could find updates for package origins that use `git-fetch`. This patch brings the github updater coverage up from 9.5% to 15.9%. At the time of this writing, the previous updater finds 254 updates, and with this patch finds 385! :) guix/import/github.scm | 30 +++++++++++++++++++----------- 1 file changed, 19 insertions(+), 11 deletions(-) diff --git a/guix/import/github.scm b/guix/import/github.scm index af9f56e1d..ad662e7b0 100644 --- a/guix/import/github.scm +++ b/guix/import/github.scm @@ -1,6 +1,7 @@ ;;; GNU Guix --- Functional package management for GNU ;;; Copyright © 2016 Ben Woodcroft ;;; Copyright © 2017, 2018 Ludovic Courtès +;;; Copyright © 2018 Eric Bavier ;;; ;;; This file is part of GNU Guix. ;;; @@ -24,6 +25,7 @@ #:use-module (srfi srfi-34) #:use-module (guix utils) #:use-module ((guix download) #:prefix download:) + #:use-module ((guix git-download) #:prefix download:) #:use-module (guix import utils) #:use-module (guix import json) #:use-module (guix packages) @@ -52,6 +54,7 @@ false if none is recognized" (github-user-slash-repository url))) (repo (github-repository url))) (cond + ((string-suffix? ".git" url) url) ((string-suffix? (string-append "/tarball/v" version) url) (string-append prefix "/tarball/v" new-version)) ((string-suffix? (string-append "/tarball/" version) url) @@ -86,26 +89,29 @@ false if none is recognized" (#t #f))) ; Some URLs are not recognised. #f)) - (let ((source-url (and=> (package-source old-package) origin-uri)) + (let ((source-uri (and=> (package-source old-package) origin-uri)) (fetch-method (and=> (package-source old-package) origin-method))) - (if (eq? fetch-method download:url-fetch) - (match source-url - ((? string?) - (updated-url source-url)) - ((source-url ...) - (find updated-url source-url))) - #f))) + (cond + ((eq? fetch-method download:url-fetch) + (match source-uri + ((? string?) + (updated-url source-uri)) + ((source-uri ...) + (find updated-url source-uri)))) + ((eq? fetch-method download:git-fetch) + (updated-url (download:git-reference-url source-uri))) + (else #f)))) (define (github-package? package) "Return true if PACKAGE is a package from GitHub, else false." - (not (eq? #f (updated-github-url package "dummy")))) + (->bool (updated-github-url package "dummy"))) (define (github-repository url) "Return a string e.g. bedtools2 of the name of the repository, from a string URL of the form 'https://github.com/arq5x/bedtools2/archive/v2.24.0.tar.gz'" (match (string-split (uri-path (string->uri url)) #\/) ((_ owner project . rest) - (string-append project)))) + (string-append (basename project ".git"))))) (define (github-user-slash-repository url) "Return a string e.g. arq5x/bedtools2 of the owner and the name of the @@ -113,7 +119,7 @@ repository separated by a forward slash, from a string URL of the form 'https://github.com/arq5x/bedtools2/archive/v2.24.0.tar.gz'" (match (string-split (uri-path (string->uri url)) #\/) ((_ owner project . rest) - (string-append owner "/" project)))) + (string-append owner "/" (basename project ".git"))))) (define %github-token ;; Token to be passed to Github.com to avoid the 60-request per hour @@ -213,6 +219,8 @@ https://github.com/settings/tokens")) (match (origin-uri origin) ((? string? url) url) ;surely a github.com URL + ((? download:git-reference? ref) + (download:git-reference-url ref)) ((urls ...) (find (cut string-contains <> "github.com") urls))))