From patchwork Sat Dec 22 10:08:39 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Arun Isaac X-Patchwork-Id: 519 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id ED5AD16928; Sat, 22 Dec 2018 10:15:42 +0000 (GMT) X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-1.1 required=5.0 tests=BAYES_00,RCVD_IN_SORBS_WEB, T_DKIM_INVALID,URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0 Received: from lists.nongnu.org (lists.nongnu.org [IPv6:2001:4830:134:3::12]) by mira.cbaines.net (Postfix) with ESMTP id 4891E16917 for ; Sat, 22 Dec 2018 10:15:42 +0000 (GMT) Received: from localhost ([::1]:40850 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gaeJp-0000Q6-Kj for patchwork@mira.cbaines.net; Sat, 22 Dec 2018 05:15:41 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:46081) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gaeDQ-0003wA-VY for guix-patches@gnu.org; Sat, 22 Dec 2018 05:09:08 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gaeDO-0006nE-LA for guix-patches@gnu.org; Sat, 22 Dec 2018 05:09:04 -0500 Received: from debbugs.gnu.org ([208.118.235.43]:54907) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1gaeDO-0006n1-HL for guix-patches@gnu.org; Sat, 22 Dec 2018 05:09:02 -0500 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1gaeDO-0006rp-7h for guix-patches@gnu.org; Sat, 22 Dec 2018 05:09:02 -0500 X-Loop: help-debbugs@gnu.org Subject: [bug#33801] import: github: Support source URIs that redirect to GitHub Resent-From: Arun Isaac Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Sat, 22 Dec 2018 10:09:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 33801 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: To: Ludovic =?utf-8?q?Court=C3=A8s?= Received: via spool by 33801-submit@debbugs.gnu.org id=B33801.154547333526386 (code B ref 33801); Sat, 22 Dec 2018 10:09:02 +0000 Received: (at 33801) by debbugs.gnu.org; 22 Dec 2018 10:08:55 +0000 Received: from localhost ([127.0.0.1]:59165 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gaeDH-0006rT-BS for submit@debbugs.gnu.org; Sat, 22 Dec 2018 05:08:55 -0500 Received: from vultr.systemreboot.net ([45.77.148.100]:51640) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1gaeDF-0006rL-Go for 33801@debbugs.gnu.org; Sat, 22 Dec 2018 05:08:54 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=systemreboot.net; s=default; h=Content-Type:MIME-Version:Message-ID:Date: References:In-Reply-To:Subject:Cc:To:From:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Id: List-Help:List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=1bT0fMhemWBreeCWJWojNwx/ob8SLTwZRQWQt/AU34g=; b=O85DSzuaX/FMywMabcVyn0GfK n812wA+5jsor8Ux70+7sqPto1UuVRfPMw1yJSzZ0Ki+w6+6Ri0+BfMekgz9pDThqGcPnpZzObGwgo ob3BkGAuWtGzxYl/YU5agPVvo9o1vs8DcjYTtsff5+w8pu0mmoaEQJjRyRy+ESojzvnNo=; Received: from [103.5.134.173] (helo=steel) by systemreboot.net with esmtpsa (TLSv1.2:ECDHE-RSA-AES256-GCM-SHA384:256) (Exim 4.91) (envelope-from ) id 1gaeDB-000645-8h; Sat, 22 Dec 2018 15:38:50 +0530 From: Arun Isaac In-Reply-To: <877eg2q56k.fsf@gnu.org> References: <87d0pxtciz.fsf@gnu.org> <87woo4qxg7.fsf@gnu.org> <87k1k4p3gl.fsf@gnu.org> <877eg2q56k.fsf@gnu.org> Date: Sat, 22 Dec 2018 15:38:39 +0530 Message-ID: MIME-Version: 1.0 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 208.118.235.43 X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: 33801@debbugs.gnu.org Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: "Guix-patches" X-getmail-retrieved-from-mailbox: Patches Please find attached an updated patch. From de88021c9a73d28f11bc2e060098484bd414da62 Mon Sep 17 00:00:00 2001 From: Arun Isaac Date: Fri, 21 Dec 2018 17:48:55 +0530 Subject: [PATCH] guix: lint: Check for source URIs redirecting to GitHub. * guix/scripts/lint.scm (check-github-uri): New procedure. (%checkers): Add it. * doc/guix.texi (Invoking guix lint): Document it. * tests/lint.scm ("github-url", "github-url: one suggestion"): New tests. --- doc/guix.texi | 10 ++++++---- guix/scripts/lint.scm | 39 +++++++++++++++++++++++++++++++++++++++ tests/lint.scm | 28 ++++++++++++++++++++++++++++ 3 files changed, 73 insertions(+), 4 deletions(-) diff --git a/doc/guix.texi b/doc/guix.texi index 8f6a8b3ed..62e0454cc 100644 --- a/doc/guix.texi +++ b/doc/guix.texi @@ -7659,12 +7659,14 @@ Identify inputs that should most likely be native inputs. @item source @itemx home-page @itemx mirror-url +@itemx github-url @itemx source-file-name Probe @code{home-page} and @code{source} URLs and report those that are -invalid. Suggest a @code{mirror://} URL when applicable. Check that -the source file name is meaningful, e.g.@: is not -just a version number or ``git-checkout'', without a declared -@code{file-name} (@pxref{origin Reference}). +invalid. Suggest a @code{mirror://} URL when applicable. If the +@code{source} URL redirects to a GitHub URL, recommend usage of the GitHub +URL. Check that the source file name is meaningful, e.g.@: is not just a +version number or ``git-checkout'', without a declared @code{file-name} +(@pxref{origin Reference}). @item cve @cindex security vulnerabilities diff --git a/guix/scripts/lint.scm b/guix/scripts/lint.scm index 2314f3b28..354f6f703 100644 --- a/guix/scripts/lint.scm +++ b/guix/scripts/lint.scm @@ -8,6 +8,7 @@ ;;; Copyright © 2017 Alex Kost ;;; Copyright © 2017 Tobias Geerinckx-Rice ;;; Copyright © 2017 Efraim Flashner +;;; Copyright © 2018 Arun Isaac ;;; ;;; This file is part of GNU Guix. ;;; @@ -44,8 +45,10 @@ #:use-module (guix cve) #:use-module (gnu packages) #:use-module (ice-9 match) + #:use-module (ice-9 receive) #:use-module (ice-9 regex) #:use-module (ice-9 format) + #:use-module (web client) #:use-module (web uri) #:use-module ((guix build download) #:select (maybe-expand-mirrors @@ -74,6 +77,7 @@ check-source check-source-file-name check-mirror-url + check-github-url check-license check-vulnerabilities check-for-updates @@ -773,6 +777,37 @@ descriptions maintained upstream." (let ((uris (origin-uris origin))) (for-each check-mirror-uri uris))))) +(define (check-github-url package) + "Check whether PACKAGE uses source URLs that redirect to GitHub." + (define (follow-redirect uri) + (receive (response body) (http-head uri) + (case (response-code response) + ((301 302) + (uri->string (assoc-ref (response-headers response) 'location))) + (else #f)))) + + (define (follow-redirects-to-github uri) + (cond + ((string-prefix? "https://github.com/" uri) uri) + ((string-prefix? "http" uri) + (and=> (follow-redirect uri) follow-redirects-to-github)) + ;; Do not attempt to follow redirects on URIs other than http and https + ;; (such as mirror, file) + (else #f))) + + (let ((origin (package-source package))) + (when (and (origin? origin) + (eqv? (origin-method origin) url-fetch)) + (for-each + (lambda (uri) + (and=> (follow-redirects-to-github uri) + (lambda (github-uri) + (emit-warning + package + (format #f (G_ "URL should be '~a'") github-uri) + 'source)))) + (origin-uris origin))))) + (define (check-derivation package) "Emit a warning if we fail to compile PACKAGE to a derivation." (define (try system) @@ -1055,6 +1090,10 @@ or a list thereof") (name 'mirror-url) (description "Suggest 'mirror://' URLs") (check check-mirror-url)) + (lint-checker + (name 'github-uri) + (description "Suggest GitHub URIs") + (check check-github-url)) (lint-checker (name 'source-file-name) (description "Validate file names of sources") diff --git a/tests/lint.scm b/tests/lint.scm index 300153e24..d4aa7c0e8 100644 --- a/tests/lint.scm +++ b/tests/lint.scm @@ -6,6 +6,7 @@ ;;; Copyright © 2016 Hartmut Goebel ;;; Copyright © 2017 Alex Kost ;;; Copyright © 2017 Efraim Flashner +;;; Copyright © 2018 Arun Isaac ;;; ;;; This file is part of GNU Guix. ;;; @@ -669,6 +670,33 @@ (check-mirror-url (dummy-package "x" (source source))))) "mirror://gnu/foo/foo.tar.gz")) +(test-assert "github-url" + (string-null? + (with-warnings + (with-http-server 200 %long-string + (check-github-url + (dummy-package "x" (source + (origin + (method url-fetch) + (uri (%local-url)) + (sha256 %null-sha256))))))))) + +(let ((github-url "https://github.com/foo/bar/bar-1.0.tar.gz")) + (test-assert "github-url: one suggestion" + (string-contains + (with-warnings + (with-http-server (301 `((location . ,(string->uri github-url)))) "" + (let ((initial-uri (%local-url))) + (parameterize ((%http-server-port (+ 1 (%http-server-port)))) + (with-http-server (302 `((location . ,(string->uri initial-uri)))) "" + (check-github-url + (dummy-package "x" (source + (origin + (method url-fetch) + (uri (%local-url)) + (sha256 %null-sha256)))))))))) + github-url))) + (test-assert "cve" (mock ((guix scripts lint) package-vulnerabilities (const '())) (string-null? -- 2.19.2