diff mbox series

[bug#48320] lint: Verify if #:tests? is respected in the 'check' phase.

Message ID 2b0fee1845a66e1fb126b4bbf1c9892b7c648a3a.camel@telenet.be
State Accepted
Headers show
Series [bug#48320] lint: Verify if #:tests? is respected in the 'check' phase. | expand

Checks

Context Check Description
cbaines/comparison success View comparision
cbaines/git branch success View Git branch
cbaines/applying patch fail View Laminar job
cbaines/issue success View issue

Commit Message

M May 9, 2021, 6:02 p.m. UTC
Hi guix,

There have been a few patches to the mailing list lately not
respecting this, and this linter detects 325 package definitions
that could be modified to support the --without-tests package
transformation.

Copyright lines were added in the previous patch I sent to guix-patches
today.

Greetings,
Maxime

Comments

Mathieu Othacehe June 18, 2021, 12:15 p.m. UTC | #1
Hello Maxime,

> +      (`(,(or 'lambda 'lambda*) ,_ (invoke . ,_) . ,_)
> +       (list (make-warning package
> +                           ;; TRANSLATORS: check and #:tests? are a Scheme
> +                           ;; symbol and keyword respectively and should not
> +                           ;; be translated.
> +                           (G_ "the 'check' phase should respect #:tests?")
> +                           #:field 'arguments)))

I like the idea behind this patch. However I think the detection pattern
could be improved for instance, here are a few unreported packages:

- dejagnu
- python-dateutil
- eigen

Maybe we should check directly if the tests? variable is used within the
'check replace phase?

Thanks,

Mathieu
M June 18, 2021, 3:34 p.m. UTC | #2
Mathieu Othacehe schreef op vr 18-06-2021 om 14:15 [+0200]:
> Hello Maxime,
> 
> > +      (`(,(or 'lambda 'lambda*) ,_ (invoke . ,_) . ,_)
> > +       (list (make-warning package
> > +                           ;; TRANSLATORS: check and #:tests? are a Scheme
> > +                           ;; symbol and keyword respectively and should not
> > +                           ;; be translated.
> > +                           (G_ "the 'check' phase should respect #:tests?")
> > +                           #:field 'arguments)))

I just noticed the following test case in (tests lint) is somewhat bogus:

> +              '((replace 'check+
> +                  (lambda (#:key tests? #:allow-other-keys?)

Instead of 'lambda', this should be 'lambda*'.

Also, the value for #:phases can now be a G-expression,
so the usage of 'package-arguments' in the patch would need to be adjusted
as well.

> I like the idea behind this patch. However I think the detection pattern
> could be improved for instance, here are a few unreported packages:
> 
> - dejagnu
> - python-dateutil
> - eigen
> 
> Maybe we should check directly if the tests? variable is used within the
> 'check replace phase?

So, basically, test if applying the following procedure to the body
succeeds?

(define (sexp-uses-tests?? sexp)
  (sexp-contains-atom? sexp 'tests?))

(define (sexp-contains-atom? sexp atom)
  ; atoms are compared with eq? and vectors are currently not supported
  (if (pair? sexp)
      (or (sexp-contains? sexp atom)
          (sexp-contains? sexp atom))
      (eq? sexp atom)))

That seems a good improvement for a v2.

Thanks,
Maxime.
diff mbox series

Patch

From 77f6fdb0158d76af9a6789bd0da45ac852ee2868 Mon Sep 17 00:00:00 2001
From: Maxime Devos <maximedevos@telenet.be>
Date: Sun, 9 May 2021 19:53:31 +0200
Subject: [PATCH] lint: Verify if #:tests? is respected in the 'check' phase.

There have been a few patches to the mailing list lately not
respecting this, and this linter detects 325 package definitions
that could be modified to support the --without-tests package
transformation.

* guix/lint.scm
  (check-optional-tests): New linter.
  (%local-checkers)[optional-tests]: Add it.
* tests/lint.scm
  (package-with-phase-changes): New procedure.
  ("optional-tests: no check phase")
  ("optional-tests: check phase respects #:tests?")
  ("optional-tests: check phase ignores #:tests?")
  ("optional-tests: do not crash when #:phases is invalid"): New tests.
---
 guix/lint.scm  | 40 ++++++++++++++++++++++++++++++++++++++++
 tests/lint.scm | 31 +++++++++++++++++++++++++++++++
 2 files changed, 71 insertions(+)

diff --git a/guix/lint.scm b/guix/lint.scm
index d1cbc9d300..f5db4664dc 100644
--- a/guix/lint.scm
+++ b/guix/lint.scm
@@ -87,6 +87,7 @@ 
             check-source
             check-source-file-name
             check-source-unstable-tarball
+            check-optional-tests
             check-mirror-url
             check-github-url
             check-license
@@ -963,6 +964,41 @@  descriptions maintained upstream."
                     (origin-uris origin))
         '())))
 
+(define (check-optional-tests package)
+  "Emit a warning if the test suite is run unconditionally."
+  (define (check-check-procedure expression)
+    (match expression
+      (`(,(or 'lambda 'lambda*) ,_ (invoke . ,_) . ,_)
+       (list (make-warning package
+                           ;; TRANSLATORS: check and #:tests? are a Scheme
+                           ;; symbol and keyword respectively and should not
+                           ;; be translated.
+                           (G_ "the 'check' phase should respect #:tests?")
+                           #:field 'arguments)))
+      ;; The 'check' phase seems ok, stop searching for a bug in this package
+      ;; definition.
+      (_ '())))
+  (define (check-phases-delta delta)
+    (match delta
+      (`(replace 'check ,expression)
+       (check-check-procedure expression))
+      (_ #f)))
+  (define (check-phases-deltas deltas)
+    (match deltas
+      (() '())
+      ((head . tail)
+       (or (check-phases-delta head)
+           (check-phases-deltas tail)))
+      (_ (list (make-warning package
+                             (G_ "incorrect call to modify-phases")
+                             #:field 'arguments)))))
+  (apply (lambda* (#:key phases #:allow-other-keys)
+           (match phases
+             (`(modify-phases ,_ . ,changes)
+              (check-phases-deltas changes))
+             (_ '())))
+         (package-arguments package)))
+
 (define (check-mirror-url package)
   "Check whether PACKAGE uses source URLs that should be 'mirror://'."
   (define (check-mirror-uri uri)                  ;XXX: could be optimized
@@ -1529,6 +1565,10 @@  them for PACKAGE."
      (description "Make sure the 'license' field is a <license> \
 or a list thereof")
      (check       check-license))
+   (lint-checker
+     (name        'optional-tests)
+     (description "Make sure tests are only run when requested")
+     (check       check-optional-tests))
    (lint-checker
      (name        'mirror-url)
      (description "Suggest 'mirror://' URLs")
diff --git a/tests/lint.scm b/tests/lint.scm
index d6e877d0d7..c9cd6366ec 100644
--- a/tests/lint.scm
+++ b/tests/lint.scm
@@ -731,6 +731,37 @@ 
                                (sha256 %null-sha256))))))
     (check-source-unstable-tarball pkg)))
 
+(define (package-with-phase-changes changes)
+  (dummy-package "x"
+                 (arguments `(#:phases
+                              (modify-phases %standard-phases ,@changes)))))
+
+(test-equal "optional-tests: no check phase"
+  '()
+  (let ((pkg (package-with-phase-changes '())))
+    (check-optional-tests pkg)))
+(test-equal "optional-tests: check phase respects #:tests?"
+  '()
+  (let ((pkg (package-with-phase-changes
+              '((replace 'check
+                  (lambda (#:key tests? #:allow-other-keys?)
+                    (when tests?
+                      (invoke "./the-test-suite"))))))))
+    (check-optional-tests pkg)))
+(test-equal "optional-tests: check phase ignores #:tests?"
+  "the 'check' phase should respect #:tests?"
+  (let ((pkg (package-with-phase-changes
+              '((replace 'check
+                  (lambda _
+                    (invoke "./the-test-suite")))))))
+    (single-lint-warning-message
+     (check-optional-tests pkg))))
+(test-equal "optional-tests: do not crash when #:phases is invalid"
+  "incorrect call to modify-phases"
+  (let ((pkg (package-with-phase-changes 'this-is-not-a-list)))
+    (single-lint-warning-message
+     (check-optional-tests pkg))))
+
 (test-equal "source: 200"
   '()
   (with-http-server `((200 ,%long-string))
-- 
2.31.1