| Message ID | 20220422162024.91287-1-paul@apatience.com |
|---|---|
| State | Accepted |
| Headers | show |
| Series | [bug#55067] gnu: nomad-optimizer: Disable failing test. | expand |
| Context | Check | Description |
|---|---|---|
| cbaines/comparison | success | View comparision |
| cbaines/git branch | success | View Git branch |
| cbaines/applying patch | success | View Laminar job |
| cbaines/issue | success | View issue |
Paul A. Patience schreef op vr 22-04-2022 om 16:20 [+0000]: > --- > It turns out the PSDMads test fails randomly. Sounds like a bug, preferably it would 100% reliably report the underlying problem, whatever it is. > I've disabled it. I don't think this is a good reason to disable the test. Disabling tests whenever they fail defeats the point of tests. Cf. with, say, clinical trials for testing safety of new medical treatments. If some drug fails one of the tests (sometimes or always), then one cannot simply ignore the test. Greetings, Maxime.
On 2022-04-22 13:25:20-04:00, Maxime Devos wrote: >> It turns out the PSDMads test fails randomly. > > Sounds like a bug, preferably it would 100% reliably report the > underlying problem, whatever it is. I agree with you, and I have already reported the issue to the developers (along with some other issues that I addressed in the package definition). >> I've disabled it. > > I don't think this is a good reason to disable the test. > Disabling tests whenever they fail defeats the point of tests. This test failure wasn't getting detected in the past due to a missing "set -o pipefail" in the test runner, which could explain why it was missed by the developers. I don't know enough to fix the bug myself, but I have a suspicion it is a bug in the test rather than in the library. Best regards, Paul
Paul A. Patience schreef op vr 22-04-2022 om 19:02 [+0000]: > On 2022-04-22 13:25:20-04:00, Maxime Devos wrote: > > > It turns out the PSDMads test fails randomly. > > > > Sounds like a bug, preferably it would 100% reliably report the > > underlying problem, whatever it is. > > I agree with you, and I have already reported the issue to the > developers (along with some other issues that I addressed in the package > definition). Then I recommend adding a link to the upstream report (in a comment), such that at the next update we know where to check if it has been resolved, and to avoid giving the impression that just skipping tests is acceptable. Greetings, Maxime.
diff --git a/gnu/packages/maths.scm b/gnu/packages/maths.scm index f4778179d6..26488a614c 100644 --- a/gnu/packages/maths.scm +++ b/gnu/packages/maths.scm @@ -2008,6 +2008,9 @@ (define-public nomad-optimizer (add-after 'fix-sources-for-build 'fix-sources-for-tests (lambda _ (substitute* "examples/CMakeLists.txt" + ;; This test passes only sometimes. + (("^ +add_subdirectory\\(\\$\\{CMAKE_CURRENT_SOURCE_DIR\\}/advanced/library/PSDMads\\)\n") + "") ;; examples/basic/batch/example3 is accidentally omitted. (("^(add_subdirectory\\(\\$\\{CMAKE_CURRENT_SOURCE_DIR\\}/basic/batch/example)2(\\)\n)" _ prefix suffix) @@ -2052,10 +2055,6 @@ (define-public nomad-optimizer "examples/advanced/library/exampleSuggestAndObserve/cache0.txt") (let ((builddir (string-append (getcwd) "/../build"))) - ;; For some reason, omitting this fix causes the - ;; examples/advanced/library/PSDMads test to fail, even though - ;; it doesn't seem to reference any part of the FixedVariable - ;; test. (let ((dir "examples/advanced/library/FixedVariable")) (substitute* (string-append dir "/fixedVariable.cpp") (("^( +std::string sExe = ).*" _ prefix)