[bug#55538] gnu: virtuoso-ose: Update to 7.2.7.

On Wed, 2022-05-25 at 17:51 +0200, Roel Janssen wrote:
> On Wed, 2022-05-25 at 10:29 -0400, Maxim Cournoyer wrote:
> > Hi Roel,
> > 
> > Roel Janssen <roel@gnu.org> writes:
> > 
> > > Dear Guix,
> > > 
> > > I'd like to update Virtuoso OSE to the latest release (see attached patch).
> > > 
> > > Other than the version number and checksum bump I noticed that some JAR files made it in the
> > > build
> > > output.  I tried removing them from the source tarball using a snippet, but then one needs to
> > > include a patch for various Makefile.am files and run the autogen.sh script.  I thought this
> > > was
> > > simpler and clearer on what's actually achieved.
> > 
> > I fear in the future it would be easy to overlook the introduction of
> > extra bundled jars in the source, that'd get installed.  I feel it'd be
> > preferable if we removed them all from a source snippet, at the cost of
> > having to patch the build system (the extra complications you
> > mentioned).
> > 
> > Could you try it and send a revised patch?
> > 
> 
> Thank you for taking a look at it! To be fair, what the patches would do, would be to avoid
> including the directories that I remove. So removing the current pre-built binaries in a snippet
> will be equally error-prone.
> 
> But I agree that you would preferably want to have a source tarball without pre-built binaries in
> it
> when running `guix build -S virtuoso-ose`.
> 
> A naive patch leads to the build getting stuck, so I need to investigate the impact further.
> I'll report back when I have adapted the patch.
> 
> THank you again for your time!

I attached an updated patch that removes the pre-built binaries in a snippet. It then removes these
directories from the build system which in turn requires one to re-run autogen.sh.

Kind regards,
Roel Janssen

On Fri, May 27, 2022 at 12:47:18PM +0200, Roel Janssen wrote:
> On Wed, 2022-05-25 at 17:51 +0200, Roel Janssen wrote:
> > On Wed, 2022-05-25 at 10:29 -0400, Maxim Cournoyer wrote:
> > > Hi Roel,
> > > 
> > > Roel Janssen <roel@gnu.org> writes:
> > > 
> > > > Dear Guix,
> > > > 
> > > > I'd like to update Virtuoso OSE to the latest release (see attached patch).
> > > > 
> > > > Other than the version number and checksum bump I noticed that some JAR files made it in the
> > > > build
> > > > output.  I tried removing them from the source tarball using a snippet, but then one needs to
> > > > include a patch for various Makefile.am files and run the autogen.sh script.  I thought this
> > > > was
> > > > simpler and clearer on what's actually achieved.
> > > 
> > > I fear in the future it would be easy to overlook the introduction of
> > > extra bundled jars in the source, that'd get installed.  I feel it'd be
> > > preferable if we removed them all from a source snippet, at the cost of
> > > having to patch the build system (the extra complications you
> > > mentioned).
> > > 
> > > Could you try it and send a revised patch?
> > > 
> > 
> > Thank you for taking a look at it! To be fair, what the patches would do, would be to avoid
> > including the directories that I remove. So removing the current pre-built binaries in a snippet
> > will be equally error-prone.
> > 
> > But I agree that you would preferably want to have a source tarball without pre-built binaries in
> > it
> > when running `guix build -S virtuoso-ose`.
> > 
> > A naive patch leads to the build getting stuck, so I need to investigate the impact further.
> > I'll report back when I have adapted the patch.
> > 
> > THank you again for your time!
> 
> I attached an updated patch that removes the pre-built binaries in a snippet. It then removes these
> directories from the build system which in turn requires one to re-run autogen.sh.
> 
> Kind regards,
> Roel Janssen
> 

> From 1f3b9db55cac790711ece613f5ebeb51474764a7 Mon Sep 17 00:00:00 2001
> From: Roel Janssen <roel@gnu.org>
> Date: Fri, 27 May 2022 12:40:46 +0200
> Subject: [PATCH] gnu: virtuoso-ose: Update to 7.2.7.
> 
> * gnu/packages/databases.scm (virtuoso-ose): Update to 7.2.7; Run autogen
>   after applying the patch below.
> * gnu/packages/patches/virtuoso-ose-remove-pre-built-jar-files.patch:
>   New file.
> ---
>  gnu/packages/databases.scm                    |  26 +++-
>  ...tuoso-ose-remove-pre-built-jar-files.patch | 117 ++++++++++++++++++
>  2 files changed, 141 insertions(+), 2 deletions(-)
>  create mode 100644 gnu/packages/patches/virtuoso-ose-remove-pre-built-jar-files.patch
> 
> diff --git a/gnu/packages/databases.scm b/gnu/packages/databases.scm
> index 2161340ef9..58b6dd088c 100644
> --- a/gnu/packages/databases.scm
> +++ b/gnu/packages/databases.scm
> @@ -100,6 +100,7 @@ (define-module (gnu packages databases)
>    #:use-module (gnu packages gnome)
>    #:use-module (gnu packages gnupg)
>    #:use-module (gnu packages golang)
> +  #:use-module (gnu packages gperf)
>    #:use-module (gnu packages gtk)
>    #:use-module (gnu packages guile)
>    #:use-module (gnu packages icu4c)
> @@ -3229,14 +3230,30 @@ (define-public python-lmdb
>  (define-public virtuoso-ose
>    (package
>      (name "virtuoso-ose")
> -    (version "7.2.6")
> +    (version "7.2.7")
>      (source
>       (origin
>         (method url-fetch)
>         (uri (string-append "mirror://sourceforge/virtuoso/virtuoso/" version "/"
>                             "virtuoso-opensource-" version ".tar.gz"))
>         (sha256
> -        (base32 "0ly7s7a3w2a2zhhi9rq9k2qlnzapqbbc1rcdqb3zqqpgg81krz9q"))))
> +        (base32 "1853ln0smiilf3pni70gq6nmi9ps039cy44g6b5i9d2z1n9hnj02"))
> +       (patches (search-patches "virtuoso-ose-remove-pre-built-jar-files.patch"))
> +       (modules '((guix build utils)))
> +       ;; This snippet removes pre-built Java archives.
> +       (snippet
> +        '(begin
> +           (delete-file-recursively "libsrc/JDBCDriverType4")
> +           (delete-file-recursively "binsrc/hibernate")
> +           (delete-file-recursively "binsrc/jena")
> +           (delete-file-recursively "binsrc/jena2")
> +           (delete-file-recursively "binsrc/jena3")
> +           (delete-file-recursively "binsrc/jena4")
> +           (delete-file-recursively "binsrc/rdf4j")
> +           (delete-file-recursively "binsrc/sesame")
> +           (delete-file-recursively "binsrc/sesame2")
> +           (delete-file-recursively "binsrc/sesame3")
> +           (delete-file-recursively "binsrc/sesame4")))))

how about
(for-each delete-file-recursively
  (list "libsrc/JDBCDriverType4"
        "binsrc/hibernate"
        ...))

>      (build-system gnu-build-system)
>      (arguments
>       `(#:tests? #f ; Tests require a network connection.
> @@ -3247,6 +3264,9 @@ (define-public virtuoso-ose
>                             "--enable-static=no")
>         #:phases
>         (modify-phases %standard-phases
> +         (add-before 'configure 'autogen
> +           (lambda _
> +             (invoke "./autogen.sh")))

This would probably be better as (replace 'bootstrap

>           ;; Even with "--enable-static=no", "libvirtuoso-t.a" is left in
>           ;; the build output.  The following phase removes it.
>           (add-after 'install 'remove-static-libs
> @@ -3256,6 +3276,8 @@ (define-public virtuoso-ose
>                             (delete-file (string-append lib "/" file)))
>                           '("libvirtuoso-t.a"
>                             "libvirtuoso-t.la"))))))))
> +    (native-inputs
> +     (list autoconf automake libtool bison flex gperf))

That's more than I expected!

>      (inputs
>       (list openssl net-tools readline zlib))
>      (home-page "http://vos.openlinksw.com/owiki/wiki/VOS/")
> diff --git a/gnu/packages/patches/virtuoso-ose-remove-pre-built-jar-files.patch b/gnu/packages/patches/virtuoso-ose-remove-pre-built-jar-files.patch
> new file mode 100644
> index 0000000000..17413c71ae
> --- /dev/null
> +++ b/gnu/packages/patches/virtuoso-ose-remove-pre-built-jar-files.patch
> @@ -0,0 +1,117 @@
> +This patch disables build targets that contain pre-built Java archives that
> +would be copied to the build's output.
> +
> +Patch by Roel Janssen <roel@gnu.org>
> +*** a-virtuoso-opensource-7.2.7/binsrc/Makefile.am	1970-01-01 01:00:01.000000000 +0100
> +--- b-virtuoso-opensource-7.2.7/binsrc/Makefile.am	2022-05-27 12:20:52.909135774 +0200
> +***************
> +*** 19,25 ****
> +  #  
> +  #  
> +  
> +! SUBDIRS = dav mono virtuoso tests rdf_mappers driver maildrop sqldoc hosting bpel fct tutorial conductor samples vsp ws sync vspx vad cached_resources virtodbc virtoledb virtuoso_sink xddl VirtuosoClient.Net oat isparql jena jena2 jena3 jena4 sesame sesame2 sesame3 sesame4 redland hibernate dbpedia rdb2rdf rdf4j
> +  
> +  
> +  # ----------------------------------------------------------------------
> +--- 19,25 ----
> +  #  
> +  #  
> +  
> +! SUBDIRS = dav mono virtuoso tests rdf_mappers driver maildrop sqldoc hosting bpel fct tutorial conductor samples vsp ws sync vspx vad cached_resources virtodbc virtoledb virtuoso_sink xddl VirtuosoClient.Net oat isparql redland dbpedia rdb2rdf
> +  
> +  
> +  # ----------------------------------------------------------------------
> +*** a-virtuoso-opensource-7.2.7/configure.ac	1970-01-01 01:00:01.000000000 +0100
> +--- b-virtuoso-opensource-7.2.7/configure.ac	2022-05-27 12:27:51.879208018 +0200
> +***************
> +*** 57,64 ****
> +  dnl	AM_INIT_AUTOMAKE([1.8])
> +  dnl	AM_INIT_AUTOMAKE([1.9 tar-ustar])
> +  dnl
> +! AM_INIT_AUTOMAKE([1.9 tar-ustar])
> +! 
> +  AM_MAINTAINER_MODE
> +  
> +  
> +--- 57,63 ----
> +  dnl	AM_INIT_AUTOMAKE([1.8])
> +  dnl	AM_INIT_AUTOMAKE([1.9 tar-ustar])
> +  dnl
> +! AM_INIT_AUTOMAKE([1.9 tar-ustar subdir-objects])
> +  AM_MAINTAINER_MODE
> +  
> +  
> +***************
> +*** 3157,3163 ****
> +  	binsrc/dbpedia/Makefile
> +  	binsrc/driver/Makefile
> +  	binsrc/fct/Makefile
> +- 	binsrc/hibernate/Makefile
> +  	binsrc/hosting/Makefile
> +  	binsrc/hosting/mono/Makefile
> +  	binsrc/hosting/mono/tests/Makefile
> +--- 3156,3161 ----
> +***************
> +*** 3169,3184 ****
> +  	binsrc/hosting/ruby/Makefile
> +  	binsrc/hosting/shapefileio/Makefile
> +  	binsrc/isparql/Makefile
> +- 	binsrc/jena/Makefile
> +- 	binsrc/jena2/Makefile
> +- 	binsrc/jena3/Makefile
> +- 	binsrc/jena4/Makefile
> +  	binsrc/maildrop/Makefile
> +  	binsrc/mono/Makefile
> +  	binsrc/oat/Makefile
> +  	binsrc/rdf_mappers/Makefile
> +  	binsrc/rdb2rdf/Makefile
> +- 	binsrc/rdf4j/Makefile
> +  	binsrc/redland/Makefile
> +  	binsrc/samples/demo/Makefile
> +  	binsrc/samples/hslookup/Makefile
> +--- 3167,3177 ----
> +***************
> +*** 3191,3200 ****
> +  	binsrc/samples/webapp/Makefile
> +  	binsrc/samples/xpath/Makefile
> +  	binsrc/samples/xquery/Makefile
> +- 	binsrc/sesame/Makefile
> +- 	binsrc/sesame2/Makefile
> +- 	binsrc/sesame3/Makefile
> +- 	binsrc/sesame4/Makefile
> +  	binsrc/sqldoc/Makefile
> +  	binsrc/sync/Makefile
> +  	binsrc/tests/biftest/Makefile
> +--- 3184,3189 ----
> +***************
> +*** 3236,3245 ****
> +  	docsrc/stylesheets/Makefile
> +  	docsrc/xmlsource/Makefile
> +  	libsrc/Dk/Makefile
> +- 	libsrc/JDBCDriverType4/Makefile
> +- 	libsrc/JDBCDriverType4/testsuite_4.0/Makefile
> +- 	libsrc/JDBCDriverType4/virtuoso/jdbc/Makefile
> +- 	libsrc/JDBCDriverType4/virtuoso/Makefile
> +  	libsrc/langfunc/Makefile
> +  	libsrc/odbcsdk/Makefile
> +  	libsrc/plugin/Makefile
> +--- 3225,3230 ----
> +*** a-virtuoso-opensource-7.2.7/libsrc/Makefile.am	1970-01-01 01:00:01.000000000 +0100
> +--- b-virtuoso-opensource-7.2.7/libsrc/Makefile.am	2022-05-27 12:30:12.658593011 +0200
> +***************
> +*** 19,25 ****
> +  #  
> +  #  
> +  
> +! SUBDIRS = util zlib odbcsdk Dk Thread langfunc Wi plugin Tidy Xml.new JDBCDriverType4
> +  
> +  noinst_HEADERS = Dk.h libutil.h
> +  
> +--- 19,25 ----
> +  #  
> +  #  
> +  
> +! SUBDIRS = util zlib odbcsdk Dk Thread langfunc Wi plugin Tidy Xml.new
> +  
> +  noinst_HEADERS = Dk.h libutil.h
> +  
> -- 
> 2.36.1
>

Hello,

Roel Janssen <roel@gnu.org> writes:

> On Wed, 2022-05-25 at 17:51 +0200, Roel Janssen wrote:
>> On Wed, 2022-05-25 at 10:29 -0400, Maxim Cournoyer wrote:
>> > Hi Roel,
>> > 
>> > Roel Janssen <roel@gnu.org> writes:
>> > 
>> > > Dear Guix,
>> > > 
>> > > I'd like to update Virtuoso OSE to the latest release (see attached patch).
>> > > 
>> > > Other than the version number and checksum bump I noticed that some JAR files made it in the
>> > > build
>> > > output.  I tried removing them from the source tarball using a snippet, but then one needs to
>> > > include a patch for various Makefile.am files and run the autogen.sh script.  I thought this
>> > > was
>> > > simpler and clearer on what's actually achieved.
>> > 
>> > I fear in the future it would be easy to overlook the introduction of
>> > extra bundled jars in the source, that'd get installed.  I feel it'd be
>> > preferable if we removed them all from a source snippet, at the cost of
>> > having to patch the build system (the extra complications you
>> > mentioned).
>> > 
>> > Could you try it and send a revised patch?
>> > 
>> 
>> Thank you for taking a look at it! To be fair, what the patches would do, would be to avoid
>> including the directories that I remove. So removing the current pre-built binaries in a snippet
>> will be equally error-prone.

The way I deal with this is to have an allow list of things to preserve
in the source instead of a deny list; this way if new binaries get added
and need to be handled, the build will break instead of using the
bundled binaries silently.

Thanks for the package update (and to Efraim for picking up the review
where I had left it)!

Maxim