diff mbox series

[bug#46376] gnu: tesseract-ocr: update to 4.1.1)

Message ID CAJsg1E8qM-_eRdQy4OBFQMWDkw68gzPv_2xiDrNt0VTdHyYYGQ@mail.gmail.com
State Accepted
Headers show
Series [bug#46376] gnu: tesseract-ocr: update to 4.1.1) | expand

Checks

Context Check Description
cbaines/comparison success View comparision
cbaines/git branch success View Git branch
cbaines/applying patch fail View Laminar job
cbaines/issue success View issue

Commit Message

Andy Tai Feb. 11, 2021, midnight UTC
updated patch:

unit tests run, with some failures due to illegal instruction and
others succeed, but these requires first manual downloading of the
training data; I am not sure how that can be done as part of Guix
package definition.  Help on that is much appreciated.  (details
commented in the patch)

On Tue, Feb 9, 2021 at 11:52 PM Andy Tai <atai@atai.org> wrote:
>
> updated patch, now tests build in parallel... the build order has to
> be explicitly set to make the training target built first
>
> also added some other optional dependencies; built in a GuixSD VM to
> ensure no dependency on non-Guix tools from host
>
> test run is disabled for now
>
> On Tue, Feb 9, 2021 at 2:43 PM Jelle Licht <jlicht@posteo.net> wrote:
> >
> > Hi Andy,
> >
> > Andy Tai <atai@atai.org> writes:
> >
> > > Hi, I updated the patch to only build in serial, with "-j 1"
> > >
> > > and with this, everything, including tests, builds successfully.
> >
> > No such luck, for me at least. Are you certain you got it to build on
> > your end? Could you try with `--check`?
> >
> > I've had to work out the following things:
> >
> > - Patched out "<tesseract/baseapi.h>" and "<tesseract/helpers.h>" to
> >   refer to "baseapi.h" and "helpers.h" in "unittest/pagesegmode_test.cc".
> >
> > - Make sure the check phase takes place after running "make training" in
> >   a phase.
> >
> > I still ended up with several failing tests, courtesy of it running
> > unsupported instructions on my cpu (educated guess: avx etc). Nothing
> > comes easy, I guess.
> >
> > Thanks,
> >  - Jelle
>
>
>
> --
> Andy Tai, atai@atai.org, Skype: licheng.tai, Line: andy_tai, WeChat: andytai1010
> Year 2021 民國110年
> 自動的精神力是信仰與覺悟
> 自動的行為力是勞動與技能

Comments

Jelle Licht Feb. 11, 2021, 10:48 a.m. UTC | #1
Hey Andy,

Andy Tai <atai@atai.org> writes:

> updated patch:
>
> unit tests run, with some failures due to illegal instruction and
> others succeed, but these requires first manual downloading of the
> training data; I am not sure how that can be done as part of Guix
> package definition.  Help on that is much appreciated.  (details
> commented in the patch)

The illegal instruction stuff seems somehow problematic either way. The
training data seems to be generated via a not-really-trivial process, so
my guess is that to properly package this, we would really have to
generate the training data 'from source'. For now, it might make more
sense to have users "BYOTD" (bring your own training data) and leave it
out of the packaging story.

Andy, what would you think about dumbing down your patch so
tesseract-ocr simply doesn't run tests (or try to build them, for that
matter)? So sorry for sending you on this wild goose chase!

I'm willing to adjust and push such a change, unless someone has a
better idea.

Thanks,
 - Jelle
Andy Tai Feb. 11, 2021, 6:34 p.m. UTC | #2
Hi, sure.  Since the tests are built, we can just skip running them by
deleting the check phase.   That would be minimal change from the
(otherwise working) patch, also with no trouble from the missing
training data.

On Thu, Feb 11, 2021 at 2:48 AM Jelle Licht <jlicht@posteo.net> wrote:
>
> Hey Andy,
>
> Andy Tai <atai@atai.org> writes:
>
> > updated patch:
> >
> > unit tests run, with some failures due to illegal instruction and
> > others succeed, but these requires first manual downloading of the
> > training data; I am not sure how that can be done as part of Guix
> > package definition.  Help on that is much appreciated.  (details
> > commented in the patch)
>
> The illegal instruction stuff seems somehow problematic either way. The
> training data seems to be generated via a not-really-trivial process, so
> my guess is that to properly package this, we would really have to
> generate the training data 'from source'. For now, it might make more
> sense to have users "BYOTD" (bring your own training data) and leave it
> out of the packaging story.
>
> Andy, what would you think about dumbing down your patch so
> tesseract-ocr simply doesn't run tests (or try to build them, for that
> matter)? So sorry for sending you on this wild goose chase!
>
> I'm willing to adjust and push such a change, unless someone has a
> better idea.
>
> Thanks,
>  - Jelle
Jelle Licht Feb. 13, 2021, 2:34 p.m. UTC | #3
Hey Andy,

Andy Tai <atai@atai.org> writes:

> Hi, sure.  Since the tests are built, we can just skip running them by
> deleting the check phase.   That would be minimal change from the
> (otherwise working) patch, also with no trouble from the missing
> training data.

I pushed a slightly modified version of your patch as 822cd628e6 to
master; Instead of building the tests for no reason, I reverted the
recursive git checkout and I removed dependencies that are not required
for building tesseract from a git checkout.

Additionally, the manpages are now generated and installed as well.

Thanks again,
 - Jelle
Andy Tai Feb. 13, 2021, 8:37 p.m. UTC | #4
Thanks!

On Sat, Feb 13, 2021 at 6:34 AM Jelle Licht <jlicht@posteo.net> wrote:

> Hey Andy,
>
> Andy Tai <atai@atai.org> writes:
>
> > Hi, sure.  Since the tests are built, we can just skip running them by
> > deleting the check phase.   That would be minimal change from the
> > (otherwise working) patch, also with no trouble from the missing
> > training data.
>
> I pushed a slightly modified version of your patch as 822cd628e6 to
> master; Instead of building the tests for no reason, I reverted the
> recursive git checkout and I removed dependencies that are not required
> for building tesseract from a git checkout.
>
> Additionally, the manpages are now generated and installed as well.
>
> Thanks again,
>  - Jelle
>
diff mbox series

Patch

From ead97cb03c783bf6e941a93ca4f2a6c669451656 Mon Sep 17 00:00:00 2001
From: Andy Tai <atai@atai.org>
Date: Wed, 10 Feb 2021 15:56:48 -0800
Subject: [PATCH] gnu: tesseract-ocr: Update to 4.1.1)

* gnu/packages/ocr.scm (tesseract-ocr): Update to 4.1.1
---
 gnu/packages/ocr.scm | 85 ++++++++++++++++++++++++++++++++++++++++----
 1 file changed, 79 insertions(+), 6 deletions(-)

diff --git a/gnu/packages/ocr.scm b/gnu/packages/ocr.scm
index dc4930918a..962492ebb2 100644
--- a/gnu/packages/ocr.scm
+++ b/gnu/packages/ocr.scm
@@ -3,6 +3,7 @@ 
 ;;; Copyright © 2016, 2020 Efraim Flashner <efraim@flashner.co.il>
 ;;; Copyright © 2019 Tobias Geerinckx-Rice <me@tobias.gr>
 ;;; Copyright © 2019 Alex Vong <alexvong1995@gmail.com>
+;;; Copyright © 2021 Andy Tai <atai@atai.org>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -26,8 +27,18 @@ 
   #:use-module (guix git-download)
   #:use-module (guix build-system gnu)
   #:use-module (guix build-system python)
+  #:use-module (gnu packages)
+  #:use-module (gnu packages autotools)
+  #:use-module (gnu packages backup)
+  #:use-module (gnu packages check)
   #:use-module (gnu packages compression)
+  #:use-module (gnu packages curl)
+  #:use-module (gnu packages gtk)
+  #:use-module (gnu packages icu4c)
+  #:use-module (gnu packages pkg-config)
   #:use-module (gnu packages python)
+  #:use-module (gnu packages wget)
+  #:use-module (gnu packages xml)
   #:use-module (gnu packages image))
 
 (define-public ocrad
@@ -52,25 +63,87 @@  it produces text in 8-bit or UTF-8 formats.")
     (license license:gpl3+)))
 
 (define-public tesseract-ocr
+  ;; some useful commits beyond last official stable release in release branch
+  (let ((commit "97079fa353557af6df86fd20b5d2e0dff5d8d5df")
+        (revision "1"))
   (package
     (name "tesseract-ocr")
-    (version "3.04.01")
+    (version (git-version "4.1.1" revision commit))
     (source
      (origin
        (method git-fetch)
        (uri (git-reference
               (url "https://github.com/tesseract-ocr/tesseract")
-              (commit version)))
+              (commit commit)
+              ;; source git repo with submodules; ensure they are fetched
+              (recursive? #t)))
        (file-name (git-file-name name version))
        (sha256
-        (base32 "0h1x4z1h86n2gwknd0wck6gykkp99bmm02lg4a47a698g4az6ybv"))))
+        (base32 "0axwla82fpzp86lc553wp3hk0fz5dylw4as0jbf4hkqcyajlbzp4"))))
     (build-system gnu-build-system)
     (inputs
-     `(("leptonica" ,leptonica)))
+     `(  ("cairo" ,cairo)
+         ("icu" ,icu4c)
+         ("leptonica" ,leptonica)
+         ("pango" ,pango)
+         ("wget" ,wget)  ;; for downloading training data to run unit tests
+         ))
+    (native-inputs
+     `(("autoconf" ,autoconf)
+       ("autoconf-archive" ,autoconf-archive)
+       ("automake" ,automake)
+       ("googletest" ,googletest)
+       ("libarchive" ,libarchive)
+       ("libcurl" ,curl)
+       ("libtool" ,libtool)
+       ("libtiff" ,libtiff)
+       ("pkg-config" ,pkg-config)
+       ("python" ,python-wrapper)
+       ("xsltproc" ,libxslt)))
     (arguments
      '(#:configure-flags
        (let ((leptonica (assoc-ref %build-inputs "leptonica")))
-         (list (string-append "LIBLEPT_HEADERSDIR=" leptonica "/include")))))
+         (list (string-append "LIBLEPT_HEADERSDIR=" leptonica "/include")))
+       #:phases
+       (modify-phases %standard-phases
+         (add-before 'configure 'disable-failing-tests-and-setup
+           (lambda _
+             ;; pagesegmode_test.cc fails to build, patch it
+             (substitute* "unittest/pagesegmode_test.cc"
+               (("<tesseract/baseapi.h>") "\"baseapi.h\""))
+             (substitute* "unittest/pagesegmode_test.cc"
+               (("<tesseract/helpers.h>") "\"helpers.h\""))
+             #t))
+         (add-before 'build 'build-training
+           (lambda _
+             (invoke "make" "-j" (number->string (parallel-job-count)) "training")))
+         (add-after 'install 'install-training
+           (lambda _
+             (invoke "make" "training-install")
+             #t))
+         (replace 'check
+           (lambda _
+             (status:exit-val (system* "make" "check")) ;;exit code ignored
+             #t))  ;; failed tests will not stop the whole instal process
+         (add-before 'check 'pre-check-setup
+           (lambda* (#:key inputs outputs #:allow-other-keys)
+             (let ((tessdata_prefix   "/tmp") ;; (tmpnam))
+                   (wget   (which "wget")))
+               ;;TESSDATA_PREFIX environment var shall be parent directory of tessdata directory
+               ;; note for now, to run tests successfully you need to manually download eng.traineddata
+               ;; to /tmp/tessdata first
+                 (if tessdata_prefix
+                   (let ((data_dir (string-append tessdata_prefix "/tessdata")))
+                     (setenv "TESSDATA_PREFIX" tessdata_prefix)
+                     (format #t  "TESSDATA_PREFIX data dir: ~a  " data_dir)
+                     (mkdir-p data_dir)  ; code below shows attempt to download; not working now
+                     ;;(with-directory-excursion data_dir
+                     ;;  (begin
+                     ;;    (invoke wget "-t" "5" "https://github.com/tesseract-ocr/tessdata/blob/master/eng.traineddata")
+                     ;;))
+                   )
+                   (format #t "No TESSDATA_PREFIX found "))
+             #t))))))
     (home-page "https://github.com/tesseract-ocr/tesseract")
     (synopsis "Optical character recognition engine")
     (description
@@ -79,7 +152,7 @@  high accuracy.  It supports many languages, output text formatting, hOCR
 positional information and page layout analysis.  Several image formats are
 supported through the Leptonica library.  It can also detect whether text is
 monospaced or proportional.")
-    (license license:asl2.0)))
+    (license license:asl2.0))))
 
 (define-public zinnia
   (let* ((commit "581faa8f6f15e4a7b21964be3a5ec36265c80e5b")
-- 
2.30.0