From patchwork Thu Jun 13 15:07:21 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Pierre Langlois X-Patchwork-Id: 14302 Return-Path: X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id 184ED170CB; Thu, 13 Jun 2019 16:52:38 +0100 (BST) X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,FREEMAIL_FROM, T_DKIM_INVALID,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTP id A012C170BE for ; Thu, 13 Jun 2019 16:52:37 +0100 (BST) Received: from localhost ([::1]:41144 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hbS1l-0005sw-A1 for patchwork@mira.cbaines.net; Thu, 13 Jun 2019 11:52:37 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:52004) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1hbRKe-0001bV-BN for guix-patches@gnu.org; Thu, 13 Jun 2019 11:08:07 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hbRKc-0002Rb-QI for guix-patches@gnu.org; Thu, 13 Jun 2019 11:08:04 -0400 Received: from debbugs.gnu.org ([209.51.188.43]:50557) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1hbRKc-0002RT-Mg for guix-patches@gnu.org; Thu, 13 Jun 2019 11:08:02 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1hbRKc-0004DT-EZ for guix-patches@gnu.org; Thu, 13 Jun 2019 11:08:02 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#36194] [PATCH 10/10] gnu: Add csvkit. Resent-From: Pierre Langlois Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Thu, 13 Jun 2019 15:08:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 36194 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 36194@debbugs.gnu.org Received: via spool by 36194-submit@debbugs.gnu.org id=B36194.156043845116159 (code B ref 36194); Thu, 13 Jun 2019 15:08:02 +0000 Received: (at 36194) by debbugs.gnu.org; 13 Jun 2019 15:07:31 +0000 Received: from localhost ([127.0.0.1]:35868 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hbRK7-0004CY-51 for submit@debbugs.gnu.org; Thu, 13 Jun 2019 11:07:31 -0400 Received: from mout.gmx.net ([212.227.15.18]:41437) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hbRK4-0004CL-T2 for 36194@debbugs.gnu.org; Thu, 13 Jun 2019 11:07:29 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gmx.net; s=badeba3b8450; t=1560438443; bh=BzuvN0Iz8NNaIZmg4jmkwxTjOXFNtkNfnok0NoVK6kw=; h=X-UI-Sender-Class:From:To:Cc:Subject:Date:In-Reply-To:References; b=K2SkpgHCwJXIU01fgzlC4KIPIbDtHMDhxSaSkuPl7EU81LCACNlYG+HBB96nSPlvX CAPuYDSQkAVU0P/cOfJx4q9DuQQ4WpGg4tNNE2U7+edCCEYout3B6O3fj6sOiW3nPl nYOvCv57XAL0QNu+ZADoSGYsIkIMDVV97YNpAIQk= X-UI-Sender-Class: 01bb95c1-4bf8-414a-932a-4f6e2808ef9c Received: from labiere.lan ([92.6.238.228]) by mail.gmx.com (mrgmx004 [212.227.17.184]) with ESMTPSA (Nemesis) id 1MIwzA-1huctc4Aoc-00KT3r; Thu, 13 Jun 2019 17:07:23 +0200 From: Pierre Langlois Date: Thu, 13 Jun 2019 16:07:21 +0100 Message-Id: <20190613150721.26314-1-pierre.langlois@gmx.com> X-Mailer: git-send-email 2.22.0 In-Reply-To: <20190613150126.17280-1-pierre.langlois@gmx.com> References: <20190613150126.17280-1-pierre.langlois@gmx.com> MIME-Version: 1.0 X-Provags-ID: V03:K1:BuP4zE2uLwkGef/lxw88uBmybdQMdeUz2rRJ8MMGgRuNOUYqQJX Fo4YLMI8+R90T/4lcUMytw47WvDO3Ve22PoEYw/Z1uIrk1YDwSpTnXkk9owcs6BrGIOaBFU iulSIPt4r6El6c3SomrRmq4/hktE38G/vQq/KcCyeIh/TzzVKQEFAr0ovbuaLLj7yWNvJN9 3SZ/AG1Nefxk9bJUtlsfw== X-UI-Out-Filterresults: notjunk:1;V03:K0:TWEd9ZyywY4=:KNlKdsFAtBA7NafsP9Y9/4 Gb8rpDDnc7TkIp4C5GhvSyQ9SiSy8LVEHAAX92rA+5PUfluFNyR8vzQHraWJ5/7v3jxeF3RtP 5hk68YoOlZ2ei/jwr5TibMvwxVnKvtDShZyw1wg8Fh1nYJhpKpVgQdOaDQUqmmflXZxOqP66a 1wQm4SowqbEhJDZX7zYQqRRg6zvMxXjkVHQQPeZtIT/O0ZvmvZR7AwRTx610qBEsKmS9QoY5Y bZqUKPMEbKaj/HOuv8mP+DGGv2LJ/4Ttx11gtqr9J7e4fZ5FEa1L9iWmjNmEk3XZG0Pq2KF5U gH60jjFiKXbAza1lPQKKGqkNlZZnDUwDBtn/QNa5GH9oU3PGfgUG6f9wmkvnRpj6///ygaR35 vBgZkXCalk+xbg/6tdorYBRQ79nW0zHmxuzzeOKS7u8o6IvInfrKdsRYQoy0DexTyrzVFCFet bW/E0anT1Xfi00IJfhHjkyDlwu2/PzbAb8eSRdYRCRHcAqPctYqnCugeOROusHm7HMgnIjsK4 8kfoQD8+/RCvWBMNMPKkq/K58pGF5FffuM6uaU3IpnYLesefLJ+bYChNySOaOI+PiTHNaV9eT opr73GFEVakxmN2pWq6BoJOtRAK/cO2kOPyTwQAYCa2k0e/puUnQnzjIuE/mL750fOBbKItdW XFspeeDZGlT81gRny15MBTty9iLgE0A/GD6CyR5JdseUebnpeis6ch/fPOQ04vFNKjX8uScF7 ch9dS352c0lDJLWcxJg20/jLfIUZ4X44VDe3EDQvp8pmU6o1Pgyu0kuDJIcdCiG+vH7EJSVFq /MXpU1+/qjAVZcMSEDcw7g+d5+rOVTaadkG6m1+3zEVQUGzasrPPamEMmUusRpGBBGlUqe6ZD vdsyFGotz5FMnbOZexLEvvGkgHZAvtdP5UzFbO9c7d7K2o7uICsB2JcPflO6+CBqzSHw9aYEo Nk0HAqtaslxOQU9ImMciL1R3jrE3s6uWmq4FlUEYiaKb7hix/+nKo X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 209.51.188.43 X-BeenThere: guix-patches@gnu.org List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: "Guix-patches" X-getmail-retrieved-from-mailbox: Patches * gnu/packages/wireservice.scm (csvkit): New variable. * gnu/packages/patches/csvkit-fix-tests.patch: New file. * gnu/local.mk (dist_patch_DATA): Add it. --- gnu/local.mk | 1 + gnu/packages/patches/csvkit-fix-tests.patch | 45 +++++++++++++ gnu/packages/wireservice.scm | 73 +++++++++++++++++++++ 3 files changed, 119 insertions(+) create mode 100644 gnu/packages/patches/csvkit-fix-tests.patch -- 2.22.0 diff --git a/gnu/local.mk b/gnu/local.mk index e9e5009276..b49592b5d5 100644 --- a/gnu/local.mk +++ b/gnu/local.mk @@ -734,6 +734,7 @@ dist_patch_DATA = \ %D%/packages/patches/cpufrequtils-fix-aclocal.patch \ %D%/packages/patches/crawl-upgrade-saves.patch \ %D%/packages/patches/crda-optional-gcrypt.patch \ + %D%/packages/patches/csvkit-fix-tests.patch \ %D%/packages/patches/clucene-contribs-lib.patch \ %D%/packages/patches/cube-nocheck.patch \ %D%/packages/patches/cursynth-wave-rand.patch \ diff --git a/gnu/packages/patches/csvkit-fix-tests.patch b/gnu/packages/patches/csvkit-fix-tests.patch new file mode 100644 index 0000000000..e62b601fe4 --- /dev/null +++ b/gnu/packages/patches/csvkit-fix-tests.patch @@ -0,0 +1,45 @@ +diff --git a/tests/test_utilities/test_csvsql.py b/tests/test_utilities/test_csvsql.py +index e6ec4af..4f47980 100644 +--- a/tests/test_utilities/test_csvsql.py ++++ b/tests/test_utilities/test_csvsql.py +@@ -197,7 +197,7 @@ class TestCSVSQL(CSVKitTestCase, EmptyFileTests): + utility.run() + output = output_file.getvalue() + output_file.close() +- self.assertEqual(output, 'a,b,c\n1,2,3\n0,5,6\n') ++ self.assertEqual(output, 'a,b,c\n1,2.0,3.0\n0,5.0,6.0\n') + + def test_no_prefix_unique_constraint(self): + self.get_output(['--db', 'sqlite:///' + self.db_file, '--insert', 'examples/dummy.csv', '--unique-constraint', 'a']) +diff --git a/tests/test_utilities/test_sql2csv.py b/tests/test_utilities/test_sql2csv.py +index a0c3d3e..babcfd6 100644 +--- a/tests/test_utilities/test_sql2csv.py ++++ b/tests/test_utilities/test_sql2csv.py +@@ -121,23 +121,23 @@ class TestSQL2CSV(CSVKitTestCase, EmptyFileTests): + input_file.close() + + def test_unicode(self): +- expected = self.csvsql('examples/test_utf8.csv') ++ self.csvsql('examples/test_utf8.csv') + csv = self.get_output(['--db', 'sqlite:///' + self.db_file, '--query', 'select * from foo']) +- self.assertEqual(csv.strip(), expected) ++ self.assertEqual(csv.strip(), 'foo,bar,baz\n1.0,2.0,3\n4.0,5.0,ʤ') + + def test_no_header_row(self): + self.csvsql('examples/dummy.csv') + csv = self.get_output(['--db', 'sqlite:///' + self.db_file, '--no-header-row', '--query', 'select * from foo']) + + self.assertTrue('a,b,c' not in csv) +- self.assertTrue('1,2,3' in csv) ++ self.assertTrue('1,2.0,3.0' in csv) + + def test_linenumbers(self): + self.csvsql('examples/dummy.csv') + csv = self.get_output(['--db', 'sqlite:///' + self.db_file, '--linenumbers', '--query', 'select * from foo']) + + self.assertTrue('line_number,a,b,c' in csv) +- self.assertTrue('1,1,2,3' in csv) ++ self.assertTrue('1,1,2.0,3.0' in csv) + + def test_wildcard_on_sqlite(self): + self.csvsql('examples/iris.csv') diff --git a/gnu/packages/wireservice.scm b/gnu/packages/wireservice.scm index 78c2a4264d..ab86f21f2f 100644 --- a/gnu/packages/wireservice.scm +++ b/gnu/packages/wireservice.scm @@ -19,6 +19,7 @@ (define-module (gnu packages wireservice) #:use-module ((guix licenses) #:prefix license:) #:use-module (guix build-system python) + #:use-module (guix download) #:use-module (guix git-download) #:use-module (guix packages) #:use-module (gnu packages) @@ -191,3 +192,75 @@ for dbf files support to all @code{agate.Table} instances."))) (synopsis "Add read support for Excel files (xls and xlsx) to agate") (description "@code{agateexcel} uses a monkey patching pattern to add read for xls and xlsx files support to all @code{agate.Table} instances."))) + +(define-public csvkit + (package + (name "csvkit") + (version "1.0.4") + (source (origin + (method url-fetch) + (uri (pypi-uri "csvkit" version)) + (sha256 + (base32 + "1830lb95rh1iyi3drlwxzb6y3pqkii0qiyzd40c1kvhvaf1s6lqk")) + (patches (search-patches "csvkit-fix-tests.patch")))) + (build-system python-build-system) + (native-inputs + `(("python-psycopg2" ,python-psycopg2) ;; Used to test PostgreSQL support. + ("python-sphinx" ,python-sphinx) + ("python-sphinx-rtd-theme" ,python-sphinx-rtd-theme))) + (inputs + `(("python-agate-dbf" ,python-agate-dbf) + ("python-agate-excel" ,python-agate-excel) + ("python-agate-sql" ,python-agate-sql) + ("python-six" ,python-six))) + (arguments + `(#:phases + (modify-phases %standard-phases + (add-after 'install 'install-docs + (lambda* (#:key outputs #:allow-other-keys) + (let* ((out (assoc-ref outputs "out")) + (man1 (string-append out "/share/man/man1"))) + (with-directory-excursion "docs" + (invoke "make" "man") + (copy-recursively "_build/man" man1)) + #t)))))) + (home-page "https://csvkit.rtfd.org") + (synopsis "Command-line tools for working with CSV") + (description "csvkit is a suite of command-line tools for converting to +and working with CSV. It provides the following commands: +@itemize +@item Input: + @itemize + @item @command{in2csv}: Convert various formats to CSV. + @item @command{sql2csv}: Execute SQL commands on a database and return the +data as CSV. + @end itemize +@item Processing: + @itemize + @item @command{csvclean}: Remove common syntax errors. + @item @command{csvcut}: Filter and truncate CSV files. + @item @command{csvgrep}: Filter tabular data to only those rows where +certain columns contain a given value or match a regular expression. + @item @command{csvjoin}: Merges two or more CSV tables together using a +method analogous to SQL JOIN operation. + @item @command{csvsort}: Sort CSV files. + @item @command{csvstack}: Stack up the rows from multiple CSV files, +optionally adding a grouping value to each row. + @end itemize +@item Output and analysis: + @itemize + @item @command{csvformat}: Convert a CSV file to a custom output format. + @item @command{csvjson}: Converts a CSV file into JSON or GeoJSON. + @item @command{csvlook}: Renders a CSV to the command line in a +Markdown-compatible, fixed-width format. + @item @command{csvpy}: Loads a CSV file into a @code{agate.csv.Reader} +object and then drops into a Python shell so the user can inspect the data +however they see fit. + @item @command{csvsql}: Generate SQL statements for a CSV file or execute +those statements directly on a database. + @item @command{csvstat}: Prints descriptive statistics for all columns in a +CSV file. + @end itemize +@end itemize") + (license license:expat)))