| Message ID | 51481207-31b7-58bc-00de-f0633a3a1059@gmail.com |
|---|---|
| State | Accepted |
| Headers | show |
| Series | None | expand |
fesoj000 schreef op ma 21-03-2022 om 22:20 [+0100]: > +@item > +SQL database back-end support: SQLite3, MySQL and PostgreSQL Does this work out-of-the-box, or do extra inputs need to be added for this?
fesoj000 schreef op ma 21-03-2022 om 22:20 [+0100]:
> + (supported-systems (filter target-linux? %supported-systems))
Conventionally, supported-systems appears at the bottom of the package
definition.
fesoj000 schreef op ma 21-03-2022 om 22:20 [+0100]: > + (lambda* (#:key outputs #:allow-other-keys) > + (let ((out-etc (string-append (assoc-ref outputs "out") "/etc")) FWIW, you can simplify this to (lambda _ (let ((out-etc (string-append #$output "/etc"))) [...])), eliminating the output label in favour of G-exps -- see e.g. 'hostapd'. IMO this is a bit tidier, but not everyone seems to agree. Greetings, Maxime.
On 3/23/22 8:41 PM, Maxime Devos wrote: > fesoj000 schreef op ma 21-03-2022 om 22:20 [+0100]: >> +@item >> +SQL database back-end support: SQLite3, MySQL and PostgreSQL > Does this work out-of-the-box, or do extra inputs need to be added for > this? Good point, this is the upstream description of the daemon. I personally only use syslog output currently. And yes, to add support for those database back-ends, we need to add them as inputs. Although i would prefer to not "blow up" the package. Postgresql and especially mysql a rather "big boys". On a router this might be a little much. This could be fixed, by providing a separate "lib" output for those. For me personally, i plan to continue using syslog output. Further i plan to use pcap and josn output for certain things, but so far i did not have time to play with that. So, what i would do is, add sqlite, libpcap and jansson as inputs. This enables sqlite, pcap and json output support in ulogd. If postgresql and mysql is something somebody needs, i would propose to introduce a new package variant which adds those inputs. Is this something we could do?
On 3/23/22 10:28 PM, fesoj000 wrote: > On 3/23/22 8:41 PM, Maxime Devos wrote: >> fesoj000 schreef op ma 21-03-2022 om 22:20 [+0100]: >>> +@item >>> +SQL database back-end support: SQLite3, MySQL and PostgreSQL >> Does this work out-of-the-box, or do extra inputs need to be added for >> this? > Good point, this is the upstream description of the daemon. I personally > only use syslog output currently. And yes, to add support for those > database back-ends, we need to add them as inputs. Although i would prefer > to not "blow up" the package. Postgresql and especially mysql a rather "big > boys". On a router this might be a little much. This could be fixed, by > providing a separate "lib" output for those. > > For me personally, i plan to continue using syslog output. Further i plan > to use pcap and josn output for certain things, but so far i did not have > time to play with that. > > So, what i would do is, add sqlite, libpcap and jansson as inputs. This > enables sqlite, pcap and json output support in ulogd. > > If postgresql and mysql is something somebody needs, i would propose to > introduce a new package variant which adds those inputs. > > Is this something we could do? Below one can find three definitions, the first is ulogd with sqlite, libpcap and jansson inputs. Following that is ulogd+postgresql and ulogd+mysql with the additional inputs they need. After the package definitions the output of guix size follows for every definition. ulogd has a size of 85.0 MiB, ulogd+postgresql has a size of 140.4 MiB, ulogd+mysql has a size of 702.9 MiB. According to this data, i would consider postgresql as input for ulogd. But mysql increases the size way too much for me. I need to move images around the network or upload them to some cloud. Maybe mariadb could be used as a mysql replacement, are they still compatible? But mariadb is also large 370 MiB ... I will wait a day or two, if i do not hear any other opinion on that i will send a new patchset where ulogd gets postgresql as input, but mysql not. (define-public ulogd (package (name "ulogd") (version "2.0.7") (source (origin (method url-fetch) (uri (string-append "https://netfilter.org/projects/" name "/files/" name "-" version ".tar.bz2")) (sha256 (base32 "0ax9959c4bapq78n13bbaibcf1gwjir3ngx8l2dh45lw9m4ha2lr")))) (build-system gnu-build-system) (supported-systems (filter target-linux? %supported-systems)) (native-inputs (list pkg-config)) (inputs (list libnfnetlink libmnl libnetfilter-log libnetfilter-conntrack libnetfilter-acct sqlite libpcap jansson)) (arguments (list #:phases #~(modify-phases %standard-phases (add-after 'install 'install-doc (lambda _ (let ((out-etc (string-append #$output "/etc")) (ulogd.conf "ulogd.conf")) (mkdir-p out-etc) (copy-file ulogd.conf (string-append out-etc "/" ulogd.conf)))))))) (synopsis "Userspace logging daemon for netfilter/iptables.") (description "ulogd is a userspace logging daemon for netfilter/iptables related logging. This includes per-packet logging of security violations, per-packet logging for accounting, per-flow logging and flexible user-defined accounting. @enumerate @item Packet and flow-based traffic accounting @item Flexible user-defined traffic accounting via nfacct infrastructure @item SQL database back-end support: SQLite3 @item Text-based output formats: CSV, XML, Netfilter's LOG, Netfilter's conntrack @end enumerate ") (home-page "https://netfilter.org/projects/nfacct/index.html") (license license:gpl2))) (define-public ulogd+postgresql (package (inherit ulogd) (name (string-append (package-name ulogd) "+postgresql")) (inputs (modify-inputs (package-inputs ulogd) (append postgresql))) (arguments (substitute-keyword-arguments (package-arguments ulogd) ((#:configure-flags configure-flags ''()) `(append ,configure-flags (list (string-append "--with-pgsql=" (assoc-ref %build-inputs "postgresql"))))))))) (define-public ulogd+postgresql (package (inherit ulogd) (name (string-append (package-name ulogd) "+postgresql")) (inputs (modify-inputs (package-inputs ulogd) (append postgresql))) (arguments (substitute-keyword-arguments (package-arguments ulogd) ((#:configure-flags configure-flags ''()) `(append ,configure-flags (list (string-append "--with-pgsql=" (assoc-ref %build-inputs "postgresql"))))))))) (define-public ulogd+mysql (package (inherit ulogd) (name (string-append (package-name ulogd) "+mysql")) (inputs (modify-inputs (package-inputs ulogd) (append mysql zlib openssl))) (arguments (substitute-keyword-arguments (package-arguments ulogd) ((#:configure-flags configure-flags ''()) `(append ,configure-flags (list (string-append "--with-mysql=" (assoc-ref %build-inputs "mysql"))))))))) $ ./pre-inst-env guix size ulogd store item total self /gnu/store/5h2w4qi9hk1qzzgi1w83220ydslinr4s-glibc-2.33 38.3 36.6 43.1% /gnu/store/094bbaq6glba86h1d4cj16xhdi6fk2jl-gcc-10.3.0-lib 71.7 33.4 39.3% /gnu/store/9rrnm5hdjw7cy96a2a9rfgh6y08wsbmf-ncurses-6.2.20210619 77.6 5.9 7.0% /gnu/store/xmzx5mzv4863yw9kmr2ykndgp37p8if0-sqlite-3.36.0 82.3 3.2 3.8% /gnu/store/720rj90bch716isd8z7lcwrnvz28ap4y-bash-static-5.1.8 1.7 1.7 2.0% /gnu/store/wcwls45278gzpjvwlvrrs1y7h30g44xh-readline-8.1.1 79.0 1.4 1.7% /gnu/store/4y5m9lb8k3qkb1y9m02sw9w9a6hacd16-bash-minimal-5.1.8 39.3 1.0 1.2% /gnu/store/c8mld9g531an1r002ksbidp224l9xgff-libpcap-1.10.1 73.3 0.7 0.8% /gnu/store/sww4g1nq9bi3hn8xqdf9x507kn3vql9v-ulogd-2.0.7 85.0 0.5 0.6% /gnu/store/l7i9mq16vy8cp05zl0a3r5awyfsps27b-libnetfilter-conntrack-1.0.8 72.0 0.2 0.2% /gnu/store/nprljhh7a86351vg6h23va3kfdnkwnd4-jansson-2.13.1 71.7 0.1 0.1% /gnu/store/mfnzmv8i64s53m0g0cn2fx2sav48ssfc-libnetfilter-log-1.0.2 71.9 0.1 0.1% /gnu/store/cpsfihchx5spv7c6y5fch0zlkvkwvlnq-libnfnetlink-1.0.1 71.7 0.1 0.1% /gnu/store/dj7kw3mqasw0rxdbm1gkajgsznhw8b4h-libmnl-1.0.4 71.7 0.1 0.1% /gnu/store/iny0cn6qbj0xxczqk4hfmjacyfal44w8-libnetfilter-acct-1.0.3 71.8 0.0 0.1% total: 85.0 MiB $ ./pre-inst-env guix size ulogd+postgresql store item total self /gnu/store/5h2w4qi9hk1qzzgi1w83220ydslinr4s-glibc-2.33 38.3 36.6 26.1% /gnu/store/094bbaq6glba86h1d4cj16xhdi6fk2jl-gcc-10.3.0-lib 71.7 33.4 23.8% /gnu/store/q6qp3521gay7izpz8p68d21zsdmz6nnm-postgresql-13.4 135.5 24.3 17.3% /gnu/store/d251rfgc9nm2clzffzhgiipdvfvzkvwi-coreutils-8.32 88.0 16.4 11.6% /gnu/store/5583c2za2jsn9g6az79rnksgvigwnsk7-util-linux-2.37.2-lib 80.7 9.0 6.4% /gnu/store/9rrnm5hdjw7cy96a2a9rfgh6y08wsbmf-ncurses-6.2.20210619 77.6 5.9 4.2% /gnu/store/plr00nij45964cyy7sfcg5rcsi8hks0h-openssl-1.1.1l 77.2 5.5 3.9% /gnu/store/xmzx5mzv4863yw9kmr2ykndgp37p8if0-sqlite-3.36.0 82.3 3.2 2.3% /gnu/store/720rj90bch716isd8z7lcwrnvz28ap4y-bash-static-5.1.8 1.7 1.7 1.2% /gnu/store/wcwls45278gzpjvwlvrrs1y7h30g44xh-readline-8.1.1 79.0 1.4 1.0% /gnu/store/4y5m9lb8k3qkb1y9m02sw9w9a6hacd16-bash-minimal-5.1.8 39.3 1.0 0.7% /gnu/store/c8mld9g531an1r002ksbidp224l9xgff-libpcap-1.10.1 73.3 0.7 0.5% /gnu/store/snh4hdpg7k75s4gjcd2s77lkxrxx2m0m-ulogd+postgresql-2.0.7 140.4 0.6 0.4% /gnu/store/8qv5kb2fgm4c3bf70zcg9l6hkf3qzpw9-zlib-1.2.11 71.9 0.2 0.2% /gnu/store/l7i9mq16vy8cp05zl0a3r5awyfsps27b-libnetfilter-conntrack-1.0.8 72.0 0.2 0.1% /gnu/store/nprljhh7a86351vg6h23va3kfdnkwnd4-jansson-2.13.1 71.7 0.1 0.1% /gnu/store/mfnzmv8i64s53m0g0cn2fx2sav48ssfc-libnetfilter-log-1.0.2 71.9 0.1 0.0% /gnu/store/cpsfihchx5spv7c6y5fch0zlkvkwvlnq-libnfnetlink-1.0.1 71.7 0.1 0.0% /gnu/store/dj7kw3mqasw0rxdbm1gkajgsznhw8b4h-libmnl-1.0.4 71.7 0.1 0.0% /gnu/store/iny0cn6qbj0xxczqk4hfmjacyfal44w8-libnetfilter-acct-1.0.3 71.8 0.0 0.0% total: 140.4 MiB $ ./pre-inst-env guix size ulogd+mysql store item total self /gnu/store/a1qdzqnqqxshdzv9andf4v9kr8dspyil-mysql-5.7.33 697.9 204.3 29.1% /gnu/store/069aq2v993kpc41yabp5b6vm4wb9jkhg-gcc-10.3.0 217.7 145.8 20.7% /gnu/store/vdlmzq6h0x5lxhr0nkr315dh2fbhm8d8-boost-1.59.0 219.7 108.5 15.4% /gnu/store/hy6abswwv4d89zp464fw52z65fkzr7h5-perl-5.34.0 147.7 58.6 8.3% /gnu/store/hzic3ddl5yvnyw7gm4a0qc5icgqy2442-icu4c-69.1 110.7 38.0 5.4% /gnu/store/5h2w4qi9hk1qzzgi1w83220ydslinr4s-glibc-2.33 38.3 36.6 5.2% /gnu/store/094bbaq6glba86h1d4cj16xhdi6fk2jl-gcc-10.3.0-lib 71.7 33.4 4.7% /gnu/store/8fpk2cja3f07xls48jfnpgrzrljpqivr-coreutils-8.32 91.6 16.4 2.3% /gnu/store/vqdsrvs9jbn0ix2a58s99jwkh74124y5-coreutils-minimal-8.32 88.0 16.4 2.3% /gnu/store/d99ykvj3axzzidygsmdmzxah4lvxd6hw-bash-5.1.8 85.3 6.2 0.9% /gnu/store/9rrnm5hdjw7cy96a2a9rfgh6y08wsbmf-ncurses-6.2.20210619 77.6 5.9 0.8% /gnu/store/plr00nij45964cyy7sfcg5rcsi8hks0h-openssl-1.1.1l 77.2 5.5 0.8% /gnu/store/dalhky8hh7ib25m63j0c3sh6iqqf2p36-mit-krb5-1.19.2 82.2 3.9 0.6% /gnu/store/55q02v1a3qz8n7rlhy3jva9qjkfwj8y0-gawk-5.1.0 88.6 3.3 0.5% /gnu/store/xmzx5mzv4863yw9kmr2ykndgp37p8if0-sqlite-3.36.0 82.3 3.2 0.5% /gnu/store/fwbiihd2sbhai63y1pvvdh0f2bakfzrf-gmp-6.2.1 74.4 2.7 0.4% /gnu/store/720rj90bch716isd8z7lcwrnvz28ap4y-bash-static-5.1.8 1.7 1.7 0.2% /gnu/store/di5bqb45hi5lvp2q08hlxqjdcl9phjb1-pcre-8.45 73.4 1.7 0.2% /gnu/store/m2wmfwk2m4390dwbnjm6ps5y4c9pchi5-procps-3.3.16 79.1 1.5 0.2% /gnu/store/wcwls45278gzpjvwlvrrs1y7h30g44xh-readline-8.1.1 79.0 1.4 0.2% /gnu/store/2b3blhwbag1ial0dhxw7wh4zjxl0cqpk-pkg-config-0.29.2 72.8 1.1 0.2% /gnu/store/4y5m9lb8k3qkb1y9m02sw9w9a6hacd16-bash-minimal-5.1.8 39.3 1.0 0.1% /gnu/store/hrgqa7m498wfavq4awai3xz86ifkjxdr-grep-3.6 75.2 0.8 0.1% /gnu/store/zhd6blbfz40xp62i4d1rcgbyrpkynbkc-sed-4.8 72.5 0.8 0.1% /gnu/store/c8mld9g531an1r002ksbidp224l9xgff-libpcap-1.10.1 73.3 0.7 0.1% /gnu/store/dxvpcggxj40bbb2pp3ddicapy4gzzzqk-ulogd+mysql-2.0.7 702.9 0.6 0.1% /gnu/store/nvqxvcx05jgixpnshxp9nypacwc2mri2-libtirpc-1.3.1 82.7 0.5 0.1% /gnu/store/s3hl12jxz9ybs7nsy7kq7ybzz7qnzmsg-bzip2-1.0.8 73.1 0.4 0.1% /gnu/store/a38k2v29l6l0iz6pmlk4dmzwdbvl10lq-acl-2.3.1 72.3 0.3 0.0% /gnu/store/a7ggx0af69gv4k5mr1k617p4vy9kgx2v-libcap-2.62 72.0 0.3 0.0% /gnu/store/jkjs0inmzhj4vsvclbf08nmh0shm7lrf-attr-2.5.1 71.9 0.2 0.0% /gnu/store/8qv5kb2fgm4c3bf70zcg9l6hkf3qzpw9-zlib-1.2.11 71.9 0.2 0.0% /gnu/store/mrd2bamw39851jpr4m2q8gimg8s48gzh-zlib-1.2.11 38.5 0.2 0.0% /gnu/store/l7i9mq16vy8cp05zl0a3r5awyfsps27b-libnetfilter-conntrack-1.0.8 72.0 0.2 0.0% /gnu/store/nprljhh7a86351vg6h23va3kfdnkwnd4-jansson-2.13.1 71.7 0.1 0.0% /gnu/store/mfnzmv8i64s53m0g0cn2fx2sav48ssfc-libnetfilter-log-1.0.2 71.9 0.1 0.0% /gnu/store/m4dgk1q4zvzw6lnccr7fd941w0sisb5b-libaio-0.3.112 71.7 0.1 0.0% /gnu/store/cpsfihchx5spv7c6y5fch0zlkvkwvlnq-libnfnetlink-1.0.1 71.7 0.1 0.0% /gnu/store/dj7kw3mqasw0rxdbm1gkajgsznhw8b4h-libmnl-1.0.4 71.7 0.1 0.0% /gnu/store/4r6f3a6n82nv48c7nznhhcl19k7pl0ig-libsigsegv-2.13 71.7 0.1 0.0% /gnu/store/iny0cn6qbj0xxczqk4hfmjacyfal44w8-libnetfilter-acct-1.0.3 71.8 0.0 0.0% total: 702.9 MiB BR
fesoj000 schreef op za 26-03-2022 om 13:31 [+0100]: > After the package definitions the output of guix size follows for every > definition. ulogd has a size of 85.0 MiB, ulogd+postgresql has a size of 140.4 > MiB, ulogd+mysql has a size of 702.9 MiB. > > According to this data, i would consider postgresql as input for ulogd. But > mysql increases the size way too much for me. I need to move images around the > network or upload them to some cloud. Maybe mariadb could be used as a mysql > replacement, are they still compatible? But mariadb is also large 370 MiB ... It looks like ulogd2 has a kind of plugin architecture (see https://git.netfilter.org/ulogd2/tree/output/Makefile.am), so perhaps the plugins can be put in separate outputs (or separate packages, whatever's the most convenient)? That should eliminate the closure size concerns. It might be be necessary to introduce some kind of ULOGD_PLUGIN_PATH though such that it will actually find the libraries it tries to dlopen, without having to explicitely pass the full /gnu/store/.../lib/....so. Greetings, Maxime.
diff --git a/gnu/packages/linux.scm b/gnu/packages/linux.scm index 3124e57ef0..94d7cd92dd 100644 --- a/gnu/packages/linux.scm +++ b/gnu/packages/linux.scm @@ -7488,6 +7488,51 @@ (define-public libnetfilter-log (home-page "https://netfilter.org/projects/libnetfilter_log/index.html") (license license:gpl2+))) +(define-public ulogd + (package + (name "ulogd") + (version "2.0.7") + (source + (origin + (method url-fetch) + (uri (string-append "https://netfilter.org/projects/" name "/files/" name + "-" version ".tar.bz2")) + (sha256 + (base32 + "0ax9959c4bapq78n13bbaibcf1gwjir3ngx8l2dh45lw9m4ha2lr")))) + (build-system gnu-build-system) + (supported-systems (filter target-linux? %supported-systems)) + (native-inputs (list pkg-config)) + (inputs (list pkg-config libnfnetlink libmnl libnetfilter-log + libnetfilter-conntrack libnetfilter-acct)) + (arguments + `(#:phases + (modify-phases %standard-phases + (add-after 'install 'install-doc + (lambda* (#:key outputs #:allow-other-keys) + (let ((out-etc (string-append (assoc-ref outputs "out") "/etc")) + (ulogd.conf "ulogd.conf")) + (mkdir-p out-etc) + (copy-file ulogd.conf (string-append out-etc "/" ulogd.conf)))))))) + (synopsis "Userspace logging daemon for netfilter/iptables.") + (description "ulogd is a userspace logging daemon for netfilter/iptables +related logging. This includes per-packet logging of security violations, +per-packet logging for accounting, per-flow logging and flexible user-defined +accounting. +@enumerate +@item +Packet and flow-based traffic accounting +@item +Flexible user-defined traffic accounting via nfacct infrastructure +@item +SQL database back-end support: SQLite3, MySQL and PostgreSQL +@item +Text-based output formats: CSV, XML, Netfilter's LOG, Netfilter's conntrack +@end enumerate +") + (home-page "https://netfilter.org/projects/nfacct/index.html") + (license license:gpl2))) + (define-public proot (package (name "proot")