diff options
author | Yun-Chih Chen <yunchih@csie.ntu.edu.tw> | 2018-03-18 20:48:41 +0800 |
---|---|---|
committer | Yun-Chih Chen <yunchih@csie.ntu.edu.tw> | 2018-03-18 20:48:41 +0800 |
commit | 4387a94aa952281636134fb99c367813cb9dcc05 (patch) | |
tree | 77be8725f80ad3e4cade1c325f2a39e679f93323 | |
parent | 2db7b99431ab899686f52a8f20f7c74333cf1fa4 (diff) | |
download | nfcollect-4387a94aa952281636134fb99c367813cb9dcc05.tar nfcollect-4387a94aa952281636134fb99c367813cb9dcc05.tar.gz nfcollect-4387a94aa952281636134fb99c367813cb9dcc05.tar.bz2 nfcollect-4387a94aa952281636134fb99c367813cb9dcc05.tar.lz nfcollect-4387a94aa952281636134fb99c367813cb9dcc05.tar.xz nfcollect-4387a94aa952281636134fb99c367813cb9dcc05.tar.zst nfcollect-4387a94aa952281636134fb99c367813cb9dcc05.zip |
Add helpful documentation to README.md
-rw-r--r-- | README.md | 58 |
1 files changed, 55 insertions, 3 deletions
@@ -2,6 +2,19 @@ Collect Netfilter NFLOG log entries and commit them to stable storage in binary (compressed) format. +The project contains two binaries: `nfcollect` and `nfextract`: + +#### `nfcollect` + +Collect packets from *Netfilter* netlink kernel interface. Packets are +aggregated onto a memory region (we call it *a trunk*), until the *trunk* is full. +A full *trunk* will be committed to disk by configurable means (currently `zstd` +compression and no compression is implemented). Trunks will be stored in a +specific directory, which will be scanned by `nfextract` to extract all trunks. + +Due to communication with the kernel, **this program requires root privilege**. + + ## Build ```bash @@ -12,10 +25,49 @@ make Run `./configure --enable-debug` to enable debug output. +## Usage + +``` bash +$ ./nfcollect --help +Usage: nfcollect [OPTION] + +Options: + -c --compression=<algo> compression algorithm to use (default: no compression) + -d --storage_dir=<dirname> log files storage directory + -h --help print this help + -g --nflog-group=<id> the group id to collect + -p --parallelism=<num> max number of committer thread + -s --storage_size=<dirsize> log files maximum total size in MiB + -v --version print version information + +$ ./nfextract -h +Usage: nfextract [OPTION] + +Options: + -d --storage_dir=<dirname> log files storage directory + -h --help print this help + -v --version print version information +``` + +#### Examples + +```bash +# Send all packets destined for localhost to the nflog group #5 +sudo iptables -A OUTPUT -p tcp -d 127.0.0.1 -j NFLOG --nflog-group 5 + +# Receive the packets from nfnetlink +mkdir my-nflog +sudo ./nfcollect -d my-nflog -g 5 -s 100 -c zstd + +# Let it collect for a while ... + +# Dump the collected packets +./nfextract -d my-nflog +``` + + ### References * libnetfilter_log: https://www.icir.org/gregor/tools/files/doc.libnetfilter_log/html/libnetfilter__log.html * zstd: https://facebook.github.io/zstd/zstd_manual.html -* mmap: https://stackoverflow.com/questions/35891525/mmap-for-writing-sequential-log-file-for-speed/35891885 -* pthread: http://www.cs.cmu.edu/afs/cs/academic/class/15492-f07/www/pthreads.html -* apr_queue: http://apr.apache.org/docs/apr-util/1.3/apr__queue_8h.html +* lz4: https://github.com/lz4/lz4 |