aboutsummaryrefslogtreecommitdiffstats
diff options
context:
space:
mode:
authorYun-Chih Chen <yunchih@csie.ntu.edu.tw>2018-03-18 20:48:41 +0800
committerYun-Chih Chen <yunchih@csie.ntu.edu.tw>2018-03-18 20:48:41 +0800
commit4387a94aa952281636134fb99c367813cb9dcc05 (patch)
tree77be8725f80ad3e4cade1c325f2a39e679f93323
parent2db7b99431ab899686f52a8f20f7c74333cf1fa4 (diff)
downloadnfcollect-4387a94aa952281636134fb99c367813cb9dcc05.tar
nfcollect-4387a94aa952281636134fb99c367813cb9dcc05.tar.gz
nfcollect-4387a94aa952281636134fb99c367813cb9dcc05.tar.bz2
nfcollect-4387a94aa952281636134fb99c367813cb9dcc05.tar.lz
nfcollect-4387a94aa952281636134fb99c367813cb9dcc05.tar.xz
nfcollect-4387a94aa952281636134fb99c367813cb9dcc05.tar.zst
nfcollect-4387a94aa952281636134fb99c367813cb9dcc05.zip
Add helpful documentation to README.md
-rw-r--r--README.md58
1 files changed, 55 insertions, 3 deletions
diff --git a/README.md b/README.md
index 63ad1f7..b08cfbb 100644
--- a/README.md
+++ b/README.md
@@ -2,6 +2,19 @@
Collect Netfilter NFLOG log entries and commit them to stable storage in binary (compressed) format.
+The project contains two binaries: `nfcollect` and `nfextract`:
+
+#### `nfcollect`
+
+Collect packets from *Netfilter* netlink kernel interface. Packets are
+aggregated onto a memory region (we call it *a trunk*), until the *trunk* is full.
+A full *trunk* will be committed to disk by configurable means (currently `zstd`
+compression and no compression is implemented). Trunks will be stored in a
+specific directory, which will be scanned by `nfextract` to extract all trunks.
+
+Due to communication with the kernel, **this program requires root privilege**.
+
+
## Build
```bash
@@ -12,10 +25,49 @@ make
Run `./configure --enable-debug` to enable debug output.
+## Usage
+
+``` bash
+$ ./nfcollect --help
+Usage: nfcollect [OPTION]
+
+Options:
+ -c --compression=<algo> compression algorithm to use (default: no compression)
+ -d --storage_dir=<dirname> log files storage directory
+ -h --help print this help
+ -g --nflog-group=<id> the group id to collect
+ -p --parallelism=<num> max number of committer thread
+ -s --storage_size=<dirsize> log files maximum total size in MiB
+ -v --version print version information
+
+$ ./nfextract -h
+Usage: nfextract [OPTION]
+
+Options:
+ -d --storage_dir=<dirname> log files storage directory
+ -h --help print this help
+ -v --version print version information
+```
+
+#### Examples
+
+```bash
+# Send all packets destined for localhost to the nflog group #5
+sudo iptables -A OUTPUT -p tcp -d 127.0.0.1 -j NFLOG --nflog-group 5
+
+# Receive the packets from nfnetlink
+mkdir my-nflog
+sudo ./nfcollect -d my-nflog -g 5 -s 100 -c zstd
+
+# Let it collect for a while ...
+
+# Dump the collected packets
+./nfextract -d my-nflog
+```
+
+
### References
* libnetfilter_log: https://www.icir.org/gregor/tools/files/doc.libnetfilter_log/html/libnetfilter__log.html
* zstd: https://facebook.github.io/zstd/zstd_manual.html
-* mmap: https://stackoverflow.com/questions/35891525/mmap-for-writing-sequential-log-file-for-speed/35891885
-* pthread: http://www.cs.cmu.edu/afs/cs/academic/class/15492-f07/www/pthreads.html
-* apr_queue: http://apr.apache.org/docs/apr-util/1.3/apr__queue_8h.html
+* lz4: https://github.com/lz4/lz4