`ulog-acctd' 0.4 Manual

TITLE

This manual is for ulog-acctd, version 0.4

Copyright © 2002 Hilko Bengen

This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA

• Overview
• Installation
• Configuration: Setting things up
• Running: Using ulog-acctd
• Credits: Whom to complain to

1 Overview

ulog-acctd is a userspace network accounting daemon which generates log files of network traffic for accounting purposes.

• Features: What does ulog-acctd do?
• Nonfeatures: What does ulog-acctd not do?
• Others: Comparision with similar packages

1.1 Features

ulog-acctd collects headers of IP packets fhat travel through the Linux 2.4+ Netfilter framework. It writes accounting information to a log which can include

• protocol type,
• source and destination address,
• port information,
• byte and packet count,
• incoming and outgoing network interfaces.

The log file format is configurable. For instance, it should be possible to directly generate files that contain SQL statements so that the raw accounting data can be fed into a DBMS.

1.2 Nonfeatures

ulog-acctd only collects, aggregates, and logs the all data it is fed by netfilter. If only traffic for specific networks needs to be collected, selection for this should be done in the netfilter rulesets, as ulog-acctd has no way to ignore some packets and log others.

ulog-acctd does not analyze them nor paint pretty pictures for traffic visualization. In the "one tool for one job" spirit of UNIX, Other tools should be used for these tasks.

1.3 Others

• net--acct: Ulrich Callmeier's net--acct
• ipcad: Lev Walkin's ipcad
• traff: Hans Marcus Krüger's traff
• ipac--ng: Moritz Both's and Al Zaharov's ipac--ng
• ulogd

1.3.1 net–acct

net--acct (http://exorsus.net/projects/net–acct/) was developed by Ulrich Callmeier. The Mekafile hints that it can be used on Solaris (using libpcap). The Linux version uses the now obsolete packet interface at protocol level. (see ip(7)).

Information for net–acct is available at http://www.exorsus.net/net–acct/.

I had tried to implement support for the newer packet interface at device level (supported since Linux kernel versions 2.2, see packet(7)), but I found that it lost packets in high load situations. Patches against net--acct are available on request.

ulog-acctd is based on net--acct and still shares some portions of the code. net--acct does not seem to be actively maintained any longer.

1.3.2 ipcad

ipcad stands for IP Cisco Accounting Daemon. It runs in background and listens traffic on the specified interfaces. Statistics are available through a Cisco–compatible rsh interface. ipcad can use the BPF interface or libpcap, which on Linux probably leads to packet less under high load, as described above.

ipcad is available from http://sourceforge.net/projects/ipcad.

It is certainly possible to set up a similar scenario with ulog-acctd with a small rsh daemon (or maybe just a shell script) which would emulate the relevant parts of Cisco command line syntax.

1.3.3 traff

traff (http://sourceforge.net/projects/traff/) is a program that attaches itself to one or more network interfaces, sniffs all IP-packages passing at it and accounts each packets size. The collected information can be dumped to an SQL database (PostgreSQL or MySQL), syslog or a file.

Like ipcad, it uses libpcap.

1.3.4 ipac–ng

ipac--ng regularly reads byte and packet counters associated with the Linux kernel's packet filter rules and stores those numbers in a database. This works well for a small number of IP addresses / ports, but it does not scale well since one rule in the packet filter is needed for each possible entry.

1.3.5 ulogd

As the name suggests, ulogd uses the same approach for collecting IP packets as ulog-acctd does. ulogd can be extended through binary plugins for adding packet interpreters and output-targets (e.g. for logging into databases, user defined filetypes, etc.). It can be downloaded from http://www.gnumonks.org/projects/ulogd.

2 Installation

1. Compile the sources: cd src; make
2. Generate the documentation from the Texinfo source (optional): cd doc; make ulog-acctd.info ulog-acctd.html ulog-acctd.dvi
3. Install the ulog-acctd binary into appropriate location, i.e. /usr/local/sbin.
4. Install the configuration file src/ulog-acctd.conf to /etc and edit it to your needs.
5. Create the directories for logging that are referenced in the configuration file. (In the default configuration, this is /var/log/ulog-acctd/.)

3 Configuration

• Netfilter: Netfilter Configuration
• ulog-acctd.conf: ulog-acctd Configuration

3.1 Netfilter

ulog-acctd relies on the kernel (more specific: Netfilter) to pipe the packet headers to user space. This has to be enabled by adding rules with the ULOG target to packet filtering tables.

The following example causes all packets that successfully pass the FORWARD table to be logged to ulog-acctd.

     iptables --append FORWARD -j ULOG --ulog-nlgroup 1 --ulog-cprange 48 --ulog-qthreshold 50 --ulog-prefix "FORWARD"

Explaination:

• -j ULOG causes the packet to passed to userspace in the first place.
• --ulog-nlgroup GROUP specifies thet the packet be sent to multicast group GROUP. ulog-acctd has to be configured to listen to this group.
• --ulog-cprange SIZE means that only the first SIZE bytes are copied to userspace. The default is to copy the whole packet, limiting the number of bytes should increase performance. However, when IP headers carry a lot of option fields, this may lead to the case that TCP, UDP, or ICMP headers are out of the copied range. If the ulog-acctd will log a warning "Short IP header. Increase copy range to RANGE" to to the system log as well as the debug log in this case.

  If the copy range is too small to even hold the IP address, an additional error message "copy range is too short to even capture IP headers. ALL IP PACKETS WILL BE DROPPED!" is logged.

• --ulog-qthreshold COUNT makes the kernel collect COUNT packets before sending the message containing these packets to userspace.
• --ulog-prefix PREFIX causes a prefix string (max. 32 characters) to be passed along with the packet. This can for example be used for including hints for billing systems.

For further information, see iptables(8).

3.2 ulog-acctd.conf

Per default, configuration is done via the file /etc/ulog-acctd.conf. The following keywords are recognized:

• multicast groups = LIST: LIST is a comma-separated list of multicast groups that ulog-acctd should listen to. Possible values are 1-32.
• socket receive buffer = SIZE: The kernel's socket receive buffer will be set to SIZE bytes instead of the net/core/rmem_default kernel parameter's value. However, it can not be set to a value higher than the net/core/rmem_max kernel parameter. (Both of these kernel parameters are set to 64kBytes by the kernel, but can be canged with sysctl(8).
• hash table size = SIZE: ulog-acctd uses a hash table and linked lists to store the collected accounting data in memory. SIZE should be set to a power of 2. The default value is 65536.

  This has changed in version 0.4.2

• hash init value = VALUE: This value is used as a "salt" to the hash function. Only set this if you know what you are doing. The default value is 0.
• logger nice value = VALUE: ulog-acctd forks off separate processes for writing accounting data to disk. The nice value of these processes are set to VALUE. It is probably not advisable to set VALUE to a negative number. By default, the nice value is not changed.
• accounting file = FILENAME, dump file FILENAME, debug file = FILENAME, pid file = FILENAME: Locations for the accounting file, dump file, debug log, pid file.
• debug = LIST: LIST is a comma-spearated list of debug message types. Valid debug message types are: config, state, syscall, misc, statistics, signal, error, annoying, error-packet.
• accounting format = STRING: STRING is a format string with somewhat similar syntax to printf(3). The following directives are recognized and replaced in the output file.

  `%h'

  hostname
  

  `%x'

  since (when the first packet of a chunk was recorded)
  

  `%t'

  till (when the last packet of a ckunk was recorded)
  

  `%p'

  protocol
  

  `%s'

  source IP
  

  `%S'

  source port
  

  `%d'

  destination IP
  

  `%D'

  destination port
  

  `%P'

  packets
  

  `%b'

  bytes
  

  `%i'

  incoming interface
  

  `%o'

  outgoing interface
  

  `%r'

  prefix
  

  `%u'

  Local user that owns a TCP connection
  

  `%m'

  MAC address, if any.
  

  `%X'

  formatted date string (since)
  

  `%Z'

  formatted date string (till), see below for defining the format. The time when the output is written is used for this.
  

  `\t'

  tab
  

  `\n'

  newline
  

  `\\'

  Literal \
  

  `\"'

  Literal "
  

  `\%'

  Literal %
  

  `%%'

  Literal %

• date format = STRING STRING is a format string for turning the date into a string. All the directives that are known to strftime(3) on your system are recognized. This option has changed in version 0.4 of ulog-acctd.
• empty interface = STRING: If no incoming or outgoing interface is passed for a packet, insert STRING instead.
• empty prefix = STRING: If no ULOG-prefix is passed for a packet, insert STRING instead.
• flush = TIME: The data that has been accumulated in a in-memory structure is written to the accounting file every TIME seconds.
• fdelay = TIME: A certain record of traffic information may be written out after TIME seconds. This helps making the logfiles smaller since only one output record will be generated for related traffic

Example:

     
     multicast groups=1
     accounting file = /var/log/ulog-acctd/account.log
     dump file =       /var/log/ulog-acctd/dump
     debug file =      /var/log/ulog-acctd/debug.log
     debug = error
     accounting format = "%h\t%x\t%t\t%p\t%s\t%S\t%d\t%D\t%P\t%b\t\"%i\"\t\"%o\"\t\"%f\"\n"
     flush = 30
     fdelay = 30

The supplied accounting format string generates output records in the following form (all in one line, with a linefeed at the end):

judith	1032521450	1032521454	6	192.168.42.11	110	192.168.215.71	1546	12	250113	"eth0"	"eth1"	"prefix"

The following format string will generate cisco "ip accounting output-packets"-style output lines:

     
     accounting format = "%s %d %P %b\n"

The lines will look like this:

     
     192.168.42.11 192.168.215.71 12 250113

When not all types of information are collected, ulog-acctd will have to do fewer comparisions for each packet while collecting data. Records are smaller and likely fewer records will be written out. This will be the case especially if port information are not collected.

4 Running

• Invokation: Running ulog-acctd
• Signals: ulog-acctd understands some signals
• Rotating Log files
• Hints

4.1 Invokation

ulog-acctd recognizes the following command-line parameters:

• -d Switch on debugging
• -c Specify alternative configuration file
• -D Don't detach from terminal

4.2 Signals

The following signals are recognized by ulog-acctd:

`SIGHUP'

causes the config file to be re-read.

`SIGTSTP'

causes ulog-acctd to stop writing to the accounting logfile

`SIGCONT'

causes ulog-acctd to re-open both debug and log file and resume writing to the account log.

`SIGIOT'

causes ulog-acctd to re-open the capture socket.

`SIGINT'

`SIGTERM'

These signals stop the daemon, gracefully. The PID file is cleaned up and the IP data that is resident in memory is flushed to the accounting file.

`SIGKILL'

These signals stop the daemon.

`SIGALRM'

This is used internally. Do not send ulog-acctd this signal.

4.3 Rotating Log files

To cleanly rotate accounting log files, send SIGTSTP to the daemon, then move the logfiles out of the way, and finally send a SIGCONT to re-enable logging accounting data.

Be sure not to compress the log files immediately after rotating. When the accounting log file is written to, the writing process will create a hard link to the accounting file and gzip and bzip2 will not compress files with more than 1 link.

4.4 Hints

4.4.1 High CPU load and "No buffer space available" messages

This message in the debug log means that the kernel is delivering packets faster than ulog-acctd can process them. This happens most likely when ulog-acctd writes its accounting log to disk.

There are a few steps that can be taken both in kernel space and in ulog-acctd.

1. Increase socket receive buffer. This requires that the kernel parameter net/core/rmem_max is set with sysctl(8) to allow for a larger socket receive buffer.
2. Increase hash table size. However, be sure that the size of the hash table is a prime number.
3. Set a nice value for the child process that writes the account log.
4. Increase the --ulog-qthreshold value, if approtpiate
5. Decrease the --ulog-cprange value, if approtpiate. Experience shows that 24 to 32 bytes are enough for normal IP traffic.

4.4.2 fdelay

The fdelay configuration parameter can be used to reduce log file size where ulog-acctd is used to account many long-lasting connections and where it is not strictly necessary that the time values are absolutely correct.

4.4.3 The dump file

When fdelay is set to a non-zero value, ulog-acctd writes an additional dump file every time it writes to the accounting log. This dump file contains those records that have been kept in memory. If ulog-acctd crashes before writing to the accounting log file, these entries can still be recovered by the system administrator.

5 Credits

ulog-acctd was written by Hilko Bengen. Source tarballs and public CVS are accessible through http://alioth.debian.org/projects/pkg-ulog-acctd.

Thanks go to

• Marc Haber, for many useful ideas, and for pointing me in the right direction at times.
• Fredrik Vang, for his code for formatted dates in log files.
• Ulrich Callmeier, for net-acct which gave me a good base to start my work on. If ulog-acctd does not work for you, don't blame Ulrich or any other contributors mentioned in the source files, though.
• Stefan Neufeind, for his useful ideas and data from operating ulog-acctd in environments with high network load. (traffic from game servers).
• Patrick Schaaf, for coming up with a quick solution to synchronization issues, in the de.comp.os.unix.programing newsgroup.
• Radek Podgorny, for his "since" patch, for logging the start of a flow.

Initial work (up to version 0.4.1) on this program was funded by toplink–plannet GmbH, Karlsruhe, Germany http://www.toplink-plannet.de/.