Filters

json

class logcabin.filters.json.Json(field='data', consume=True, on_error='reject')

Parse a json encoded field.

Parameters:
  • field (string) – the field containing the json (default: data)
  • consume (boolean) – whether to remove the field after decoding (default: true)

Example:

Json()

syslog

class logcabin.filters.syslog.Syslog(field='data', consume=True, on_error='reject')

Parse a syslog encoded field.

This sets the fields:

  • timestamp
  • facility
  • severity
  • host
  • program
  • pid
  • message
Parameters:
  • field (string) – the field containing the syslog message (default: data)
  • consume (boolean) – whether to remove the field after decoding (default: true)

Example:

Syslog()

mutate

class logcabin.filters.mutate.Mutate(set={}, rename={}, copy={}, unset=[])

Filter that allows you to add, rename, copy and drop fields

Parameters:
  • set (map) – fields to set (optional). The values if strings may format other fields from the event.
  • rename (map) – fields to rename (a: b renames b to a) (optional)
  • unset (list) – fields to unset (optional)

Example:

Mutate(set={'fullname': '{first} {last}'})

Renaming:

Mutate(rename={'@timestamp': 'timestamp', '@message': 'message'})

Unsetting:

Mutate(unset=['junk', 'rubbish'])

python

class logcabin.filters.python.Python(function, on_error='reject')

Call out to a python function for adding custom functionality.

Parameters:function (callable) – callable taking the event as an argument

Example:

Python(function=lambda ev: ev.count = int(ev.count))

Alternatively, a function can be passed, for more complex functionality:

def clean(ev):
    ev.header = ev.header.strip()
    ev.message = ev.message.strip()

Python(function=clean)

regex

class logcabin.filters.regex.Regex(regex, field='data', on_error='reject')

Parse a field with a regular expression. The regex named groups (?P<name>...) will be create event fields (overwriting any existing).

If you extract a ‘timestamp’ field, this will automatically be parsed as a datetime and used as the event timestamp (instead of the default of the time received).

Parameters:
  • regex (string) – the regular expression
  • field (string) – the field to run the regex on (default: data)

Example:

Regex(regex='(?P<timestamp>.+) - (?P<message>.+)')

stats

class logcabin.filters.stats.Stats(period=5, metrics=None, zero=True)

Filter that produces aggregate statistics.

It will produce:

  • name.count: number of data points
  • name.rate: the data points per second
  • name.mean: mean of data points
  • name.min: minimum data point
  • name.median: median data point
  • name.upper95: 95th% data point
  • name.upper99: 99th% data point
  • name.max: maximum data point
  • name.stddev: standard deviation

This is emitted as a single event, every period.

Parameters:
  • period (integer) – period to report stats, in seconds
  • metrics (map) – field names => values. Any fields from the events can be formatting into the field names. Values can be an event field, nested path to a field (separated by .) and can contain wildcard ‘*’, to indicate generating statistics from any numerical fields.
  • zero (boolean) – output zero for previously seen metrics (useful to disambiguate no activity and output broken)

Example:

Stats(metrics={'rails.{controller}.{action}.duration': 'duration'})

Wildcards can be used to pull out nested structures:

Stats(metrics={'app.{1}': 'timings.*'})