Filters¶
json¶
- class logcabin.filters.json.Json(field='data', consume=True, on_error='reject')¶
Parse a json encoded field.
Parameters: - field (string) – the field containing the json (default: data)
- consume (boolean) – whether to remove the field after decoding (default: true)
Example:
Json()
syslog¶
- class logcabin.filters.syslog.Syslog(field='data', consume=True, on_error='reject')¶
Parse a syslog encoded field.
This sets the fields:
- timestamp
- facility
- severity
- host
- program
- pid
- message
Parameters: - field (string) – the field containing the syslog message (default: data)
- consume (boolean) – whether to remove the field after decoding (default: true)
Example:
Syslog()
mutate¶
- class logcabin.filters.mutate.Mutate(set={}, rename={}, copy={}, unset=[])¶
Filter that allows you to add, rename, copy and drop fields
Parameters: - set (map) – fields to set (optional). The values if strings may format other fields from the event.
- rename (map) – fields to rename (a: b renames b to a) (optional)
- unset (list) – fields to unset (optional)
Example:
Mutate(set={'fullname': '{first} {last}'})
Renaming:
Mutate(rename={'@timestamp': 'timestamp', '@message': 'message'})
Unsetting:
Mutate(unset=['junk', 'rubbish'])
python¶
- class logcabin.filters.python.Python(function, on_error='reject')¶
Call out to a python function for adding custom functionality.
Parameters: function (callable) – callable taking the event as an argument Example:
Python(function=lambda ev: ev.count = int(ev.count))
Alternatively, a function can be passed, for more complex functionality:
def clean(ev): ev.header = ev.header.strip() ev.message = ev.message.strip() Python(function=clean)
regex¶
- class logcabin.filters.regex.Regex(regex, field='data', on_error='reject')¶
Parse a field with a regular expression. The regex named groups (?P<name>...) will be create event fields (overwriting any existing).
If you extract a ‘timestamp’ field, this will automatically be parsed as a datetime and used as the event timestamp (instead of the default of the time received).
Parameters: - regex (string) – the regular expression
- field (string) – the field to run the regex on (default: data)
Example:
Regex(regex='(?P<timestamp>.+) - (?P<message>.+)')
stats¶
- class logcabin.filters.stats.Stats(period=5, metrics=None, zero=True)¶
Filter that produces aggregate statistics.
It will produce:
- name.count: number of data points
- name.rate: the data points per second
- name.mean: mean of data points
- name.min: minimum data point
- name.median: median data point
- name.upper95: 95th% data point
- name.upper99: 99th% data point
- name.max: maximum data point
- name.stddev: standard deviation
This is emitted as a single event, every period.
Parameters: - period (integer) – period to report stats, in seconds
- metrics (map) – field names => values. Any fields from the events can be formatting into the field names. Values can be an event field, nested path to a field (separated by .) and can contain wildcard ‘*’, to indicate generating statistics from any numerical fields.
- zero (boolean) – output zero for previously seen metrics (useful to disambiguate no activity and output broken)
Example:
Stats(metrics={'rails.{controller}.{action}.duration': 'duration'})
Wildcards can be used to pull out nested structures:
Stats(metrics={'app.{1}': 'timings.*'})