Event Export¶

New in version v2.3.7.

There are two parts to the configuration:

exporter definitions, and
event definition.

Exporter Definition¶

The event_exporter block defines how events should be exported. The basic definition is split into two orthogonal parts: the format and the transport.

The format and its arguments specify how an event is serialized, while the transport and its arguments specify where the serialized event is sent.

In both cases, the behavior is tweaked via the corresponding arguments setting.

For example, the following block defines an exporter that uses the foo transport and json format:

event_exporter ABC {
  format = json
  format_args = time-rfc3339
  transport = foo
  transport_args = bar
  transport_timeout = 500msec
}

Formats¶

The format and its arguments specify how an event is serialized.

Since some formats cannot express certain values natively (e.g., JSON does not have a timestamp data type), the format_args setting can be used to influence the serialization algorithm’s output.

There are two formats and two format args.

Formats:¶

json
tab-text

Format Args:¶

time-rfc3339 - serialize timestamps as strings using the RFC 3339 format (YYYY-MM-DDTHH:MM:SS.uuuuuuZ)
time-unix - serialize timestamps as a floating point number of seconds since the Unix epoch

Example JSON¶

Note: This example is pretty-printed. The actual exported event omits the whitespace between the various tokens.

{
   "event" : "imap_command_finished",
   "hostname" : "dovecot-dev",
   "start_time" : "2019-06-19T10:38:25.422744Z",
   "end_time" : "2019-06-19T10:38:25.424812Z",
   "categories" : [
      "imap"
   ],
   "fields" : {
      "bytes_in" : 7,
      "bytes_out" : 311,
      "last_run_time" : "2019-06-19T10:38:25.422709Z",
      "lock_wait_usecs" : 60,
      "name" : "SELECT",
      "running_usecs" : 1953,
      "session" : "xlBB1KqLz1isGwB+",
      "tag" : "a0005",
      "tagged_reply" : "OK [READ-WRITE] Select completed",
      "tagged_reply_state" : "OK",
      "user" : "jeffpc"
   }
}

Example tab-text¶

event:imap_command_finished        hostname:dovecot-dev    start_time:2019-06-19T10:38:25.422744Z  end_time:2019-06-19T10:38:25.424812Z    category:imap   field:user=jeffpc       field:session=xlBB1KqLz1isGwB+  field:tag=a0005 field:cmd_name=SELECT       field:tagged_reply_state=OK     field:tagged_reply=OK [READ-WRITE] Select completed     field:last_run_time=2019-06-19T10:38:25.422709Z field:running_usecs=1953        field:lock_wait_usecs=60        field:bytes_in=7        field:bytes_out=311

Transports¶

The transport and its arguments specify where the serialized event is sent.

Currently, there are three transports:

drop - ignore the serialized event
log - send serialized event to syslog
http-post - send the serialized event as a HTTP POST payload to the URL specified in the transport_arg setting with a timeout specified by transport_timeout

The drop transport is useful when one wants to disable the event exporter temporarily. Note that serialization still occurs, but the resulting payload is simply freed.

The log transport is useful for debugging as typically one is already looking at the logs.

Caution: It is possible for the stats process to consume a large amount of memory buffering the POST requests if the timeout for http-post is set very high, a lot of events are being generated, and the HTTP server is slow.

Event Definition¶

The event definition reuses and extends the metric config block used for statistics gathering. The only additions to the block are the exporter and exporter_include settings.

These are only meaningful if the event matches the predicate (categories, filter, etc.) specified in the metric block.

Filtering Events¶

One uses the metric block settings documented in Statistics to select and filter the event to be exported.

exporter¶

The exporter setting identifies which exporter should be used to export this event. If the setting is not specified, this event is not exported. (This is to allow certain metrics to be used only for statistics.)

exporter_include¶

There are five possible parts that can be included in a serialized event:

name - the name of the event
hostname - the name of the host generating this event
timestamps - the event start and end timestamps
categories - a set of categories associated with this event
fields - the fields associated with this event; the fields that will be
exported are defined by the fields setting in the parent metric block

The exporter_include setting is made up of these tokens which control what parts of an event are exported. It can be set to any set of those (including empty set) and the order doesn’t matter. It defaults to all 5 tokens.

For example, exporter_include=name hostname timestamps includes just the 3 specified parts, while exporter_include= includes nothing - the exported event will be empty (e.g., {} in JSON).

Example Configs¶

If one wishes to send the events associated with IMAP commands completion to a datalake having a HTTP API, one could use config such as:

event_exporter datalake {
  format = json
  format_args = time-rfc3339
  transport = http-post
  transport_args = https://datalake.example.com/api/endpoint/somewhere
  transport_timeout = 1sec
}

metric imap_commands {
  exporter = datalake
  exporter_include = name hostname timestamps
  filter = event=imap_command_finished
}

When debugging, it is sometimes useful to dump information to the log. For example, to output all named events from the IMAP service:

event_exporter log {
  format = json
  format_args = time-rfc3339
  transport = log
}

metric imap_commands {
  exporter = log
  filter = event=* AND category=service:imap
}