Dovecot CE
Search K

Appearance

Statistics ​

See All Dovecot Events for list of all events that can be used in statistics.

Dovecot supports gathering statistics from events (see Events Design). Currently there are no statistics logged by default, and therefore they must be explicitly added using the metric configuration blocks.

The metric_filter setting is the only required setting in a metric block. It specifies which events should be used when calculating the statistics for a given metric block. Event filtering is described in detail in Event Filtering.

Note that Dovecot also has many unnamed events. These aren't generally useful for statistics, but in some situations they may become visible in statistics. To avoid surprises, it's a good idea to always specify event=name in the filter setting. You can also use event=* to match all named events.

Settings ​

metric_description

Default[None]
Valuestring

Human-readable description of the metric. This is included in the HELP text sent to OpenMetrics.

metric_exporter_include

Defaultname hostname timestamps categories fields
ValueBoolean List

Specifies which parts of the event are exported to the serialized event:

Values Description
name The name of the event.
hostname The name of the host generating this event.
timestamps The event start and end timestamps.
categories A set of categories associated with this event.
fields The fields associated with this event. The fields that will be exported are defined by the metric_fields setting.

For example:

metric example {
  exporter_include = name hostname timestamps
}

includes just the 3 specified parts, while

metric another_example {
  exporter_include =
}

includes nothing and the exported event will be empty (i.e. {} in JSON).

metric_fields

Default[None]
ValueBoolean List
Changes
  • Changed: 2.4.0

    All listed fields are exported to OpenMetrics as well.

A list of fields included in the metric. All events have a default duration field that does not need to be listed explicitly.

metric_group_by_method_discrete_modifier

Default[None]
ValueString without variables
See Also

Configures a modifier string for values grouped by the discrete method. %variables and their functions can be used:

%{value}
The original value.
%{user | domain}
If the value is in user@domain format, this contains the domain text. Otherwise empty.

metric_name

Default[None]
Valuestring

Name of the metric. It is visible in statistics outputs. The metric filter name refers to this setting.

Group By ​

The metric_group_by setting allows dynamic hierarchical metric generation based on event fields' values.

Each listed metric_group_by generates one level of "sub-metrics". These automatically generated metrics are indistinguishable from those statically defined in the config file. "sub-metric" names can be up to 256 bytes in total.

Dovecot supports a number of aggregation methods that can be used to quantize a field's value before it is used to generate a metric.

discrete ​

The simplest aggregation method is to use the value as is. Because this is a very common use case, this is the default aggregation method.

The value can be further modified by metric_group_by_method_discrete_modifier.

Added: 2.4.0 An additional parameter can be added to provide modifiers to the discrete value. This is done as %variables and their modifiers. The following variables are provided:

  • %{value} - The original value
  • %{user | domain} - Text after the @ character, or empty string if there is no @.

Example:

metric imap_command {
  filter = event=imap_command_finished
  group_by cmd_name {
  }
  group_by tagged_reply_state {
  }
}
metric login_domains {
  filter = event=auth_request_finished
  fields = user
  group_by user {
    method discrete {
      modifier = %{value | domain | lower}
    }
  }
}

This example configuration will generate statistics for each IMAP command. The first "sub-metric" level is based on the IMAP command name, and the second (and in this example final) level is based on the tagged reply. For example, a SELECT IMAP command that succeeded (in other words, it had an OK reply) will generate the metric imap_command_SELECT_ok.

In addition to the final level metric, all intermediate level metrics are generated as well. For example, the same SELECT IMAP command will generate all of the following metrics:

  • imap_command
  • imap_command_SELECT
  • imap_command_SELECT_ok

Note: While the top level metrics (e.g., imap_command above) are generated at start up, all group_by metrics are generated dynamically when first observed.

exponential ​

The field's integer value is quantized into exponentially sized ranges.

The exponential aggregation method uses three settings:

The first range starts at negative infinity and ends at pow(base, min_magnitude). The second range begins at pow(base, min_magnitude) + 1 and ends at pow(base, min_magnitude + 1), the next covers pow(base, min_magnitude + 1) + 1 to pow(base, min_magnitude + 2), and so on. The last range covers pow(base, max_magnitude) + 1 to positive infinity.

For example, given the settings metric_group_by_method_exponential_min_magnitude = 1, metric_group_by_method_exponential_max_magnitude = 5 and metric_group_by_method_exponential_base = 10, the ranges would be:

  • (-inf, 10]
  • [11, 100]
  • [101, 1000]
  • [1001, 10000]
  • [10001, 100000]
  • [100001, +inf)

Much like the metric names generated with the discrete aggregation method, the ones generated by the exponential method include information about the value of the field. However, in this case it is the range the value belongs to.

Specifically, it is the name of the field being quantized, and the lower and upper bounds for the range.

Example:

metric imap_command {
  filter = event=imap_command_finished
  group_by cmd_name {
  }
  group_by duration {
    method exponential {
      min_magnitude = 1
      max_magnitude = 5
      base = 10
    }
  }
}

This will generate metric names of the format imap_command_{cmd}_duration_{min}_{max} where {cmd} is the IMAP command name, and {min} and {max} are the range bounds. Therefore, for a SELECT IMAP command, the possible generated metric names are:

  • imap_command_SELECT_ninf_10
  • imap_command_SELECT_11_100
  • imap_command_SELECT_101_1000
  • imap_command_SELECT_1001_10000
  • imap_command_SELECT_10001_100000
  • imap_command_SELECT_100001_inf

Note: Since the metric names cannot contain -, the string ninf is used to denote negative infinity.

Note: Much like in the discrete case, the metrics are allocated only when first observed.

Finally, because all intermediate level metrics are generated as well. The above example, will also generate all of the following metrics:

  • imap_command
  • imap_command_SELECT

linear ​

The field's integer value is quantized into linearly sized ranges.

The linear aggregation method uses three settings:

The first range starts at negative infinity and ends at min. The second range begins at min + 1 and ends at min + step, the next covers min + step + 1 to min + (2 * step), and so on. The last range covers max + 1 to positive infinity.

For example, given settings metric_group_by_method_linear_min = 0, metric_group_by_method_linear_max = 5000 and metric_group_by_method_linear_step = 1000, the ranges would be:

  • (-inf, 0]
  • [1, 1000]
  • [1001, 2000]
  • [2001, 3000]
  • [3001, 4000]
  • [4001, 5000]
  • [5001, +inf)

See the description of the exponential aggregation method for how metric names are formed from these ranges.

Listing Statistics ​

The gathered statistics are available by running:

sh
doveadm stats dump

Each event has a duration field, which tracks in microseconds how long the event existed. For example with imap_command_finished field it could be:

metric_name          field      count  sum       min  max     avg      median stddev %95
imap_commands        duration   35     1190122   162  340477  34003    244    31215  188637

The above means:

FieldDescription
countThere have been 35 IMAP commands
sumThe IMAP commands were running in total for 1190122 microseconds (= 1.1 seconds)
minThe fastest IMAP command took 162 microseconds
maxThe slowest IMAP command took 340477 microseconds
avgThe average time spent on an IMAP commands was 34003 microseconds
medianThe median time spent on an IMAP command was 244 microseconds
stddevStandard deviation for the time spent on IMAP commands
%9595% of the IMAP commands took 188637 microseconds or less

The other fields (than duration) track whatever that field represents. For example with imap_command_finished's net_in_bytes field could be tracking how many bytes were being used by the IMAP commands. Non-numeric fields can also be tracked, although only the count is relevant to those.

The list of fields can be specified with the -f parameter. The default is:

sh
doveadm stats dump -f 'count sum min max avg median stddev %95'

It's also possible to specify other percentiles than just 95%, for example:

sh
doveadm stats dump -f 'count sum min max avg median stddev %95 %99 %99.9 %99.99'

The stats counters are reset whenever the stats process is started, which also means a dovecot reload will reset statistics. Using doveadm stats -r parameter will also reset the statistics atomically after they're dumped.

Modifying Statistics Dynamically ​

Metrics can be added or removed dynamically. The changes do not persist after configuration reload.

Metrics can be added dynamically by running doveadm stats add.

For example:

sh
doveadm stats add --description "IMAP SELECT commands" \
  --exporter log-exporter --exporter-include "name timestamps" \
  --fields "net_in_bytes net_out_bytes" \
  --group-by "cmd_name tagged_reply_state" imap_cmd_select \
  "event=imap_command_finished AND cmd_name=SELECT"

Metrics can be removed dynamically by running doveadm stats remove.

For example:

sh
doveadm stats remove imap_cmd_select

Examples ​

IMAP Command Statistics ​

metric imap_select_no {
  filter = event=imap_command_finished AND cmd_name=SELECT AND \
      tagged_reply_state=NO
}

metric imap_select_no_notfound {
  filter = event=imap_command_finished AND cmd_name=SELECT AND \
      tagged_reply="NO*Mailbox doesn't exist:*"
}

metric storage_http_gets {
  filter = event=http_request_finished AND category=storage AND \
      method=get
}

# generate per-command metrics on successful commands
metric imap_command {
  filter = event=imap_command_finished AND tagged_reply_state=OK
  group_by cmd_name {
  }
}

Push Notifications ​

metric push_notifications {
  filter = event=push_notification_finished
}

# for OX driver
metric push_notification_http_finished {
  filter = event=http_request_finished AND category=push_notification
}

OpenMetrics ​

Basic Configuration ​

Dovecot has support for OpenMetrics exposition format for statistics.

This can be enabled by adding following configuration:

service stats {
  inet_listener http {
    port = 9900
  }
}

This will enable Dovecot to expose all configured metrics in OpenMetrics format on http://host:9900/metrics using text-based format.

Statistics Format ​

By default, Dovecot exposes all configured metrics.

If the metric name does not conform with OpenMetrics requirements, it is not exported.

All metric names are prefixed with dovecot_ and each non-histogram metric is exported as dovecot_<metric_name>_total and dovecot_<metric_name>_duration_seconds_total.

Dynamically generated statistics with group_by will be exported too.

The name of the base metric is used as above, and any dynamically generated sub-metrics are exported using labels.

Quantized sub-metrics are exported as histograms.

Histograms are exported as dovecot_<metric_name>_bucket with corresponding labels. Each histogram will have an automatically generated _sum (specifying sum of all values in quantiles) and _count (total number of samples in the quantiles) metrics.

Dovecot will also export version information and startup time as special metrics even if nothing is configured. These are called dovecot_build_info and process_start_time_seconds.

Example ​

An excerpt of an example Dovecot configuration that defines a set of metrics, and the sample exported data with such metrics configuration:

metric auth_success {
  filter = (event=auth_request_finished AND success=yes)
}

metric imap_command {
  filter = event=imap_command_finished
  group_by cmd_name {
  }
  group_by tagged_reply_state {
  }
}

metric smtp_command {
  filter = event=smtp_server_command_finished
  group_by cmd_name {
  }
  group_by status_code {
  }
  group_by duration {
    method exponential {
      min_magnitude = 1
      max_magnitude = 5
    }
  }
}

metric mail_delivery {
  filter = event=mail_delivery_finished
  group_by duration {
    method exponential {
      min_magnitude = 1
      max_magnitude = 5
    }
  }
}
# HELP process_start_time_seconds Timestamp of service start
# TYPE process_start_time_seconds gauge
process_start_time_seconds 1606393397
# HELP dovecot_build Dovecot build information
# TYPE dovecot_build info
dovecot_build_info{version="2.4.devel",revision="38ecc424a"} 1
# HELP dovecot_auth_success Total number of all events of this kind
# TYPE dovecot_auth_success counter
dovecot_auth_success_total 892
# HELP dovecot_auth_success_duration_seconds Total duration of all events of this kind
# TYPE dovecot_auth_success_duration_seconds counter
dovecot_auth_success_duration_seconds_total 0.085479
# HELP dovecot_imap_command Total number of all events of this kind
# TYPE dovecot_imap_command counter
dovecot_imap_command_total{cmd_name="LIST"} 423
dovecot_imap_command_total{cmd_name="LIST",tagged_reply_state="OK"} 423
dovecot_imap_command_total{cmd_name="STATUS"} 468
dovecot_imap_command_total{cmd_name="STATUS",tagged_reply_state="OK"} 468
dovecot_imap_command_total{cmd_name="SELECT"} 890
dovecot_imap_command_total{cmd_name="SELECT",tagged_reply_state="OK"} 890
dovecot_imap_command_total{cmd_name="APPEND"} 449
dovecot_imap_command_total{cmd_name="APPEND",tagged_reply_state="OK"} 449
dovecot_imap_command_total{cmd_name="LOGOUT"} 892
dovecot_imap_command_total{cmd_name="LOGOUT",tagged_reply_state="OK"} 892
dovecot_imap_command_total{cmd_name="UID FETCH"} 888
dovecot_imap_command_total{cmd_name="UID FETCH",tagged_reply_state="OK"} 888
dovecot_imap_command_total{cmd_name="FETCH"} 2148
dovecot_imap_command_total{cmd_name="FETCH",tagged_reply_state="OK"} 2148
dovecot_imap_command_total{cmd_name="STORE"} 794
dovecot_imap_command_total{cmd_name="STORE",tagged_reply_state="OK"} 794
dovecot_imap_command_total{cmd_name="EXPUNGE"} 888
dovecot_imap_command_total{cmd_name="EXPUNGE",tagged_reply_state="OK"} 888
dovecot_imap_command_count 7840
# HELP dovecot_imap_command_duration_seconds Total duration of all events of this kind
# TYPE dovecot_imap_command_duration_seconds counter
dovecot_imap_command_duration_seconds_total{cmd_name="LIST"} 0.099115
dovecot_imap_command_duration_seconds_total{cmd_name="LIST",tagged_reply_state="OK"} 0.099115
dovecot_imap_command_duration_seconds_total{cmd_name="STATUS"} 0.161195
dovecot_imap_command_duration_seconds_total{cmd_name="STATUS",tagged_reply_state="OK"} 0.161195
dovecot_imap_command_duration_seconds_total{cmd_name="SELECT"} 0.184907
dovecot_imap_command_duration_seconds_total{cmd_name="SELECT",tagged_reply_state="OK"} 0.184907
dovecot_imap_command_duration_seconds_total{cmd_name="APPEND"} 0.273893
dovecot_imap_command_duration_seconds_total{cmd_name="APPEND",tagged_reply_state="OK"} 0.273893
dovecot_imap_command_duration_seconds_total{cmd_name="LOGOUT"} 0.033494
dovecot_imap_command_duration_seconds_total{cmd_name="LOGOUT",tagged_reply_state="OK"} 0.033494
dovecot_imap_command_duration_seconds_total{cmd_name="UID FETCH"} 0.181319
dovecot_imap_command_duration_seconds_total{cmd_name="UID FETCH",tagged_reply_state="OK"} 0.181319
dovecot_imap_command_duration_seconds_total{cmd_name="FETCH"} 1.169456
dovecot_imap_command_duration_seconds_total{cmd_name="FETCH",tagged_reply_state="OK"} 1.169456
dovecot_imap_command_duration_seconds_total{cmd_name="STORE"} 0.368621
dovecot_imap_command_duration_seconds_total{cmd_name="STORE",tagged_reply_state="OK"} 0.368621
dovecot_imap_command_duration_seconds_total{cmd_name="EXPUNGE"} 0.247657
dovecot_imap_command_duration_seconds_total{cmd_name="EXPUNGE",tagged_reply_state="OK"} 0.247657
dovecot_imap_command_duration_seconds_sum 2.719657
# HELP dovecot_smtp_command Histogram
# TYPE dovecot_smtp_command histogram
dovecot_smtp_command_bucket{cmd_name="LHLO",status_code="250",le="10"} 0
dovecot_smtp_command_bucket{cmd_name="LHLO",status_code="250",le="100"} 1
dovecot_smtp_command_bucket{cmd_name="LHLO",status_code="250",le="1000"} 1
dovecot_smtp_command_bucket{cmd_name="LHLO",status_code="250",le="10000"} 1
dovecot_smtp_command_bucket{cmd_name="LHLO",status_code="250",le="100000"} 1
dovecot_smtp_command_bucket{cmd_name="LHLO",status_code="250",le="+Inf"} 1
dovecot_smtp_command_sum{cmd_name="LHLO",status_code="250"} 0.000020
dovecot_smtp_command_count{cmd_name="LHLO",status_code="250"} 1
dovecot_smtp_command_bucket{cmd_name="MAIL",status_code="250",le="10"} 0
dovecot_smtp_command_bucket{cmd_name="MAIL",status_code="250",le="100"} 1
dovecot_smtp_command_bucket{cmd_name="MAIL",status_code="250",le="1000"} 1
dovecot_smtp_command_bucket{cmd_name="MAIL",status_code="250",le="10000"} 1
dovecot_smtp_command_bucket{cmd_name="MAIL",status_code="250",le="100000"} 1
dovecot_smtp_command_bucket{cmd_name="MAIL",status_code="250",le="+Inf"} 1
dovecot_smtp_command_sum{cmd_name="MAIL",status_code="250"} 0.000021
dovecot_smtp_command_count{cmd_name="MAIL",status_code="250"} 1
dovecot_smtp_command_bucket{cmd_name="RCPT",status_code="250",le="10"} 0
dovecot_smtp_command_bucket{cmd_name="RCPT",status_code="250",le="100"} 0
dovecot_smtp_command_bucket{cmd_name="RCPT",status_code="250",le="1000"} 1
dovecot_smtp_command_bucket{cmd_name="RCPT",status_code="250",le="10000"} 1
dovecot_smtp_command_bucket{cmd_name="RCPT",status_code="250",le="100000"} 1
dovecot_smtp_command_bucket{cmd_name="RCPT",status_code="250",le="+Inf"} 1
dovecot_smtp_command_sum{cmd_name="RCPT",status_code="250"} 0.000195
dovecot_smtp_command_count{cmd_name="RCPT",status_code="250"} 1
dovecot_smtp_command_bucket{cmd_name="DATA",status_code="250",le="10"} 0
dovecot_smtp_command_bucket{cmd_name="DATA",status_code="250",le="100"} 0
dovecot_smtp_command_bucket{cmd_name="DATA",status_code="250",le="1000"} 0
dovecot_smtp_command_bucket{cmd_name="DATA",status_code="250",le="10000"} 1
dovecot_smtp_command_bucket{cmd_name="DATA",status_code="250",le="100000"} 1
dovecot_smtp_command_bucket{cmd_name="DATA",status_code="250",le="+Inf"} 1
dovecot_smtp_command_sum{cmd_name="DATA",status_code="250"} 0.001249
dovecot_smtp_command_count{cmd_name="DATA",status_code="250"} 1
dovecot_smtp_command_bucket{cmd_name="QUIT",status_code="221",le="10"} 1
dovecot_smtp_command_bucket{cmd_name="QUIT",status_code="221",le="100"} 1
dovecot_smtp_command_bucket{cmd_name="QUIT",status_code="221",le="1000"} 1
dovecot_smtp_command_bucket{cmd_name="QUIT",status_code="221",le="10000"} 1
dovecot_smtp_command_bucket{cmd_name="QUIT",status_code="221",le="100000"} 1
dovecot_smtp_command_bucket{cmd_name="QUIT",status_code="221",le="+Inf"} 1
dovecot_smtp_command_sum{cmd_name="QUIT",status_code="221"} 0.000010
dovecot_smtp_command_count{cmd_name="QUIT",status_code="221"} 1
# HELP dovecot_mail_delivery Histogram
# TYPE dovecot_mail_delivery histogram
dovecot_mail_delivery_bucket{le="10"} 0
dovecot_mail_delivery_bucket{le="100"} 0
dovecot_mail_delivery_bucket{le="1000"} 1
dovecot_mail_delivery_bucket{le="10000"} 1
dovecot_mail_delivery_bucket{le="100000"} 1
dovecot_mail_delivery_bucket{le="+Inf"} 1
dovecot_mail_delivery_sum 0.000656
dovecot_mail_delivery_count 1
# EOF

Copyright © Open-Xchange GmbH