Upgrading Dovecot CE from 2.3 to 2.4

Upgrade Path

Before upgrading, please look at the list of removed features carefully.

If you are doing in-place upgrade, ensure that you first upgrade to latest 2.3 release, and then upgrade to 2.4.

Alternatively, you can migrate your data to new setup.

Dovecot CE

Configuration Changes

WARNING

Dovecot 2.3.x settings will NOT work unless the configuration is changed as described in this section.

Required Settings

The first setting in dovecot.conf MUST now be dovecot_config_version. This helps to avoid unexpected configuration changes in the future.

Another new required setting is dovecot_storage_version. This helps to avoid unexpected storage file format incompatibilities.

Note that the configuration syntax has been changed, and your old configuration will not work without changes.

Variable expansion

A new variable expansion design syntax has been introduced. All old one-letter %variables have been removed, and some old variable aliases have been dropped too.

List of common short variables and their replacements.

Old variable	New syntax
%a	%{local_port}
%b	%{remote_port}
%c	%{certificate}
%c	%{secured}
%d	%{user | domain }
%h	%{home}
%l	%{local_ip }
%m	%{mechanism}
%n	%{user | username }
%p	%{client_pid }
%r	%{remote_ip }
%s	%{protocol}
%u	%{user}
%w	%{password}

Common modifier syntaxes and their replacements.

Old syntax	New syntax
%1Mu	%{user | md5 | hexlify(1)}
%2.1Mu	%{user | md5 | hexlify | substr(2,1)}
%2.256N	%{ user | md5 | substr(0, 8) % 256 | hex(2)}
%256Nu	%{ user | md5 | substr(0, 8) % 256 | hex}
%Hu	No conversion available, use one of the syntaxes.
%Ln	%{user | username | lower }
%Lu	%{user | lower }
%Mu	%{user | md5}

Common long variables and their replacements.

Old variable	New syntax
%{auth_domain}	%{auth_user | domain}
%{auth_username}	%{auth_user | username}
%{lip}	%{local_ip}
%{lport}	%{local_port}
%{mech}	%{mechanism}
%{orig_domain }	%{original_user | domain }
%{orig_username}	%{original_user | username }
%{orig_user}	%{original_user}
%{real_lip}	%{real_local_ip}
%{real_lport}	%{real_local_port }
%{real_rip}	%{real_remote_ip }
%{real_rport}	%{real_remote_port}
%{rip}	%{remote_ip}
%{rport}	%{remote_port }
%{service}	%{protocol}

Directory hashing

If you have been using /home/%2.256N/%u or similar constructs:

• How to replace %N in new format:

  • %2.256Nu becomes %{ user | md5 | substr(0, 8) % 256 | hex(2)} to return maximum 256 different hashes in range 00..ff.

  • %256Nu becomes %{ user | md5 | substr(0, 8) % 256 | hex} to return maximum 256 different hashes in range 0..ff (without 0-padding in the front).

• How to replace '%M' in new format:

  • %1Mu/%2.1Mu/%u becomes %{user | md5 | hexlify(1)}/%{user | md5 | hexlify | substr(2,1)}/%{user} to returns directories from 0/0/user to f/f/user.

• There is no way to use '%H' anymore.

Unknown/Invalid Variables

Unknown/invalid %{variables} cause Dovecot errors. This may cause, e.g., authentication failures if the old (broken) behavior was relied on.

passdb/userdb Section Naming

passdb and userdb sections now require a name, i.e.:

# This gives an error:
passdb {
  ...
}

# Use this instead:
passdb some_name {
}

Empty userdb Variables

userdb fields can be set to empty value. Previously they became changed to yes value.

Added Features

Auth Policy Parameters

See authentication policy.

Parameter	Notes
%{fail_type} variable to auth_policy_request_attributes	Variable was added.

Cryptographic Features

Feature	Notes
ARGON2 password scheme	Support for the ARGON2 password scheme was added.
SCRAM-SHA-1, SCRAM-SHA-256	Support SASL mechanisms for outgoing connections.
X25519, X448	mail-crypt plugin and Mail Crypt Plugin: fs-crypt now support these curves.

imapc_features Parameters

See imapc_features.

Feature	Notes
no-qresync	Parameter was added.

Settings

• auth_internal_failure_delay
• fts_message_max_size
• login_socket_path
• quota_mailbox_count
• quota_mailbox_message_count
• submission_add_received_header
• cassandra_log_retries

Deprecated Features

Feature	Notes
acl_global_path	Using ACL file is deprecated, see below for details on migration.
fs-sis	Saving new mails' attachments via fs-sis is disabled, but reading SIS attachments is still supported. Missing SIS attachments are replaced with files filled with spaces.

Removed Features

Feature	Notes
fts-lucene, fts-squat	Use fts-flatcurve plugin or fts-solr plugin.
dsync: Remove -D parameter	Parameter for disabling mailbox rename syncing removed. It hasn't been necessary for a long time, and it is broken.
dsync	Use doveadm sync instead. dsync has been a symlink to doveadm already, this release removed the symlink completely.
director	Support for the Dovecot Director architecture has been removed. For a small-scale, unsupported replacement, see Lua Director.
replicator	Use NFS or some other shared filesystem instead.
Dovecot director role	Replaced with the Dovecot Pro Palomar architecture.
Global ACL directory	Use acl instead. See below for details on migration.
IMAP SETQUOTA command	Quota limits can no longer be modified using the IMAP SETQUOTA command. The set_quota setting has been removed.
IPC process	Has been merged to anvil.
OpenSSL support for older than 1.0.2	Older versions are not supported anymore.
Sieve extensions: notify, imapflags, vnd.dovecot.duplicate	These deprecated Sieve extensions have been removed.
ssl-parameters.dat	This file is no longer converted automatically by config process, you need to set ssl_server_dh_file setting if you need non-ECC Diffie-Hellman.
TCP wrapper support	Use Lua authentication database instead.
Weak password schemes	Weak password schemes are disabled by default; you need to use auth_allow_weak_schemes to enable them.

Cassandra Parameters

See Cassandra configuration.

Parameter	Notes
Cassandra ssl_verify=cert-dns setting	Removed, as it was deprecated by Cassandra cpp-driver due to it being insecure against MITM attacks.

Backends and Plugins

Backend	Notes
checkpassword auth database	Use Lua authentication database instead.
Dict passdb & userdb driver	Use Lua authentication database instead.
Dict quota; Dirsize quota	These drivers are removed. You should use Quota Driver: Count instead along with quota-clone plugin. Note that switching to quota count can cause all users' indexes to update, so reserve time for this.
imap-zlib plugin	The IMAP COMPRESS extension is now automatically enabled.
listescape plugin	Use mailbox_list_storage_escape_char instead.
mailbox-alias plugin	Depending on the use case, replacement may be the mailbox_special_use mailbox setting and/or Sieve filters.
Memcached dict driver	Use Redis instead.
old-stats plugin	Use Statistics instead. auth_stats setting has been removed too.
shadow auth driver	Use PAM authentication database instead.
XZ Compression	You need to perform migration using a different compression format. With Maildir, you can try uncompressing all your mail and compressing them with another algorithm while Dovecot is not running.
zlib plugin	Use mail-compress plugin with the mail_compress_write_method setting instead.

Settings

Setting	Notes
auth_stats
dict_db_config	Berkeley DB is not supported anymore.
imap_id_log	Replaced by the imap_id_received event.
login_access_sockets	Use Lua authentication database instead. Dovecot will fail to start if this setting is present in configuration.
quota_set
sieve_dir	See Script storage.
sieve_global_dir	See Script storage.
sieve_global_path	See Script storage.
sieve_vacation_max_subject_codepoints

Changed Features

Settings

Setting	Notes
ssl	Connections from login_trusted_networks are now also required to be SSL/TLS encrypted with the setting ssl=required.
ssl_min_protocol	The SSLv3 option was removed, as it is no longer secure.

Converted Settings

Setting	Notes
auth_debug	Use log_debug filter instead. Example: log_debug=category=auth.
auth_default_realm	Replaced by auth_default_domain.
auth_worker_max_count	Use service-specific process limit.
disable_plaintext_auth	Replaced by auth_allow_cleartext.
push_notification_backend	Use push_notification_driver instead.
auth_policy_server_timeout_msecs	See: Auth Policy Configuration.
mail_crypt_require_encrypted_user_key	Replaced by crypt_user_key_require_encrypted, crypt_user_key_password and crypt_user_key_encryption_key.
verbose_ssl	Use log_debug = category=ssl instead.
mail_location setting & mail userdb field	Split into multiple mail_* settings.
fts	Replaced by fts named filter.
fts_autoindex_exclude	Replaced by boolean property fts_autoindex, nested inside mailbox blocks. Note that the values are inverted as compared to those in the old fts_autoindex_exclude.
fts_decoder	Replaced by fts_decoder_driver and fts_decoder_script_socket_path.
fts_solr	Replaced by fts_solr_url, fts_solr_batch_size, fts_solr_soft_commit, and http_client_rawlog_dir settings.
fts_tika	Replaced by fts_decoder_driver and fts_decoder_tika_url.
fts_language_config	Renamed to textcat_config_path.
fts_languages	Converted into language blocks.
fts_filters	Split into language_filters, language_filter_normalizer_icu_id, language_filter_stopwords_dir.
fts_tokenizers	Split into language_tokenizers, language_tokenizer_address_token_maxlen, language_tokenizer_generic_algorithm, language_tokenizer_generic_token_maxlen, language_tokenizer_generic_wb5a, language_tokenizer_kuromoji_icu_id, language_tokenizer_kuromoji_split_compounds, language_tokenizer_kuromoji_token_maxlen.
service_count	Renamed to service_restart_request_count. The default value is set to unlimited. Value 0 is now a configuration error.
passdb/userdb :protected	Renamed to :default
passdb { default_fields, override_fields }	Replaced by passdb_fields
userdb { default_fields, override_fields }	Replaced by userdb_fields
quota, quota_rule	Split into separate quota settings.
quota_grace	Renamed to quota_storage_grace.
quota_over_flag	Renamed to quota_over_status_current.
quota_over_flag_lazy_check	Renamed to quota_over_status_lazy_check.
quota_over_flag_value	Renamed to quota_over_status_mask.
quota_over_script	Replaced by quota_over_status named filter with execute.
quota_max_mail_size	Renamed to quota_mail_size. The default value is set to unlimited.
fts_index_timeout	Renamed to fts_search_timeout. The default value is set to 30 secs.

dict Proxy Parameters

See Dictionary Proxy Process.

Parameter	Notes
idle_msecs	Use dictionary idle timeout setting setting instead.
warn_slow_msecs	Use dictionary slow warning setting setting instead.

Default Settings

Setting	Old Default	New Default	Notes
imapc_features	Features "delay-login", "search", "fetch-headers", "fetch-bodystructure", "fetch-size" by default. Enable "acl" and "modseq" by default, if the remote server supports it.
mail_cache_max_headers_count	unlimited	100	New feature, explicitly set to 0 for the old behavior.
mail_cache_max_header_name_length	unlimited	100	New feature, explicitly set to 0 for the old behavior.
mail_log_prefix	%s(%u)<%{pid}><%{session}>:	%{protocol}(%{user})<%{process:pid}><%{session}>:	New variable expansion syntax.
mailbox_list_drop_noselect	no	yes	\NoSelect folders are now dropped by default.
service/anvil/chroot	empty	<no value>	Anvil is no longer chrooted.
service/anvil/user	$default_internal_user	<no value>	Anvil runs as root.
service/auth-worker/process_limit	1	30
protocols	imap pop3 lmtp	``	No protocols are enabled by default.

fs-crypt

Mail Crypt Plugin: fs-crypt now requires encryption keys by default.

Changed types for FTS header settings

Setting Name	Update
fts_header_excludes	Changed to Boolean List.
fts_header_includes	Changed to Boolean List.

Service defaults

LMTP

Default LMTP proxy destination port is now 24.

Event Changes

Core Events

Event	Change
fts_flatcurve_expunge	This event was added.
fts_flatcurve_index	This event was added.
fts_flatcurve_last_uid	This event was added.
fts_flatcurve_optimize	This event was added.
fts_flatcurve_query	This event was added.
fts_flatcurve_rescan	This event was added.
fts_flatcurve_rotate	This event was added.
auth_client_cache_flush_started	Event was removed.
auth_client_cache_flush_finished	Event was removed.
imap_id_received	Event was added.
login_aborted	Event was added.
mail_metadata_accessed	Event was added.
pop3_command_finished	Event was added.

Event Fields

Event	Field	Change
dns_worker_request_finished	cached	Field was added.
Mail user events	service	Field was added.
proxy_session_finished	error_code	Field was added.
proxy_session_finished	idle_usecs	Field was changed from idle_secs.
smtp_server_transaction_rcpt_finished	dest_host	Field was added.
smtp_server_transaction_rcpt_finished	dest_ip	Field was added.
sql_query_finished	consistency	Field was added.
sql_query_finished	error_consistency	Field was added.
Various	net_bytes_in	Field was changed from bytes_in.
Various	net_bytes_out	Field was changed from bytes_out.
Various	transport	transport=trusted was changed to transport=secured. See also secured connections.

Event Filters

Size units are allowed when specifying event filter values. For example, 100kb is accepted as equivalent to 102400.

Interval units are allowed when specifying event filter values. For example, 1min is accepted as equivalent to 60000000.

Event filters support escaping wildcard * and ? characters by prefixing them with \.

Exports

Events can now be exported to a local file or a unix socket. See Event Export: Drivers.

ACLs

Use ACL settings instead of Global ACL Directories or Global ACL File

With the following starting configuration:

dovecot.conf

namespace {
  prefix = INBOX/
  separator = /
}

plugin {
  acl = vfile:/etc/dovecot/acls/
}

/etc/dovecot/acls/INBOX

owner lrwstipekxa
anyone lr
user=kim l

/etc/dovecot/acls/INBOX/foo/.DEFAULT

user=timo lr
user=kim lrw

/etc/dovecot/acls/INBOX/foo/bar

user=kim lrw

You have to create the new ACLs to dovecot.conf:

namespace inbox {
  # previously from /etc/dovecot/acls/INBOX
  mailbox INBOX {
    acl owner {
      rights = lrwstipekxa
    }
    acl anyone {
      rights = lr
    }
    acl user=kim {
      rights = l
    }
  }
  # previously from /etc/dovecot/acls/foo/.DEFAULT
  mailbox INBOX/foo {
    acl user=timo {
      rights = lr
    }
    acl user=kim {
      rights = lrw
    }
  }
  # previously from /etc/dovecot/acls/foo/bar
  mailbox INBOX/foo/bar {
    acl user=kim {
      rights = lrw
    }
  }

Note that at this point you could simplify specific rules, e.g. use mailbox name wildcards to replace lines for a specific user:

mailbox INBOX/* {
  acl user=kim {
    rights = lrw
  }
}

And re-configure the ACL plugin:

acl_driver = vfile
acl_globals_only = yes

Afterwards you can remove the old global ACL directory parent:

rm -rf /etc/dovecot/acls/

Doveadm

doveadm batch

The doveadm batch command was removed.

doveadm fs

doveadm fs put can now put metadata also.

doveadm indexer

Added doveadm indexer command.

doveadm save

Added -r received-date parameter. See doveadm save.

dsync

The dsync command symlink was removed. Use doveadm sync or doveadm backup commands directly instead.

Mailbox Commands

USER environment variable can be used only with --no-userdb-lookup parameter.

All mail commands require providing -u, -F, -A parameter or --no-userdb-lookup parameter. This will always be subject to user database lookup and requires access to auth userdb socket, unless --no-userdb-lookup was used.

Doveadm HTTP API

Boolean Request Values

The doveadm HTTP API now requires valid boolean values. Providing invalid boolean values will result in a 400 response.

Lua Authentication

Lua passdb/userdb now passes all args key/values to an initialization function. See Lua authentication initialization.