Fail2Ban: Changelog

ver. 1.0.1 (2022/09/27) - energy-equals-mass-times-the-speed-of-light-squared

Compatibility

• the minimum supported python version is now 2.7, if you have previous python version you can use the 0.11 version of fail2ban or upgrade python (or even build it from source).
• potential incompatibility by parsing of options of backend, filter and action parameters (if they are partially incorrect), because fail2ban could throw an error now (doesn't silently bypass it anymore).
• due to fix for CVE-2021-32749 (GHSA-m985-3f3v-cwmm) the mailing action using mailutils may require extra configuration, if it is not compatible or doesn't support -E 'set escape' (e. g. with mailcmd parameter), see gh-3059
• automatic invocation of 2to3 is removed in setup now (gh-3098), there is also no option --disable-2to3 anymore, ./fail2ban-2to3 should be called outside before setup
• to v.0.11:
• due to change of actioncheck behavior (gh-488), some actions can be incompatible as regards the invariant check, if actionban or actionunban would not throw an error (exit code different from 0) in case of unsane environment.
• actions that have used tag <ip> (instead of <fid> or <F-ID>) to get failure-ID may become incompatible, if filter uses IP-related tags (like <ADDR> or <HOST>) additionally to <F-ID> and the values are different (gh-3217)

Fixes

• theoretical RCE vulnerability in mailing action using mailutils (mail-whois), CVE-2021-32749, GHSA-m985-3f3v-cwmm
• readline fixed to consider interim new-line character as part of code point in multi-byte logs (e. g. unicode encoding like utf-16be, utf-16le);
• [stability] solves race condition with uncontrolled growth of failure list (jail with too many matches, that did not cause ban), behavior changed to ban ASAP, gh-2945
• fixes search for the best datepattern - e. g. if line is too short, boundaries check for previously known unprecise pattern may fail on incomplete lines (logging break-off, no flush, etc), gh-3020
• [stability, performance] backend systemd:
• fixes error "local variable 'line' referenced before assignment", introduced in 55d7d9e2, gh-3097
• don't update database too often (every 10 ticks or ~ 10 seconds in production)
• fixes wrong time point of "in operation" mode, gh-2882
• better avoidance of landing in dead space by seeks over journals (improved seek to time)
• fixes missing space in message (tag <matches>) between timestamp and host if the message read from systemd journal, gh-3293
• [stability] backend pyinotify: fixes sporadic runtime error "dictionary changed size during iteration"
• several backends optimizations (in file and journal filters):
• don't need to wait if we still had log-entries from last iteration (which got interrupted for servicing)
• rewritten update log/journal position, it is more stable and faster now (fewer DB access and surely up-to-date at end)
• paths-debian.conf:
• add debian path to roundcube error logs
• action.d/firewallcmd-*.conf (multiport only): fixed port range selector, replacing : with -;" reverted the incompatibility gh-3047 introduced in a038fd5, gh-2821, because this depends now on firewalld backend (e. g. - vs. : related to iptables vs. nftables)
• action.d/nginx-block-map.conf: reload nginx only if it is running (also avoid error in nginx-errorlog, gh-2949)
• action.d/ufw.conf:
• fixed handling on IPv6 (using prepend, gh-2331, gh-3018)
• application names containing spaces can be used now (gh-656, gh-1532, gh-3018)
• filter.d/apache-fakegooglebot.conf:
• better, more precise regex and datepattern (closes possible weakness like gh-3013)
• filter.d/ignorecommands/apache-fakegooglebot - added timeout parameter (default 55 seconds), avoid fail with timeout (default 1 minute) by reverse lookup on some slow DNS services (googlebots must be resolved fast), gh-2951
• filter.d/apache-overflows.conf - extended to match AH00126 error (Invalid URI ...), gh-2908
• filter.d/asterisk.conf - add transport to asterisk RE: call rejection messages can have the transport prefixed to the IP address, gh-2913
• filter.d/courier-auth.conf:
• consider optional port after IP, gh-3211
• regex is rewritten without catch-all's and right anchor, so it is more stable against further modifications now
• filter.d/dovecot.conf:
• adjusted for updated dovecot log format with read(size=...) in message (gh-3210)
• parse everything in parenthesis by auth-worker info, e. g. can match (pid=...,uid=...) too (amend to gh-2553)
• extended to match prefix like conn unix:auth-worker (uid=143): auth-worker<13247>: (authenticate from external service like exim), gh-2553
• fixed "Authentication failure" regex, matches "Password mismatch" in title case (gh-2880)
• filter.d/drupal-auth.conf - more strict regex, extended to match "Login attempt failed from" (gh-2742)
• filter.d/exim-common.conf - pid-prefix extended to match mx1 exim[...]: (gh-2553)
• filter.d/lighttpd-auth.conf - adjusted to the current source code + avoiding catch-all's, etc (gh-3116)
• filter.d/named-refused.conf:
• added support for alternate names (suffix), FreeIPA renames the BIND9 named daemon to named-pkcs11, gh-2636
• fixes prefix for messages from systemd journal (no mandatory space ahead, because don't have timestamp), gh-2899
• filter.d/nginx-*.conf - added journalmatch to nginx filters, gh-2935
• filter.d/nsd.conf - support for current log format, gh-2965
• filter.d/postfix.conf: fixes and new vectors, review and combining several regex to single RE:
  • mode ddos (and aggressive) extended:
  • to consider abusive handling of clients hitting command limit, gh-3040
  • to handle postscreen's PREGREET and HANGUP messages, gh-2898
  • matches rejects with "undeliverable address" (sender/recipient verification) additionally to "Unknown user", gh-3039 both are configurable now via extended parameter and can be disabled using exre-user= supplied in filter parameters
  • reject: BDAT/DATA from, gh-2927
  • (since regex is more precise now) token selector changed to [A-Z]{4}, e. g. no matter what a command is supplied now (RCPT, EHLO, VRFY, DATA, BDAT or something else)
  • matches "Command rejected" and "Data command rejected" now
  • matches RCPT from unknown, 504 5.5.2, need fully-qualified hostname, gh-2995
  • matches 550 5.7.25 Client host rejected, gh-2996
• filter.d/sendmail-auth.conf:
• detect several "authentication failure" messages, sendmail 8.16.1, gh-2757
• detect user not found, gh-3030
• detect failures without user part, gh-3324
• filter.d/sendmail-reject.conf:
• fix reverse DNS for ... (gh-3012)
• fixed regex to consider "Connection rate limit exceeded" with different combination of arguments
• filter.d/sshd.conf:
• mode ddos extended - recognizes messages "kex_exchange_identification: Connection closed / reset by pear", gh-3086 (fixed possible regression of f77398c)
• mode ddos extended - recognizes new message "banner exchange: invalid format" generated by port scanner (https payload on ssh port), gh-3169
• filter.d/zoneminder.conf - support new log format (ERR instead of WAR), add detection of non-existent user login attempts, gh-2984
• amend to gh-980 fixing several actions (correctly supporting new enhancements now)
• fixed typo by --dump-pretty option which did never work (only --dp was working)
• fixes start of fail2ban-client in docker: speedup daemonization process by huge open files limit, gh-3334
• provides details of failed regex compilation in the error message we throw in Regex-constructor (it's good to know what exactly is wrong)
• fixed failed update of database didn't signal with an error, gh-3352:
• client and server exit with error code by failure during start process (in foreground mode)
• added fallback to repair if database cannot be upgraded

New Features and Enhancements

• python 3.10 and 3.11 compatibility (and GHA-CI support)
• actioncheck behavior is changed now (gh-488), so invariant check as well as restore or repair of sane environment (in case of recognized unsane state) would only occur on action errors (e. g. if ban or unban operations are exiting with other code as 0)
• better recognition of log rotation, better performance by reopen: avoid unnecessary seek to begin of file (and hash calculation)
• file filter reads only complete lines (ended with new-line) now, so waits for end of line (for its completion)
• datedetector:
• token %Z must recognize zone abbreviation Z (GMT/UTC) also (similar to %z)
• token %Z recognizes all known zone abbreviation besides Z, GMT, UTC correctly, if it is matching (%z remains unchanged for backwards-compatibility, see comment in code)
• date patterns %ExY and %Exy accept every year from 19xx up to current century (+3 years) in fail2ban-regex
• better grouping algorithm for resulting century RE for %ExY and %Exy
• actions differentiate tags <ip> and <fid> (<F-ID>), if IP-address deviates from ID then the value of <ip> is not equal <fid> anymore (gh-3217)
• action info extended with new members for jail info (usable as tags in command actions), gh-10:
• <jail.found>, <jail.found_total> - current and total found failures
• <jail.banned>, <jail.banned_total> - current and total bans
• filter.d/monitorix.conf - added new filter and jail for Monitorix, gh-2679
• filter.d/mssql-auth.conf - new filter and jail for Microsoft SQL Server, gh-2642
• filter.d/nginx-bad-request.conf - added filter to find bad requests (400), gh-2750
• filter.d/nginx-http-auth.conf - extended with parameter mode, so additionally to auth (or normal) mode fallback (or combined as aggressive) can find SSL errors while SSL handshaking, gh-2881
• filter.d/scanlogd.conf - new filter and jail, add support for filtering out detected port scans via scanlogd, gh-2950
• action.d/apprise.conf - added Apprise support (50+ Notifications), gh-2565
• action.d/badips.* - removed actions, badips.com is no longer active, gh-2889
• action.d/cloudflare.conf - better IPv6 capability, gh-2891
• action.d/cloudflare-token.conf - added support for Cloudflare Token APIs. This method is more restrictive and therefore safter than using API Keys.
• action.d/ipthreat.conf - new action for IPThreat integration, gh-3349
• action.d/ufw.conf (gh-3018):
• new option add (default prepend), can be supplied as insert 1 for ufw versions before v.0.36 (gh-2331, gh-3018)
• new options kill-mode and kill to drop established connections of intruder (see action for details, gh-3018)
• iptables and iptables-ipset actions extended to support multiple protocols with single action for multiport or oneport type (back-ported from nftables action);
• iptables actions are more breakdown-safe: start wouldn't fail if chain or rule already exists (e. g. created by previous instance and doesn't get purged properly); ultimately closes gh-980
• ipset actions are more breakdown-safe: start wouldn't fail if set with this name already exists (e. g. created by previous instance and don't deleted properly)
• replace internals of several iptables and iptables-ipset actions using internals of iptables include:
• better check mechanism (using -C, option --check is available long time);
• additionally iptables-ipset is a common action for iptables-ipset-proto6-* now (which become obsolete now);
• many features of different iptables actions are combinable as single chain/rule (can be supplied to action as parameters);
• iptables is a replacement for iptables-common now, several actions using this as include now become obsolete;
• new logtarget SYSTEMD-JOURNAL, gh-1403
• fail2ban.conf: new fail2ban configuration option allowipv6 (default auto), can be used to allow or disallow IPv6 interface in fail2ban immediately by start (e. g. if fail2ban starts before network interfaces), gh-2804
• invalidate IP/DNS caches by reload, so inter alia would allow to recognize IPv6IsAllowed immediately, previously retarded up to cache max-time (5m), gh-2804
• OpenRC (Gentoo, mainly) service script improvements, gh-2182
• suppress unneeded info "Jail is not a JournalFilter instance" (moved to debug level), gh-3186
• implements new interpolation variable %(fail2ban_confpath)s (automatically substituted from config-reader path, default /etc/fail2ban or /usr/local/etc/fail2ban depending on distribution); ignorecommands_dir is unneeded anymore, thus removed from paths-common.conf, fixes gh-3005
• fail2ban-regex: accepts filter parameters containing new-line