Configuring and Debugging Exim

Stephen Bee

Tuesday, October 27, 2009

(I <3) Exims Guts

Tuesday, October 27, 2009

The Configuration File

Where is it? /etc/exim.conf on Linux systems /usr/local/etc/exim/configure on FreeBSD systems exim -bV | grep Config (if all else fails)

Tuesday, October 27, 2009

The Configuration File

Whats in it?
Partitioned into seven different sections:

The main section Routers ACL definitions Transports Retry Rules Authenticators Rewrite Rules
Note: Other than the main section, all sections are headed with begin section_name

Tuesday, October 27, 2009

The main section

Contains global settings and variables Always located at the top of exim.conf
smtp_receive_timeout = 165s daemon_smtp_ports = 25 : 465 ignore_bounce_errors_after = 3d system_filter = /etc/cpanel_exim_system_filter

Documentation for all available settings can be found at: http://exim.org/exim-html-current/doc/html/spec_html/ch14.html

Tuesday, October 27, 2009

List variables
Contain domains, hosts, addesses, or local parts Colon separated, type based
Static Lists:
hostlist bad_hosts = 192.168.99.123 : 192.168.87.243 domainlist trusted_domains = foo.example : bar.example addresslist spammers = foo@bar.example : bar@example.com localpartlist sysusers = foo : bar : root

Dynamic Lists:
hostlist trustedmailhosts = lsearch;/etc/trustedmailhosts domainlist local_domains = lsearch;/etc/localdomains
Comprehensive documentation on list variables can be found at: http://exim.org/exim-html-current/doc/html/spec_html/ch10.html
Tuesday, October 27, 2009

Routers and Directors

What are they?

The decision makers for how a message is handled Routers result in message delivery, directors do not
Router
remote_delivery: driver = dnslookup domains = ! +local_domains transport = remote_smtp

Director
fail_remote_domains: driver = redirect domains = ! +local_domains : ! localhost : ! localhost.localdomain allow_fail data = "remote deliveries are not permitted from this server"
Tuesday, October 27, 2009

Transports
What are they? The executioners of the actual message delivery
remote_smtp: driver = smtp interface = 1.2.3.4 local_delivery: driver = appendfile file = /home/foo/mail/foo.example/joe/inbox

Tuesday, October 27, 2009

Example: Smart Hosts

Objective
Route messages for a specific list of domains to a third party mail server.

Tuesday, October 27, 2009

Example: Smart Hosts

How its done Create the domain list file
touch /etc/smartdomains chown root:mail /etc/smartdomains chmod 0750 /etc/smartdomains echo foo.example > /etc/smartdomains

Add a named domainlist for that list file

domainlist smart_domains = lsearch;/etc/smartdomains

Create a manualroute router (after democheck)

router smarthost: driver = manualroute transport = remote_smtp route_list = +smart_domains 192.168.99.232
Tuesday, October 27, 2009

Example: Smart Hosts

Making it more flexible. Use a colon separated domain to host mapping
root@gibson [~]# cat /etc/smartdomains foo.example: 192.168.99.232 bar.example: 192.168.99.254 root@gibson [~]#

Amend the router to use a key based lookup

smarthost: driver = manualroute transport = remote_smtp route_data = ${lookup{$domain}lsearch{/etc/smartdomains}}

Tuesday, October 27, 2009

ACLs
Used for validation, scanning, whitelisting, etc. Only called during the SMTP reception process Conditions consist of the following - An action (accept, deny, drop, defer) - Criteria that if evaluated true, triggers the action
acl_connect: accept hosts = +trustedmailhosts deny
Comprehensive documentation on ACLs can be found at: http://exim.org/exim-html-current/doc/html/spec_html/ch40.html
Tuesday, October 27, 2009

Defining the ACL

ACLs are executed based on the ACL selector
theyve been assigned.
acl_smtp_connect = acl_connect acl_smtp_data = check_message acl_smtp_mail = acl_mail acl_smtp_notquit = acl_notquit acl_smtp_rcpt = check_recipient

All of the available ACL selection options are documented at: http://exim.org/exim-html-current/doc/html/spec_html/ch40.html#SECID189
Tuesday, October 27, 2009

Whitelisting domains for spam

Objective
Disable only spamassassin scans for a list of host addresses.

Tuesday, October 27, 2009

Whitelisting domains for spam

How its done
Create the domain list file
touch /etc/spamfreehosts chown root:mail /etc/spamfreehosts chmod 0750 /etc/spamfreehosts echo 1.2.3.4 > /etc/spamfreehosts

Add this near the top of the check_message ACL

accept hosts = net-iplsearch;/etc/spamfreehosts

Tuesday, October 27, 2009

ALL? No, ALL!

Objective
Deny incoming connections from all hosts, except for our third party spam filtering service.

Tuesday, October 27, 2009

ALL? No, ALL!

How its done Add the following to the top of the acl_connect ACL
accept hosts = : deny !hosts = @[] : net-iplsearch;/etc/trustedmailhosts message = This server does not handle mail directly

Tuesday, October 27, 2009

Half-time Q&A

Tuesday, October 27, 2009

Log Files

Tuesday, October 27, 2009

Exims Log Files

/var/log/exim_mainlog
Logs message arrival and delivery attempts Logs delivery rejections based on policy (e.g. ACL) Prints a fresh copy of exim.conf to standard output

/var/log/exim_rejectlog /var/log/exim_paniclog

Tip: exim -bP log_file_path will display log file paths

Tuesday, October 27, 2009

Main log formatting

Message Status Indicators

=> indicates message arrival <= indicates successful message delivery == indicates message delivery has been deferred ** indicates that a delivery failure has occurred
Successful Message Delivery
2009-09-30 12:23:40 1Mt2tw-0003vE-Ea <= stephen@cpanel.net H=(cpanel.net) [127.0.0.1] P=esmtpa A=fixed_login:stephen@cpanel.net S=745 id=0373931685581ab29f56199c78755f1a.squirrel@techdump.net 2009-09-30 12:23:42 1Mt2tw-0003vE-Ea => stephenbee@gmail.com R=lookuphost T=remote_smtp H=gmail-smtp-in.l.google.com [209.85.211.67] 2009-09-30 12:23:42 1Mt2tw-0003vE-Ea Completed 2009-10-03 23:47:33 1MuEK5-0008S6-Io == stephen@cpanel.net R=smarthost T=remote_smtp defer (111): Connection refused 2009-09-30 18:33:00 1Mt8fH-0005xJ-Oe ** user@bar.example R=fail_remote_domains: The mail server could not deliver mail to user@bar.example. The account or domain may not exist, they may be blacklisted, or missing the proper dns entries.
Tuesday, October 27, 2009

Main log formatting

Router and Transport Information

R= indicates the assigned router T= indicates the assigned transport

2009-09-30 12:23:42 1Mt2tw-0003vE-Ea => stephenbee@gmail.com R=lookuphost T=remote_smtp H=gmail-smtp-in.l.google.com [209.85.211.67] 2009-10-03 23:47:33 1MuEK5-0008S6-Io == stephen@cpanel.net R=smarthost T=remote_smtp defer (111): Connection refused

Tuesday, October 27, 2009

The reject log

Logs only policy-based rejections Makes it easier to differentiate rejections
Examples
2009-10-03 13:39:53 H=source.host.example.com [10.0.0.1] F=<blacklisted@foo.example> rejected RCPT <user@bar.example>: "JunkMail rejected - source.host.example.com [10.0.0.1] is in an RBL, see http:// www.spamhaus.org/query/bl?ip=10.0.0.1" 2009-10-03 17:07:20 H=75-170-234-130.desm.qwest.net (wergvan) [75.170.234.130] rejected MAIL <>: Access denied - Invalid HELO name (See RFC2821 4.1.1.1) 2009-09-30 09:44:28 fixed_login authenticator failed for (example.com) [10.0.0.3]: 535 Incorrect authentication data (set_id=inna) 2009-09-30 09:44:29 SMTP call from (cracker.example) [10.0.0.4] dropped: too many nonmail commands (last was "AUTH")

Tuesday, October 27, 2009

Extracting log information

exigrep [-t<n>] [-I] [-l] [-v] <pattern> [<log file>]

Written specifically for searching exim log files Returns all entries for matching messages Takes input via STDIN, or by specifying a file name
root@foo [~]# exigrep foo@example.com /var/log/exim_mainlog 2009-09-30 12:38:12 1Mt37t-000405-4r <= foo@example.com H=(cpanel.net) [10.1.1.2] U=root P=esmtp S=1423 2009-09-30 12:38:12 1Mt37t-000405-4r => /home/foo/mail/ <bar@example.com> R=central_filter T=address_directory 2009-09-30 12:38:12 1Mt37t-000405-4r Completed

Tuesday, October 27, 2009

Testing Message Delivery

Tuesday, October 27, 2009

Launching an SMTP Session

exim -bh <HOST-IP>

Launches a fake SMTP session from the provided IP Provides a verbose amount of debugging output No DNS lookups or callouts will occur

root@gibson [~]# exim -bh 1.2.3.4 **** SMTP testing session as if from host 1.2.3.4 **** but without any ident (RFC 1413) callback. **** This is not for real! ...TRUNCATED... >>> check condition = ${if eq {$interface_port}{25}{no}{yes}} >>> = yes >>> accept: condition test succeeded 220-gibson.steve.cpanel.net ESMTP Exim 4.69 #1 Mon, 05 Oct 2009 10:22:59 -0400 220-We do not authorize the use of this system to transport unsolicited, 220 and/or bulk e-mail.

Tuesday, October 27, 2009

Launching an SMTP Session

exim -bs

Launches a full fledged local SMTP session

root@gibson [~]# exim -bs 220-gibson.steve.cpanel.net ESMTP Exim 4.69 #1 Mon, 05 Oct 2009 10:28:58 -0400 220-We do not authorize the use of this system to transport unsolicited, 220 and/or bulk e-mail. root@gibson [~]#

Useful option for -bs:

exim -oMa [host-ip] -bs Allows you to forge the messages host origin

root@gibson [~]# exim -oMa 1.2.3.4 -bs 220-gibson.steve.cpanel.net ESMTP Exim 4.69 #1 Mon, 05 Oct 2009 10:28:58 -0400
Tuesday, October 27, 2009

Testing Message Routing

exim -bt [email-address]

Determines how exim would route a given address

root@gibson [~]# exim -bt stephen@cpanel.net stephen@cpanel.net router = lookuphost, transport = remote_smtp host mx1.cpanel.net [208.74.121.68] MX=0 host mx3.cpanel.net [208.74.121.69] MX=5 host mx2.cpanel.net [208.74.123.60] MX=10 root@gibson [~]#

Tuesday, October 27, 2009

Queue Management

Tuesday, October 27, 2009

Querying The Queue

exim -bp prints all queued messages to STDOUT
root@gibson [~]# exim -bp 25m 2.9K 0t5C6f-0000c8-00 <sender@foo.example> joe@foo.example jim@bar.example ....... root@gibson [~]#

exim -bpc prints out total queued messages

root@gibson [~]# exim -bpc 88289129827 root@gibson [~]#

Tuesday, October 27, 2009

Summarizing The Queue

exiqsumm [-a] [-c]

Meant to be used in a pipe from exim -bp -a flag causes statistics to sort by message age -c flag causes statistics to sort by message count
root@mx1 [~]# exim -bp | exiqsumm Count Volume Oldest Newest Domain ----- ------ ------ ------ -----1 2252 9m 9m foobar.example.com 1 1843 19h 19h mail3.local.example 1 1331 19h 19h mx.example.com 122 266KB 69h 3h cpanel.net --------------------------------------------------------------125 266KB 69h 9m TOTAL

Tuesday, October 27, 2009

Queued Messages
Why would a message be queued?

Its frozen!

A non-permanent error occurred (e.g. host down) during message delivery, and delivery has been deferred. This occurs when the load average surpasses the value of queue_only_load in exim.conf

Exim is in queue only mode

Tuesday, October 27, 2009

exiqgrep
exiqgrep [frsyozq] [expression] Uses regular expressions to search the mail queue
root@gibson [~]# exiqgrep -f 'meow@kittens.com' 20m 355 1MufOA-0003EF-Nh <meow@kittens.com> user@foo.example root@gibson [~]#

Useful application of: exiqgrep -i -f user@foo.bar | xargs exim -Mrm Remove all messages with selected criteria

Tuesday, October 27, 2009

Processing Individual Messages

exim -M <MSG-ID> forces delivery of a message

Useful options for -M

-Mrm removes the specified message from queue -Mvl displays a log of all previous delivery attempts
2009-10-05 04:41:44 Received from meow@kittens.foo U=root P=local-esmtp S=355 2009-10-05 04:41:44 192.168.99.232 [192.168.99.232] Connection refused 2009-10-05 04:41:44 user@foo.example R=dumbhost T=remote_smtp defer (111): Connection refused

Tuesday, October 27, 2009

exinext
exinext <user@domain.com> determines next scheduled delivery attempt
root@gibson [~]# exinext user@foo.example Transport: 192.168.99.232 [192.168.99.232/NULL] error 111: Connection refused first failed: 05-Oct-2009 04:41:44 last tried: 05-Oct-2009 04:41:44 next try at: 05-Oct-2009 04:56:44 root@gibson [~]#

Tuesday, October 27, 2009

Processing the Queue

exim -q launches a queue runner process

Useful options for -q

-qi only processes initial delivery attempts -qf forces delivery of all non-frozen messages -qff forces delivery of all messages, frozen or not -qfl forces delivery of locally destined messages

Tuesday, October 27, 2009

Processing the Queue

exim -S [email-address] Processes queued messages matching the provided address. Partial e-mail addresses are permitted
root@toothpick [~]# exim -v -S @kittens.com LOG: queue_run MAIN Start queue run: pid=1494 -S @kittens.com delivering 1Mup8P-0000O2-2W (queue run pid 1494) .....

exim -R [email-address] Same as above, except based on the recipient(s) rather than the message sender.

Tuesday, October 27, 2009

The Debugger

Tuesday, October 27, 2009

Why use the debugger?

You get all of the gory details on whats going on
behind the scenes with Exim.

Can be used with any call to Exim

Tuesday, October 27, 2009

Enabling The Debugger

Passing -d to exim enables the debugger
root@gibson [~]# exim -d -bs Exim version 4.69 uid=0 gid=0 pid=15556 D=fbb95cfd .... TRUNCATED .... using ACL "acl_connect" processing "accept" accept: condition test succeeded SMTP>> 220-foo.example.com ESMTP Exim 4.69 #1 Wed, 30 Sep 2009 12:51:08 -0500 220-We do not authorize the use of this system to transport unsolicited, 220 and/or bulk e-mail. smtp_setup_msg entered

Use the script command to store output to file

root@gibson [~]# script -c 'exim -d -bs' debugger.output ^C Script done, file is debugger.output root@gibson [~]#

Tuesday, October 27, 2009

Isolating The Output

The -d flag accepts a chain of modifiers, which enable/disable debugging on certain components of the delivery process.

exim -d+all
Enables for all components of delivery process

Examples:

exim -d-all+router+transport
Enables only for router and transport logic

exim -d-all+verify
Enables only sender verification logic

Tuesday, October 27, 2009

Thats It! Q&A

Tuesday, October 27, 2009