Exim The Mail Transfer Agent ,Title.10724 Page 1 Tuesday, October 9, 2001 9:25 AM ,Title.10724 Page 2 Tuesday, October 9, 2001 9:25 AM Exim The Mail Transfer Agent Philip Hazel Beijing • Cambridge • Farnham • Köln • Paris • Sebastopol • Taipei • Tokyo ,Title.10724 Page 3 Tuesday, October 9, 2001 9:25 AM Exim: The Mail Transfer Agent by Philip Hazel Copyright © 2001 O’Reilly & Associates, Inc. All rights reserved. Printed in the United States of America. Published by O’Reilly & Associates, Inc., 101 Morris Street, Sebastopol, CA 95472. Editor: Andy Oram Production Editor: Mary Brady Cover Designer: Ellie Volckhausen Printing History: June 2001: First Edition. Nutshell Handbook, the Nutshell Handbook logo, and the O’Reilly logo are registered trademarks of O’Reilly & Associates, Inc. Many of the designations used by manufacturers and sellers to distinguish their products are claimed as trademarks. Where those designations appear in this book, and O’Reilly & Associates, Inc. was aware of a trademark claim, the designations have been printed in caps or initial caps. The association between the image of an aye-aye and Exim is a trademark of O’Reilly & Associates, Inc. While every precaution has been taken in the preparation of this book, the publisher assumes no responsibility for errors or omissions, or for damages resulting from the use of the information contained herein. Library of Congress Cataloging-in-Publication Data Hazel, Philip Exim: the mail transfer agent/by Philip Hazel p.cm. ISBN 0-596-00098-7 1. Exim (Computer program) 2. Email Computer programs I. Title TK5105.73 .H39 2001 004.692 dc21 2001036079 [DS] ,Copyright.10561 Page 1 Tuesday, October 9, 2001 9:25 AM Ta ble of Contents Preface xiii 1. Introduction 1 2. How Inter net Mail Works 5 Dif ferent Types of MTA 10 Inter net Message Standards 11 RFC 822 Message Format 11 The Message ‘‘On the Wir e’’ 13 Summary of the SMTP Protocol 15 Forgery 18 Authentication and Encryption 18 Routing a Message 18 Checking Incoming Mail 19 Overview of the DNS 21 DNS Records Used for Mail Routing 24 Related DNS Records 25 Common DNS Errors 27 Role of the Postmaster 29 3. Exim Over view 30 Exim Philosophy 30 Exim’s Queue 31 Receiving and Delivering Messages 31 Exim Processes 32 v 9 October 2001 09:13 vi Table of Contents Coordination Between Processes 32 How Exim Is Configured 33 How Exim Delivers Messages 35 Local and Remote Addresses 37 Pr ocessing an Address 38 A Simple Example 40 Complications While Directing and Routing 46 Complications During Delivery 48 Complications After Delivery 49 Use of Transports by Directors and Routers 49 4. Exim Operations Over view 52 How Exim Identifies Messages 52 Watching Exim at Work 53 The Runtime Configuration File 54 The Default Qualification Domain 61 Handling Frozen Bounce Messages 62 Reducing Activity at High Load 62 Limiting Message Sizes 65 Parallel Remote Delivery 65 Contr olling the Number of Delivery Processes 66 Large Message Queues 66 Large Installations 67 5. Extending the Deliver y Configuration 71 Multiple Local Domains 71 Virtual Domains 74 Mailing Lists 78 Using an External Local Delivery Agent 85 Multiple User Addresses 87 Mixed Local/Remote Domains 88 Delivering to UUCP 90 Ignoring the Local Part in Local Deliveries 91 Handling Local Parts in a Case-Sensitive Manner 93 Scanning Messages for Viruses 94 Modifying Message Bodies 99 9 October 2001 09:13 Ta ble of Contents vii 6. Options Common to Director s and Routers 101 Conditional Running of Routers and Directors 102 Changing a Driver’s Successful Outcome 107 Adding Data for Use by Transports 108 Debugging Directors and Routers 113 Summary of Director/Router Generic Options 114 7. The Director s 118 Conditional Running of Directors 119 Optimizing Single-Level Aliasing 120 Adding Data for Use by Transports 121 The aliasfile and forwardfile Directors 121 The aliasfile Director 133 The forwardfile Director 138 The localuser Director 146 The smartuser Director 147 8. The Routers 150 Timeouts While Routing 150 Domains That Route to the Local Host 151 The lookuphost Router 154 The domainlist Router 158 The ipliteral Router 169 The queryprogram Router 169 9. The Transpor ts 173 Options Common to All Transports 174 The smtp Transport 184 Envir onment for Local Transports 194 Options Common to the appendfile and pipe Transports 196 The appendfile Transport 203 The pipe Transport 222 The lmtp Transport 231 The autoreply Transport 232 10. Message Filter ing 238 Examples of Filter Commands 239 Filtering Compared with an External Delivery Agent 241 Setting Up a User Filter 242 Setting Up a System Filter 242 Testing Filter Files 244 9 October 2001 09:13 viii Table of Contents For mat of Filter Files 246 Significant Actions 248 Filter Commands 249 The add Command 249 Delivery Commands 250 Mail Commands 253 Logging Commands 256 The testprint Command 256 The finish Command 257 Obeying Filter Commands Conditionally 257 Additional Features for System Filters 262 11. Shared Data and Exim Processes 265 Message Files 266 Locking Message Files 268 Hints Files 269 Log Files 271 User and Group IDs for Exim Processes 271 Pr ocess Relationships 272 The Daemon Process 273 Reception Processes 277 Queue Runner Processes 279 Delivery Processes 281 Summary of Message Handling Process Types 283 Other Types of Process 283 12. Deliver y Er ror s and Retrying 284 Retrying After Errors 284 Remote Delivery Errors 285 Local Delivery Errors 288 Routing and Directing Errors 289 Retry Rules 289 Computing Retry Times 292 Using Retry Times 293 Retry Rule Examples 294 Timeout of Retry Data 295 Long-Ter m Failur es 295 Ultimate Address Timeout 297 Inter mittently Connected Hosts 297 9 October 2001 09:13 Ta ble of Contents ix 13. Message Reception and Polic y Controls 302 Message Sources 303 Message Size Control 303 Messages from Local Processes 304 Unqualified Addresses from Remote Hosts 307 Checking a Remote Host 308 Checking Remote Sender Addresses 314 Checking Recipient Addresses 322 Checking Header Line Syntax 326 Relay Control 326 Customizing Prohibition Messages 332 Incoming Message Processing 333 14. Rewr iting Addresses 339 Automatic Rewriting 339 Configur ed Rewriting 340 Rewriting Rules 343 Rewriting Patterns 345 Rewriting Flags 347 A Further Rewriting Example 351 Testing Rewriting Rules 354 15. Authentication, Encryption, and Other SMTP Processing 355 SMTP Authentication 355 Encrypted SMTP Connections 367 SMTP over TCP/IP 372 Local SMTP 376 Batched SMTP 377 16. File and Database Lookups 378 Single-Key Lookup Types 379 Query-Style Lookup Types 382 Quoting Lookup Data 382 NIS+ 383 LDAP 384 MySQL and PostgreSQL 386 DNS Lookups 388 Implicit Keys in Query-Style Lookups 388 Temporary Errors in Lookups 389 Default Values in Single-Key Lookups 389 9 October 2001 09:13 x Table of Contents Partial Matching in Single-Key Lookups 390 Lookup Caching 391 17. String Expansion 392 Variable Substitution 394 Header Insertion 394 Operations on Substrings 395 Character Translation 398 Text Substitution 399 Conditional Expansion 399 Lookups in Expansion Strings 406 Extracting Fields from Substrings 410 IP Address Masking 412 Quoting 413 Reexpansion 416 Running Embedded Perl 417 Testing String Expansions 418 18. Domain, Host, and Address Lists 420 Negative Items in Lists 421 List Items in Files 422 Lookup Items in Lists 423 Domain Lists 423 Host Lists 426 Addr ess Lists 432 19. Miscellany 435 Security Issues 435 Privileged Users 442 RFC Conformance 444 Timestamps 449 Checking Spool Space 450 Contr ol of DNS Lookups 451 Bounce Message Handling 451 Miscellaneous Controls 456 20. Command-Line Interface to Exim 458 Input Mode Control 459 Additional Message Data 462 Immediate Delivery Control 464 Err or Routing 465 9 October 2001 09:13 [...]... receive mail (often just called ‘‘mailers’’) are formally called mail user agents (MUAs) They are concerned with providing a convenient mail interface for users They display incoming mail that is in users’ mailboxes, assist the user in constructing messages for sending, and provide facilities for managing folders of saved messages They are the ‘‘front end’’ of the mail system Many different user agents... computer, thereby providing a choice of different user interfaces However, when an MUA sends a message, it does not take on the work of actually delivering it to the recipients Instead, it sends it to a mail transfer agent (MTA), which may be running on the same host or on some local server Mail transfer agents do the job of transferring messages from one host to another, and, after they reach their destination... University were running a variety of mail transfer agents, including Sendmail, Smail 3, and PP Some years before, I had converted the systems whose mail I managed from Sendmail to Smail to make it easier to handle the special requirements of the early 1990s in UK academic networking during the transition from a private X.25-based network to the Internet By 1995, the transition was complete, and it was... references to these important new features have therefore been included in the book, which now covers all the major features of the 3.2x releases No further functional enhancements to Exim 3 are planned, though in due course a new major release (Exim 4) is expected The Exim reference manual and a FAQ are online at the Exim web site, at http://www .exim. org and its mirrors Here you will also find the latest... ‘‘spool’’ used for the disk area in which users’ mailboxes are kept, but this is not the sense in which it is used here Messages that are destined for other hosts are transmitted over the Internet to other MTAs using the Simple Mail Transfer Protocol (SMTP) When the originating host and the final host are both directly connected to the Internet, the message can be delivered directly to the final host, but... often arrange for all their incoming mail to be routed via a central mail hub, which then delivers it to other hosts within the organization’s local network These may be behind a firewall and therefore inaccessible to the Internet at large When a message reaches its destination host, the * RFCs are the documents that lay down the standards by which the Internet operates You can find them online at http://www.ietf.org... For personal clients, the ISP normally provides a mailbox for each account, from which the mail is collected by some means when the client connects As far as the MTA is concerned, it is doing a local delivery into a mailbox on the ISP’s server • For corporate clients, ISPs are more likely to transfer mail to the clients’ MTAs based purely on the domains in the addresses, with the ISP’s MTA acting as... and RCPT The envelope contains the sender address and one or more recipient addresses These addresses are of the form without the additional textual information, such as the user’s full name, that may appear in message header lines The deliveries done by the receiving MTA (either to local mailboxes or by passing the message on to other hosts) are based on the recipients listed in the envelope,... gateway, the more general term is used here, as it is in the Exim reference documentation The Message ‘‘On the Wire’’ A message that is transmitted between MTAs has several things added to it over and above what the composing user sees In addition to the header section and the body, another piece of data called the envelope is transmitted immediately before the RFC 822 data, using the SMTP commands MAIL. .. MUA need not be running on the same host as its MTA; Figure 2-3 illustrates the relationship between MUAs and MTAs in two common configurations In the top part of the figure, the MUA, MTA, and the disk storage are all part of a single system, indicated by the dashed line The users access the system by logging on and authenticating themselves by a password or some other means The MUA is started by a user . Exim The Mail Transfer Agent ,Title.10724 Page 1 Tuesday, October 9, 2001 9:25 AM ,Title.10724 Page 2 Tuesday, October 9, 2001 9:25 AM Exim The Mail Transfer Agent Philip Hazel Beijing • . MTAs, Exim has adopted the Sendmail command interface so that it can be a straight replacement for /usr/sbin/sendmail or /usr/lib/sendmail. All the rele- vant Sendmail options are implemented. There. Incoming Mail 19 Overview of the DNS 21 DNS Records Used for Mail Routing 24 Related DNS Records 25 Common DNS Errors 27 Role of the Postmaster 29 3. Exim Over view 30 Exim Philosophy 30 Exim s