1/ Introduction --------------- libEtPan! is mainly a library that will handle all kind of mailbox access. For example: IMAPrev4, POP3, NNTP, mbox, MH. You have two kinds of mailbox access, either using low-level functions with a different interface for each kind of access or using higher-level functions, using a driver to wrap the higher-level API. 2/ Low-level ------------ 2.1/ IMAP4rev1 - Internet Message Access Protocol - Version 4rev1 ----------------------------------------------------------------- Each command of the IMAP4rev1 Standard (RFC 2060) is implemented in the IMAP4rev1 module. Directory imap/. 2.1.1/ References - RFC 2060 - INTERNET MESSAGE ACCESS PROTOCOL - VERSION 4rev1 - draft-crispin-imapv-15.txt Not yet implemented : - draft-crispin-imapv-20.txt 2.1.2/ Dependencies - tools/ 2.1.3/ Files descriptions description of header files : mailimap.[ch] -- functions that implements each IMAP4rev1 command mailimap_helper.[ch] -- helper interface for the previous functions mailimap_types.[ch] -- definition of types and constructors for these types mailimap_types_helper.[ch] -- contains function definitions that will help to create data necessary to use IMAP4rev1 module mailimap_socket.[ch] -- provides a function to connect to an IMAP4rev1 server over TCP mailimap_ssl.[ch] -- provides a function to connect to an IMAP4rev1 server over TLS layer 2.1.4/ Interface Include for this module is mailimap.h and includes all other headers. The interface of IMAP4rev1 is documented in the following files : mailimap.h mailimap_types.h mailimap_types_helper.h 2.2/ POP3 - Post Office Protocol - Version 3 -------------------------------------------- Each command of the POP3 Standard (RFC 1939 and RFC 2449) is implemented in the POP3 module. Directory pop3/. 2.1.1/ References - RFC 1939 - Post Office Protocol - Version 3 - RFC 2449 - POP3 Extension Mechanism (CAPA) Not yet implemented : - RFC 1734 - POP3 AUTHentication command 2.1.2/ Dependencies - tools/ 2.2.3/ Files descriptions mailpop3.[ch] -- functions that implements each POP3 command mailpop3_helper.[ch] -- helper interface for the previous functions mailpop3_socket.[ch] -- provides a function to connect to a POP3 server over TCP mailpop3_ssl.[ch] -- provides a function to connect to a POP3 server over TLS layer 2.2.4/ Interface Include for this module is mailpop3.h and includes all other headers. There is not yet documentation for POP3 module. 2.3/ NNTP - Network News Transfer Protocol ------------------------------------------ Each command of the NNTP Standard (RFC 977 and RFC 2980) is implemented in the NNTP module. Directory nntp/. 2.3.1/ References - RFC 977 - Network News Transfer Protocol - RFC 2980 - Common NNTP Extensions Not yet implemented : - RFC 1036 - Standard for Interchange of USENET Messages - son of RFC 1036 : FTP://zoo.toronto.edu/pub/news.txt.Z 2.3.2/ Dependencies - tools/ 2.3.3/ Files descriptions newsnntp.[ch] -- functions that implements each NNTP command newsnntp_socket.[ch] -- provides a function to connect to a NNTP server over TCP newsnntp_ssl.[ch] -- provides a function to connect to a POP3 server over TLS layer 2.3.4/ Interface Include for this module is newsnntp.h and includes all other headers. There is not yet documentation for NNTP module. 2.4/ mbox --------- The mbox module provides a set of functions to manipulate mbox mailboxes. These functions make a safe lock on the mailbox they work with. This module will assign to each message a unique message identifier so that we can work with message numbers in mbox files without other programs interfer. Directory mbox/. 2.4.1/ References - http://wp.netscape.com/eng/mozilla/2.0/relnotes/demo/content-length.html - http://www.qmail.org/qmail-manual-html/man5/mbox.html 2.4.2/ Dependencies - tools/ - imf/ 2.5.3/ Specific to libEtPan! - "X-LibEtPan-UID" header 2.5.4/ Files descriptions mailmbox.[ch] -- functions to manipulate mbox mailboxes. mailmbox_parse.[ch] -- this module is in charge of parsing the mbox file content mailmbox_types.[ch] -- definition of types and constructors for these types 2.4.5/ Interface Include for this module is mailmbox.h and includes all other headers. There is not yet documentation for mbox module. 2.5/ MH ------- The MH module provides a set of functions to manipulate MH mailboxes. Directory mh/. 2.5.1/ References - almost none 2.5.2/ Dependencies - tools/ 2.5.3/ Files descriptions mailmh.[ch] -- functions to manipulate MH mailboxes. 2.5.4/ Interface Include for this module is mailmh.h. There is not yet documentation for MH module. 2.6/ IMF - Internet Message Format ---------------------------------- The IMF module provides functions to parse data given in RFC 2822 format (Internet Message Format). Directory imf/. 2.6.1/ References - RFC 2822 - Internet Message Format (Not entirely implemented) - RFC 2076 - Common Internet Message Headers Not yet implemented : - RFC 2298 - An Extensible Message Format for Message Disposition Notifications 2.6.2/ Dependencies - tools/ 2.6.3/ Files descriptions mailimf.[ch] -- functions to parse RFC 2822 messages. mailimf_types.[ch] -- definition of types and constructors for these types mailimf_types_helper.[ch] -- contains function definitions that will help to create data necessary to use IMF module. mailimf_write.[ch] -- functions that output RFC 2822 messages or sub-part of the messages in a (FILE *). 2.6.4/ Interface Include for this module is mailimf.h and includes all other headers. The interface of IMAP4rev1 is documented in the following files : mailimf.h mailimf_types.h mailimf_types_helper.h mailimf_write.h 2.7/ MIME - Multipurpose Internet Mail Extensions ------------------------------------------------- The MIME module provides functions to parse structure of MIME messages. Directory mime/. 2.7.1/ References - RFC 2045 - Multipurpose Internet Mail Extensions (MIME) Part One: Format of Internet Message Bodies. - RFC 2047 - MIME (Multipurpose Internet Mail Extensions) Part Three: Message Header Extensions for Non-ASCII Text. - RFC 2183 - Communicating Presentation Information in Internet Messages: The Content-Disposition Header Field Not implemented : - RFC 2046 - Multipurpose Internet Mail Extensions (MIME) Part Two: Media Types. 2.7.2/ Dependencies - tools/ - imf/ 2.7.3/ Files descriptions mailmime.[ch] -- functions to parse the MIME fields (RFC 2045). mailmime_content.[ch] -- functions to parse the MIME message. You get the different parts and you can decode them. mailmime_decode.[ch] -- functions to parse the MIME-encoded fields. mailmime_disposition.[ch] -- functions to parse the Content-Disposition field (RFC 2183) mailmime_types.[ch] -- definition of types and constructors for these types mailmime_types_helper.[ch] -- contains function definitions that will help to create data necessary to use MIME module. mailmime_write.[ch] -- functions that output MIME messages or sub-part of the messages in a (FILE *). 2.7.4/ Interface Include for this module is mailmime.h and includes all other headers. There is not yet documentation for MIME module. 2.8/ SMTP - Simple Mail Transfer Protocol ----------------------------------------- Each command of the SMTP Standard (RFC 2821 and RFC 1891) is implemented in the SMTP module. Directory smtp/. 2.8.1/ References - RFC 2821 - Simple Mail Transfer Protocol (Not entirely implemented) - RFC 1891 - SMTP Service Extension for Delivery Status Notifications 2.8.2/ Depencencies - tools/ 2.8.3/ Files descriptions mailsmtp.[ch] -- functions that implements each SMTP command mailsmtp_helper.[ch] -- functions to get an easier use of SMTP module mailsmtp_socket.[ch] -- provides a function to connect to a SMTP server over TCP mailsmtp_ssl.[ch] -- provides a function to connect to a SMTP server over TLS layer mailsmtp_types.h -- definition of types 2.8.4/ Interface Include for this module is mailsmtp.h and includes all other headers. There is not yet documentation for MIME module. 2.9/ Miscellaneous 2.9.1/ References - RFC 2234 - Augmented BNF for Syntax Specifications: ABNF - RFC 2595 - Using TLS with IMAP, POP3 and ACAP 2.9.2/ Tools tools/ directory contains some tools functions and useful data structures. alloc.h -- a wrapper on malloc() carray.[ch] -- an array, that grows automatically when elements are added. charconv.[ch] -- character set converter. For example, it will translate an iso-8859-1 string to an utf-8 string. chash.[ch] -- a hash table which keys can be anything cinthash.[ch] -- a hash table which keys are integers (should be removed and replaced with chash) clist.[ch] -- a double-linked list connect.[ch] -- easy interface to connect a TCP server hmac_md5.h md5.[ch] md5global.h -- MD5 calculation mail.h -- some constants maildb_helper.[ch] -- wrappers to DB 2.x 3.x or 4.x maillock.[ch] -- safely lock a given file mailstream.[ch] -- stream interface - buffered reading and writing on files/socket/SSL connection mailstream_helper.[ch] -- useful functions for stream (for example: read a line) mailstream_low.[ch] -- driver interface for a stream mailstream_socket.[ch] -- stream driver for file descriptors (includes socket) mailstream_ssl.[ch] -- stream driver for SSL connection mailstream_types.h -- data structure definition mapping.[ch] -- map parts of files in memory (no more used) mmapstring.[ch] -- a string, that grows automatically when data are added. 3/ Higher-level --------------- The higher level will allow us to query folder informations or to get messages information or content. There is four kinds of identities : - storage - folders - session - messages In the higher-level interface, you manipulate data types from IMF and MIME module, plus additionnal data types of higher-level. 3.1/ Objects ------------ 3.1.1/ Storage A storage (struct mail_storage) represents whether a server or a main path, It can be an IMAP server, the root path of a MH or a mbox file. 3.1.2/ Folders A folder can be created from a storage. Folders (struct mail_folder) are the mailboxes we can choose in the server or as sub-folder of the main path. Folders for IMAP are the IMAP mailboxes, for MH this is one of the folder of the MH storage, for mbox, there is only one folder, the mbox file content; 3.1.3/ Session Storage and folders communicate with the lower-layer through the mail session data structure. A mail session (struct mailsession) is a mail access to a server or a mail access in the local file system. It allow us to send commands to the mail access. A mail storage is using a mail session to communicate. A folder folder also uses a mail session to get information or to send information. It can be the same session or not, depdending of the implementation. 3.1.4/ Messages From a session, we can get a message (struct mailmessage) to read. 3.2/ Drivers ------------ For a mail access, three drivers exist. One for storage, one for session, one for message. Note that the folder access rely only on session driver. 3.2.1/ storage driver interface mail_storage_driver is the driver structure for mail storage - name is the name of the driver - connect() connects the storage to the remote access or to the path in the local filesystem. - get_folder() can have two kinds of behaviour. Either it creates a new session and independant from the session used by the storage and select the given mailbox or it selects the given mailbox in the current session. It depends on the efficiency of the mail driver. - free_data() frees the data created with mail_storage constructor. a constructor for each kind of access has to be implemented. 3.2.2/ session driver interface maildriver is the driver structure for mail sessions - name is the name of the driver - initialize() is the function that will initializes a data structure specific to the driver, it returns a value that will be stored in the field data of the session. The field data of the session is the state of the session, the internal data structure used by the driver. It is called when creating the mailsession structure with mailsession_new(). - uninitialize() frees the structure created with initialize() - parameters() implements functions specific to the given mail access - connect_stream() connects a stream to the session - connect_path() notify a main path to the session - starttls() changes the current stream to a TLS stream - login() notifies the user and the password to authenticate to the session - logout() exits the session and closes the stream - noop() does no operation on the session, but it can be used to poll for the status of the connection. - check_folder() makes a checkpoint of the session - select_folder() selects a mailbox - expunge_folder() deletes all messages marked \Deleted - status_folder() queries the status of the folder (number of messages, number of recent messages, number of unseen messages) - append_message() adds a RFC 2822 message to the current given mailbox - get_messages_list() returns the list of message numbers of the current mailbox. - get_envelopes_list() fills the parsed fields in the mailmessage structures of the mail_envelopes_list. - remove_message() removes the given message from the mailbox. The message is permanently deleted. - get_message returns a mailmessage structure that corresponds to the given message number. 3.2.3/ message driver interface mailmessage_driver is the driver structure to get information from messages. - name is the name of the driver - initialize() is the function that will initializes a data structure specific to the driver, it returns a value that will be stored in the field data of the mailsession. The field data of the session is the state of the session, the internal data structure used by the driver. It is called when initializing the mailmessage structure with mailmessage_init(). - uninitialize() frees the structure created with initialize(). It will be called by mailmessage_free(). - flush() will free from memory all temporary structures of the message (for example, the MIME structure of the message). - fetch_result_free() will free all strings resulted by fetch() or any fetch_xxx() functions that returns a string. - fetch() returns the content of the message (headers and text). - fetch_header() returns the content of the headers. - fetch_body() returns the message text (message content without headers) - fetch_size() returns the size of the message content. - get_bodystructure() returns the MIME structure of the message. - fetch_section() returns the content of a given MIME part - fetch_section_header() returns the header of the message contained by the given MIME part. - fetch_section_mime() returns the MIME headers of the given MIME part. - fetch_section_body() returns the text (if this is a message, this is the message content without headers) of the given MIME part. - fetch_envelope() returns a mailimf_fields structure, with a list of fields chosen by the driver. - get_flags() returns a the flags related to the message. When you want to get flags of a message, you have to make sure to call get_flags() at least once before using directly message->flags. 3.3/ Higher level interface --------------------------- 3.3.1/ Files descriptions generic_cache.[ch] -- functions that implements cache and flags storing mechanism imapdriver.[ch] -- IMAP driver for session imapdriver_cached.[ch] -- IMAP driver for session, using cache, IMAP already has flags. imapdriver_cached_message.[ch] -- IMAP driver for message, using cache IMAP already has flags. imapdriver_message.[ch] -- IMAP driver for message imapdriver_types.[ch] -- tools function for IMAP driver (types conversion from IMAP module). imapstorage.[ch] -- IMAP driver for storage imfcache.[ch] -- implements cache for parsed fields libetpan.h -- includes all necessary header files to use libEtPan! maildriver.[ch] -- wrappers to calls to the session driver maildriver_tools.[ch] -- default implementation for drivers, when the driver does not parse the messages. maildriver_types.[ch] -- data types declaration and constructors maildriver_types_helper.[ch] -- easy data creation mailmessage.[ch] -- wrappers to calls to the message driver mailstorage.[ch] -- storage creation, calls to the storage driver and implementation of folders. mailstorage_tools.[ch] -- tools for storage (connection) mailthread.[ch] -- threading: collection of the mails into a treee mboxdriver.[ch] -- mbox driver for session mboxdriver_cached.[ch] -- mbox driver for session, using flags and cache mboxdriver_cached_message.[ch] -- mbox driver for message, using flags and cache mboxdriver_message.[ch] -- mbox driver for message mboxdriver_tools.[ch] -- mbox driver common functions mboxstorage.[ch] -- mbox driver for storage mhdriver.[ch] -- MH driver for session mhdriver_cached.[ch] -- MH driver for session, using flags and cache mhdriver_cached_message.[ch] -- MH driver for message, using flags and cache. mhdriver_message.[ch] -- MH driver for message mhdriver_tools.[ch] -- MH driver common functions mhstorage.[ch] -- MH driver for storage nntpdriver.[ch] -- NNTP driver for session nntpdriver_cached.[ch] -- NNTP driver for session, using flags and cache nntpdriver_cached_message.[ch] -- NNTP driver for message, using flags and cache nntpdriver_message.[ch] -- NNTP driver for message nntpdriver_tools.[ch] -- NNTP driver common functions nntpstorage.[ch] -- NNTP driver for storage pop3driver.[ch] -- POP3 driver for session pop3driver_cached.[ch] -- POP3 driver for session, using flags and cache pop3driver_cached_message.[ch] -- POP3 driver for message, using flags and cache pop3driver_message.[ch] -- POP3 driver for message pop3driver_tools.[ch] -- POP3 driver common functions pop3storage.[ch] -- POP3 driver for storage 3.3.2/ Interfaces Include for this module is libetpan.h and includes all other headers. The interface of higher layer is documented in the following files : maildriver.h maildriver_types.h maildriver_types_helper.h mailmessage.h mailstorage.h mailstorage_types.[h] mailthread.h 4/ Architecture --------------- (see layer.fig) 5/ Example of use ----------------- You can find some example in tests/ 6/ Constraints -------------- - libEtPan! must run on a system where mmap() is available. - for mbox particularly, libEtPan! make assumption on the fact that a file can be entirely mapped into memory. But if you don't read mailboxes of 1 Go, it should be fine.