Process Incoming Mail with PHP Script with Exim

Does your web application allow people to email a unique e-mail address to perform actions on your site? If not, it’s a great feature to add and can be immensely powerful. The possibilities are endless, but a few common examples of this include customer support systems, posting to social media, or adding tasks to a to-do list. With the To-do List application example, users will send messages to a generated e-mail address that is unique to their account. The incoming message then is processed by a program or script to add that message as a to-do list task.

This task may seem daunting, but it’s fairly simple once you know how it works. It just requires some additional server configuration with your MTA (Message Transfer Agent). This tutorial is going to use Debian along with the MTA Exim 4 to forward incoming e-mails to a PHP script. You can modify it to forward the email to any other type of application you want, such as Python, Perl, C, or even shell scripts.

Installing Exim

The first requirement is to have the proper software running on your server. Exim comes pre-installed inside most Debian releases. You can verify that it’s installed by running the following command.

$ exim -bV

If the command doesn’t exist, you need to install Exim. If you are using Debian, run the following command to install Exim 4.

$ apt-get install exim4

Configuring Exim

We now need to configure Exim. Since explaining how to configure Exim could fill an entire book, please read the documentation on how to setup Exim on your server. MTA’s and mail servers are generally considered difficult to setup properly, so take your time if you’re not familiar with configuring mail servers.

Once the base configuration is finished, you need to open up the Exim configuration file. Depending on your Linux distro, this file could possibly be named differently or located in a different folder. Also Debian has an option to split configuration files. If you have a split configuration files, you will need to edit different files which we mention in the upcoming steps. For everyone else, open up the file /etc/exim4/exim4.conf.template.

Creating a new Router

Routers in Exim can be explained as if they are network routers. It directs incoming mail to its proper destination. We have to create a new router that will capture the messages we want forwarded to our script. The below example will capture and route e-mails where the destination e-mail address has the suffix of .project.

Exim processes routers in order from top to bottom. Be sure to place this below code in the appropriate position. The below code has to be positioned after the line begin routers inside of exim4.conf.template.

If you are using a split configuration file, create a new file inside of /etc/exim4/conf.d/router/ with a numeric prefix to designate the proper position. For example, create the file 100_catch_project_mail.

catch_project_mail:
	driver = accept
	local_part_suffix = .project
	transport = transport_projectshell

The above code sets a router that will accept mail, and then transport it to the transport_projectshell that we are going to create next.

The router only transports the mail if all the conditions are met, which is where the local_part_suffix is applied. Only messages where the destination e-mail address contains the suffix .project will be routed to the transport. This is important so you can still have your normal site e-mail addresses, such as “support@example.com”.

Creating a new Transport

Now we need to configure how we transport the mail (deliver it). Usually mail would be written to a file on the server, but instead we want a script to run. The code below should be placed inside of exim4.conf.template unless you are using split configuration. Make sure to place the transport code under the line begin transports.

For users with a split configuration, create a new file inside of /etc/exim4/conf.d/transport/.

transport_projectshell:
	driver = pipe
	command = /var/www/example.com/bin/mail.sh
	user = nobody
	group = nogroup

The above code creates a new transport called transport_projectshell, which we referenced in the router. We set the driver to pipe which does what it sounds like. It runs the command and the message is piped to the command. We also set user and group which affect who runs the command. Finally the command value is the path to the script we want to run.

The pipe driver has a few quirks you should know about. If more than two messages arrive at the same time, the command will run concurrently. Your script has to be aware of this and take appropriate action to not create a race condition. Also check out the documentation for the pipe driver to learn more about how the command is executed.

Create the Transport Command

When the transport is triggered, it is set to run the command we specified. Now it’s time to create that command. I set my transport command to run a shell script that will then run a PHP script. It may seem a little redundant, but in the future I’d rather just have to modify the shell script instead of reconfiguring Exim. Below is the shell script mail.sh that has the same path as the command we set in the transport.

#!/bin/sh
MESSAGE=$(cat)
CURDIR=$(dirname "${0}")
SCRIPT="project_mail.php"

echo ${MESSAGE} | /usr/bin/php "${CURDIR}/${SCRIPT}" "${MESSAGE_ID}" "${RECIPIENT}"
exit 0

Below is the PHP script the shell script is calling. I have php setup to just write the message to a new unique file. Here is where you would usually connect to your database and perform the magic functions of your site.

<?php
	/* Get passed arguments from mail.sh */
	if($argc !== 3) {
		trigger_error("Missing Arguments", E_USER_WARNING);
		exit(0);
	}

	$messageID = $argv[1];
	$recipient = $argv[2];

	/* Read e-mail from stdin */
	$message = "";
	$phpStdin = fopen("php://stdin", "r");
	if($phpStdin === false) {
		trigger_error("Failed to open stdin", E_USER_WARNING);
		exit(0);
	}

	while(feof($phpStdin) === false) {
		$message .= fread($phpStdin, 1024);
	}

	fclose($phpStdin);

	/* Write File */
	$directory = dirname(__FILE__);
	$fp = fopen("${directory}/${messageID}.txt", "w");
	if($fp === false) {
		trigger_error("Failed to open output file", E_USER_WARNING);
		exit(0);
	}

	fwrite($fp, "Recipient: ${recipient}\n\n${message}");
	fclose($fp);

Restarting Exim

Now that all the settings are jotted down into our configuration files and our scripts are coded, we need to tell Exim to update the configuration file and restart.

$ update-exim4.conf
$ /etc/init.d/exim4 restart

Don’t let the command update-exim4.conf confuse you. It is actually a program and not a file.

Testing the Script

You’re done. Pat yourself on the back, close out of the countless shells you have open, and let’s hope everything works. To test the script you need to send yourself an e-mail and then check to see if the script ran. Testing locally on the same device is the quickest, but you should also check from a remote location after the local testing is finished.

Send a quick message to yourself with the following command. Make sure to modify the destination e-mail address to match the rules you have previously configured in your router.

$ echo "Message Body" | mail -n -s "Test Message Subject" "0000.project@localhost"

It’s time to see if our shell script ran. You could just check to see if the command did it’s action, such as saving the email to a database, but a better method would be to check exim’s logs to rule out bugs in your script.

Open up the log located in /var/log/exim4/mainlog and look for the message you sent. A line should appear that looks like the following for successful messages being sent and processed by our script.

2016-11-29 02:41:59 1cBd39-0001Rd-On => 0000 <0000.project@localhost> R=catch_project_mail T=transport_projectshell

As you can see, the R and T values show your router and transport used. If you get an error about Unrouteable address, make sure you are sending messages to the right e-mail address and don’t have any spelling mistakes.

Single Email Address or Capture All

With the above router, we configured Exim to catch all e-mails that are sent to addresses that end with the string .project. But sometimes having a single e-mail address will do just fine. For instance, if you have a support system that will add all messages to a website for viewing and replies (similar to Zendesk), a single e-mail address would be better. With a single e-mail address you will need to add a unique identifier to the message (subject reference number, or reference number inside the body). For messages without a unique identifier, these could be considered a new “thread” or you could send an automated warning message asking users to quote the original message or to not modify the message’s subject.

The Exim configuration will be as follows. This replaces the previous router we created. The below configuration will route all messages destined to support@* to the transport. Configure the domain if you have multiple sites on the same server. The transport has also been renamed, but the transport can be exactly the same.

catch_support_mail:
	driver = accept
	local_parts = support
	transport = transport_supportshell

Security Considerations

Email is fairly insecure and the origination of the message can easily be spoofed. This has to be taken into consideration when working with e-mail and having e-mails able to make changes to an account.

If you are creating an application where each user has a uniquely generated e-mail address to send items to (for instance the To-do List example), only allow new tasks to be added if the author e-mail address is the same as the account e-mail address. You should also check DKIM, SPF, and other authenticity headers before accepting the message. These settings are configured inside Exim, which makes it easier to configure. It’s important to remember that not every e-mail server adds DKIM and SPF headers.

With unique e-mail addresses that are generated, make sure you have enough entropy for random numbers and generate enough random bytes. Also create an option to revoke e-mail addresses and generate a new unique e-mail address to use.