Tinsology

If you are implementing your own user management system you may want to ensure that emails associated with users’ accounts are valid. The most straightforward way of doing this is to send an email to this account and verify that the user received it. Obviously we don’t want to do this manually so the solution is to write a script that automates the process.

High Level Concept
What we will need to do is:

1. Generate a confirmation code for our users are registration time
2. Store that code and track which users are not yet confirmed
3. Send an email containing the confirmation code to the user
4. Facilitate the user in using the code to activate their account

Confirmation Codes

The easiest way to generate a confirmation code is to take a piece of information about the user and hash it. What we don’t want to do however, is reveal how this hash is generated or make it easy for a user to generate this hash on their own. I find that the easiest way to approach this problem is to use the current timestamp in the hash; unless the user knew the time that their hash was generated down to the second they could not generated one on their own. Here is the code:

$hash = md5( $username . time() );

Notice that I assume you already have the username.

Storing Your Confirmation Code

In order to confirm a user, both the user and the server need to know the code. This means you’ll have to store the code once it is generated. You’ll also need to track which users are confirmed and which aren’t. Fortunately we can do both these things at once. You’ll need to either add another column to your `users` table or use another method to associate the code with the user. I’ll assume that you have a column called `status` in your `users` table.

Once you’ve generated your confirmation code store it in the database. We’ll worry about determining which users are confirmed later.

Sending the Email

Now that we have generated our code we will need to send it to the user. The easiest way for a user to confirm their account is to just click a link that has the code encoded into the url:

//the link
$link = "http://yoursite.com/confirm.php?user=$username&code=$hash";

Notice that I use the page confirm.php. We will write the script in the next step, you may call it something different if you like. Also notice that I include the username and hash in the url.

Now we need to create the rest of the email and send it. One thing you’ll want to do is keep things simple. Some email clients filter out any html so it is best to use plain text when possible.

$link = 'http://yoursite.com/confirm.php?code=' . $hash;
ob_start();
?>
Confirmation Required:
Please copy and paste the following link into your browser to confirm your account
<?php echo $link; ?>
If you did not register an account just ignore this message
<?php
$message = ob_get_clean();
$headers = 'From: you@yoursite.com' . "\r\n" .
    'Reply-To: you@yoursite.com' . "\r\n" .
    'X-Mailer: PHP/' . phpversion();

mail($userEmail, 'Confirmation Required', $message, $headers);

I used an output buffer to write the message, if you are not familiar with this approach you may want the read the output buffering section Here. Notice the $headers variable. It is important that you include the From header whenever sending email. Even though the headers parameter is optional, the from header is not. I highly recommend that you take a close look at the mail documentation.

Confirming Users

Now that the user has their confirmation url we need to write the confirm.php script I mentioned earlier. I will leave comments whenever a database transaction is needed:

$userCode = $_GET['code'];
$user = $_GET['username'];

/* get the code stored in the database
if the user does not exist, or has already been
confirmed display an error message */
$serverCode;

if($serverCode == $userCode)
{
	//change the value of the status field in the database to 'confirmed' or something else if you prefer
}
else
{
	//Invalid code. Display an error message
}

In short, all we need to do is compare the user’s code with the one stored in the database. If the match change the value of the user’s status field in the database to some value that will indicated the user is confirmed. Anytime you need to check if a user is confirmed just use that field.

Additional Concerns

There are some additional concerns that I did not address that you may want to.

A user may need to have their confirmation email resent. This is just a matter of pulling the confirmation code out of the database and sending it again.

Some users or bots may register accounts that they never confirm. You may want to setup a crontab or some other means of periodically clearing out these accounts (ie if an account hasn’t been confirmed after a week, delete it).

Protect yourself from SQL Injections whenever sending user data to the database!!!

Anyone who has ever had an email account is probably very familiar with the “Do not reply” email. They can take the form of notifications from your bank, reminders to pay a bill, a newsletter, or just plain spam. Generally the read a little something like:

Please do not reply to this email. This mailbox is not monitored and you will not
receive a response.

Usually this is followed by some instructions to follow if you have a question or concern. I imagine at some point in the history of automated emailers someone decided that they did not want to receive any reply from their users via email, or at least from email directed to the address in question, and everyone blindly proceeded to follow this example. That is what contact forms and support email addresses are for correct? To a certain degree this is true. Having to monitor another email account (or more) for support questions would only exacerbate things in the nightmare that is customer support. So the solution is to toss out any incoming email to these accounts and direct users to ask their questions via support form or an alternate email address.
In my mind this is a cop-out. Consider what you lose by doing this. In most cases if I suddenly have a question after receiving an email from my bank, this question is with regard to the message I just received. If I were able to just hit the reply button the original message would be right there on the bottom of my message. If I am to use some other means to ask my question I might forget to include it or decided that it is of no importance.

In addition to this there is the hassle of the alternative process. I have to go to my bank’s website, find the support form, fill out all the necessary information which in my case means first and last name, date of birth, account number, social, and email address (twice). Believe it or not this is the case even if I’m logged into my account. This is a minor inconvenience, but the process introduces the chance for user error. If I put the wrong account number or some other typo occurs my question might never get answered or I might be asked to do it all over again. If I were able to just hit reply all of this information could be derived (automatically) from my email address.

The problems don’t stop there. Often it is the case that support forms ask you to choose the department or type that your question falls under (i.e. is it a billing question, a technical question, etc.). Sometimes the lines between question categories can easily be blurred and submitting a question to the wrong department could potentially increase the response time. It takes x time for person A to get to your question, if A is not the right person to answer the question then it is redirected to person B. Your wait time is now x + y. If you were able to just reply to the email the type of question would be determined automatically and directed to person B. Your wait time is now just y.

Obviously the ubiquitous “do not reply” emails are no the result of someone failing to realize that it may be convenient for the user. The problem is that it complicates things for the support team. Or it least it appears this way at first glance.

Wouldn’t this mean monitor additional email accounts? There is no reason why any person working in support should have to monitor an email account; there are better ways. A properly implemented support system would make the source of the message, whether it be an email or a form, transparent.

It is important the users be directed to the website so that they can find answers in our FAQ or knowledge base before inundating the support system. The solution here is to replace your “do not reply” messages with links to places where the user can find answers. Additionally this gives you the opportunity to serve specialized questions and answers based on the content of the message. Ultimately, users asking obvious questions or questions that have been answered many times in the past is unavoidable.

What about emails containing sensitive information? A user may inadvertently expose their private information to the support team by hitting reply. The solution here is to be careful about what information you put in an email. I am not suggesting that Do not reply emails should be done away with entirely. If it is absolutely necessary to put sensitive information into an email (password resetting for example) then it is appropriate to use a Do not reply address.

I’d like to conclude this entry by imaging a support system with Do not reply (almost) eliminated. Image the support system for a web hosting company. There are three support departments: Billing, accounts, and technical, each of which has an email address. There is also a Do not reply address used exclusively for password resetting. There is a support form which asks the user to select the appropriate department. Finally, there is a knowledge base containing common and previously answered questions. The knowledge base also has a feature that allows it, given a question or some other block of text, to attempt to find similar questions that have already been answered.

When a member of the technical support team accesses the support system, they see only questions for their department, both from the support form and from the technical department email account. The same is true for the other departments.

If one of the departments sends out a notification, whether it be automatic (bill is due, a change to your account, etc.) or manual (planned downtime, a new feature, or some other announcement), before the message is sent, its content is passed to the knowledge base which determines common questions and answers relating to the content of the message. It selects a few and appends them to the message. The message is then sent. If the user has a question regarding the message they have the option of hitting reply and having their message (original message and all) sent to the correct department.

For a concise overview of some of the more common functions and a few tips read PHP IMAP Notes.

Previously I wrote about Intercepting Email with PHP and so far, that has been the most popular post on this blog. That being the case, I decided to write about how you can use PHP’s IMAP functions to access email.

The main difference between the two method is that when accessing email through IMAP you are accessing a mailbox that already exists. If you use piping to intercept an email you bypass the mailbox altogether. One advantage of using PHP’s IMAP functions to access Email is that you can avoid working with raw data, which would be the case with piping. With the functions provided by PHP, there would be nothing to stop you from building a webmail application with just as many capabilities as any of the major applications out there.

Before you can retrieve your email you must establish a connection to the mailbox. In order to do this you will need the IP and port of the mail server, along with a valid username and password. To connect to a mailbox on your local host use this code:

$host = 'localhost:143';
$user = 'user@domain.com';
$password = 'drowssap';
$mailbox = "{$host}INBOX";

$mbx = imap_open($mailbox , $user , $password);

Note that port 143 is the default, but is not always the correct port. If you don’t know the port you can use just the IP (or localhost) by itself, though it is recommended that you find the correct port. Also note that you can use this code to open a POP stream, as well as many other protocols. I’ll show you how to open a POP stream here, but for other protocols (IMAP over SSL, NNTP, etc.) you’ll have to read the documentation for imap_open.

$pophost = 'localhost:110/pop3';
$user = 'user@domain.com';
$password = 'drowssap';
$mailbox = "{$host}INBOX";

$mbx = imap_open($mailbox , $user , $password);

In this case as well, port 110 is the default but might not be the case for you.

Now that we’ve opened our IMAP (or other) stream we can start using PHP’s various IMAP functions to work with our mailbox. Note that if you’re stream is not an IMAP stream, not all of these functions will work as intended. Here I will show you to retrieve all of the emails in your inbox and how to view one of those emails, but I will only be using a small portion of the functions available to you, so you may want to take a look at the functions.

<?php
$mbx = imap_open($mailbox , $user , $password);

/*imap_check returns information about the mailbox
including mailbox name and number of messages*/
$check = imap_check($mbx);

/*imap_fetch_overview returns an overview for a message.
An overview contains information such as message subject,
sender, date, and if it has been seen. Note that it does
not contain the body of the message. Setting the second
parameter to "1:n" will cause it to return a sequence of messages*/
$overviews = imap_fetch_overview($mbox,"1:{$check->Nmsgs}");
?>
<table>
     <tr>
          <td>From</td>
          <td>Date</td>
          <td>Subject</td>
     </tr>

<?php
foreach($overviews as $overview)
{
     ?>
     <tr>
          <td><?php echo $overview->from; ?></td>
          <td><?php echo $overview->date; ?></td>
          <td><a href="open.php?id=<?php echo $overview->uid; ?>"><?php echo $overview->subject; ?></a></td>
     </tr>
     <?php
}
?>
</table>

The code above will generate a list of emails in your inbox, similar to what you would see if you logged into your email account. Each subject line is a link to open.php which is an arbitrary script that will open the message with the corresponding uid. The code below is an example of what open.php might look like:

<?php
$uid = $_GET['id']; //I won't validate this input, but you should

$mbx = imap_open($mailbox , $user , $password);
$overview = imap_fetch_overview($mbx, $uid, FT_UID);
$body = imap_body($mbx, $uid, FT_UID);
?>
<table>
     <tr>
          <td>From:</td>
          <td><?php echo $overview->from; ?></td>
     </tr>
     <tr>
          <td>Date:</td>
          <td><?php echo $overview->date; ?></td>
     </tr>
     <tr>
          <td>Subject:</td>
          <td><?php echo $overview->subject; ?></td>
     </tr>
     <tr>
          <td colspan="2"><?php echo $body; ?></td>
     </tr>
</table>

Notice that I set the FT_UID parameter in order to use a UID rather than a message_id. A UID will always remain the same, while a message_id may change as the content of the mailbox changes. The script would most likely work with message ids, but UIDs are guaranteed to work. If you are using a POP stream, you might find that in some cases you have to use message_ids in place of UIDs due to compatibility issues.

The imap_body function returns the body of the message with the corresponding id, but it also sets the message to “seen”. If you do not want to mark the message as seen, pass it the FT_PEEK option.

There is a great deal that you can do with these functions; far too much for me to detail all of it here. If there is anything in particular you would like to see, feel free to post a comment detailing your question.

Here I will show you how to intercept an incoming email through piping. I have also written a guide on How to Access Email in a Mailbox through IMAP

One of the most frequent questions I’m asked, or I see asked on forums is “how do I send out an email using PHP?” The answer to that is fairly simple and well documented. It more or less involves the use of a single function mail().

Something a bit more complicated and, I think, more interesting is how you can intercept an incoming email with a pretty small chunk of PHP code. The answer to how to do this, though not so difficult to find out, is a far less trivial thing.

The Scenario (optional if you don’t feel like reading)

Say we are building a contact form for a web application. This pretty basic contact form takes whatever input  the user gives, to keep things simple we’ll just say its the users email address and a message. This message is then sent to a database where it can be read from an administrative control panel of some sort. This is pretty simple and for the most part effective. Personally I have a problem with this type of setup. If users are not logged in when using the form there is no guaranteed way to respond. The email address they provide may or may not be valid, or they could just be spamming some nonsense because they are anonymous in the Internet and no one can stop them. A simple solution: Force users to log in. But Wait! what if users are having trouble creating an account or logging in and this is the reason why thy need to contact you? OK, to solve this problem we’ll just set up an email address that users can contact us with in addition to the form. This solves the problem of reliable contact information and minimizes the spam problem (the email address can still be spammed but a lot of it can be filtered). So once this is all setup will have a contact form that sends info that can be read by all of our admins in our admin panel… and the occasional email to some email account that all of our admins will need the password to to check regularly.

This may be acceptable to some… but not to us! We know of a better way to consolidate all of our communications with our users while at the same time having our desired setup. OK maybe you don’t or else why would you read this. Here’s how we do it:

The Code(this is where you should start reading if you like to get to the point)

Some of you may be thinking that PHP is so magical that there is a simple function that is going to go out and fetch the email we want from where ever it is on the server. Well you’re wrong (not about the magical part). What we need to do is setup a script that can capture the email when the server recieves it. This is actually pretty simple, all we need to do is extract the email data from standard input (php://stdin).

$file = fopen('php://stdin', 'r');

Once we establish a file pointer to our input stream the rest is the same as reading in any other file. Here’s the complete code:

< ?php
$data = '';
$file = fopen('php://stdin', 'r');
while(!feof($file))
{
     $data .= fgets($file, 4096);
}
fclose($file);
?>

$data now stores the raw content of the email, headers and body. At this point we just need to decide what to do with the email, I’ll leave that for you to decide. So other than that our script is complete correct? False. We have not yet addressed one crucial step. The server needs to be told how to execute this script. You may be wondering, why would we have to tell the server what to do with a .php file… its .php! The main difference between a user requesting a .php file via the internet and what we are doing here is what is handling the request. When a user goes to index.php on your website, apache knows what to do with the file because that is how it has been configured. The server on the otherhand will not necissarily assume that the file should be parsed by php. So how do we tell it to you say? Hashbang. A hashbang tells the server how to handle a file, in our case we will use the following hashbang #!/usr/bin/php -q. Note that this may or may not work for you depending on the path to php on your server. The one I’m using is the default, so if you’re not sure use that one, otherwise contact your hosting provider. Ok so here is our completed, hashbanged code:

#!/usr/bin/php -q
< ?php
$data = '';
$file = fopen('php://stdin', 'r');
while(!feof($file))
{
     $data .= fgets($file, 4096);
}
fclose($file);
?>

Note that the hashbang is outside of the php tags. This is absolutely necissary for the script to work. Save this code and take note of the path to it.

The Setup

Now that we’ve written our script we need to tell the server what to do with incoming email. This setup is going to vary depending on your server configuration, I’ll tell you how to do it on a linux server with sendmail and cpanel but for anything else you’re on your own (or you can post back explaining how and I’ll add it to the tutorial). What we need to do is setup a forwarder. If you have cPanel this is fairly straightforward. In your mail section find the forwarders option and click it. From there click add new forwarder. At this point you’ll have a few different options: you can forward to another address, discard and send an error message, do nothing, or pipe to a program. We want to pipe to a program. Just type in the address that you want piped, select the pipe option and then set the path to the path to the file you just saved. If you have access to your home directory, you can forego using cpanel and use a .forward file. All you have to do is create a file called .forward and add the following line:

email@address.com,”|/path/to/script.php”

Of course replace the email address with the address you want forwarded and /path/to/script.php with the actual path to the script. You can also omit the email address portion to have all mail forwarded to the script. I believe this process is the same using Exim or Qmail.

Well that’s it for the most part. Feel free to post back with any problems you have or additions you’d like to make. I’ll write about taking a raw email and converting it to a more friendly format, in this case an associative array, in the near future.