Wednesday 19 September 2007

How to move your email to GMail using free tools

I've been using GMail for quite some time now, and have recently decided to import a lot of archived mail from my old mail accounts. Currently I have this mail stored in various formats, most notable Outlook PST-files. PST files are notorious for not being able to be read by anything but Outlook. Worse yet, I have an archive in Outlook 2003 format, which can't be read by the only (old) Outlook version that I have access too. I'll solve this situation one way or another, but that's not what this post is about.

I'm trying to find a good way to import my archiving email, using free tools. My first attempt some time ago, involved importing e-mail from an imminently closing POP account into Thunderbird and then redirecting/bouncing the e-mails using the MailRedirect plugin. [1] This worked fine, except that all mails showed up in my GMail inbox with the date of when I redirected them (apparently this can be fixed by importing them into a temporary GMail account, and then letting MailFetcher retrieve them all via POP, which fixes the date issue).

Using the Thunderbird MailRedirect plugin is not very user friendly when you have lots of emails to transfer: some will fail and it is a lot of work to pick out the ones that have failed and re-redirect them. Also, one reason why they might fail is that the smtp server you are using for the redirecting is refusing connections because you are sending too many e-mails in too short a time interval.

The best solution would be for all your archived mail to exist in a POP-accessible account and have GMail retrieve them. GMail will keep trying until it is done, which means that if it fails for some reason, it will just try again in an hour. If you have your e-mails in local files this doesn't work, unless you could somehow get your e-mails back into the mail server. Using IMAP you can. If you have an IMAP account available, then great. If not, you can still do it for free in a few steps, as long as your e-mail client understands IMAP and you have some time.

Firstly you need to install a free smtp server that supports both IMAP and POP. I used MailEnable Professional Evaluation Edition, which is fully usable for three months, for evaluation purposes. I set the mail server up inside a virtual machine (running an evaluation version of Windows) using the free VMWare Server. Using a virtual machine is good for security as it allows you to open a port on your firewall and forward it to just the virtual machine, which should leave your main machine protected. [2]

In MailEnable create a post office and add user accounts for each GMail label that you want to import mail to. If you have a lot of e-mail to import, this will make things easier to sort when it finally arrives in GMail, as the MailFetcher allows you to automatically assign a label to the imported mail. Using separate POP accounts means you can set the labels when they come in, rather then sorting through your mail manually after everything has been imported. For instance, if you want to add some mail to a label called Friends and some to a label called Family, add two accounts: "friends" and "family".

Open your e-mail client and connect to the accounts you have just created in MailEnable. You need to know the IP-address of your virtual machine. If you have called your post-office "mailenable" then the above accounts would have a login name of "friends@mailenable" and "family@mailenable" respectively. Now you're ready to start copying e-mails from your mailbox to your new IMAP accounts. Just drag them into the correct IMAP account's Inbox and they will be uploaded to the server.

Once you've got your e-mail in your IMAP server, it is time to open a port on your firewall and forward it to the IMAP server virtual machine. The default POP port is 110.

Add your accounts to the GMail MailFetcher (GMail settings / Accounts). You need to give them a real e-mail address, but that won't actually be used. Make sure the login name is the same as you have set up in your mail server (so friends@mailenable or family@mailenable in the above example). Set the mail/pop server address to your external IP address. Select the label you want GMail to use for the e-mails in this account (it will provide a reasonable default). Also select it to archive all the e-mails, so they don't clutter up your inbox. Don't select "leave a copy on the server", so you can easily tell when it is done by looking at the mail server. When you click "Add Account", GMail will start retrieving your e-mails!

GMail appears to retrieve e-mail from your pop3 account about once per hour, taking 200 e-mails at a time, so if you have more than 200 e-mails, you'll have to wait a while before all of them have been retrieved. 200 e-mails per hour = 4800 per day. If you have a lot of email, it can easily take a couple of days. Because you didn't select "leave a copy on the server", the e-mails will be deleted from the mail server when GMail retrieves them. This means that you can easily tell when it is done retrieving by looking at the user account list in the mail server (well, it was easy in MailEnable): It's done when the number of messages in the mailbox is 0.

When it is done retrieving all e-mails from all accounts you should close the port on your firewall. You can now stop your virtual machine and get rid of it if you want.

Don't remove the pop accounts yet from the GMail accounts settings, as GMail hasn't processed all the e-mails yet, and won't label/archive them if you remove the accounts, but dump them in your inbox instead. Leave it a while (I have no idea how long, one hour? two?), and the labelling should go on.

[1] I now wonder why I didn't just use GMail's MailFetcher for this particular case, but don't remember why. Perhaps MailFetcher wasn't available yet? Perhaps I wanted a local copy in Thunderbird? Perhaps I had other reasons? Or perhaps I just wasn't happy with handing my password for one service to another service.

[2] This of course assumes you're using a firewall (or router with built-in firewall) that supports port-forwarding. Note that I'm not a security expert. Don't forget to close the port when you're done.

No comments: