Converting Mbox mailboxes to Maildir format:

page version: 2006-05-07
software version: 3.20

News:

Introduction:

Mb2md.pl was originally developed by Robin Whittle <rw at firstpr.com.au> (you can find his old page here). It is a Perl script that takes one or more Mbox format mailbox files in a directory and convert them to Maildir format mailboxes. As the Mbox format has some drawbacks, D. J. Bernstein created the Maildir format when he wrote Qmail. With the Mbox format all mail of a specific folder is stored as one large text file. The Maildir format stores each mail as a seperate file. It is a faster and more efficient way to store mail. It works particularly well over NFS, which has a long history of locking-related woes.
The Mbox format is used by most of the POP3/IMAP servers, most mail servers (MTAs) and mail readers (MUAs). The Maildir format is used by Qmail, Courier-MTA and can be also used as a alternative mail storage format by Postfix and Exim. A good POP3/IMAP server for Maildirs is Courier IMAP.

Mb2md.pl does not only convert mailbox files into a Maildir but also the /var/spool/mail/$USER mailspool file. It is smart enough to not transfer a dummy message such as the UW IMAPD puts at the start of Mbox mailboxes - and you could add your own search terms into the script to make it ignore other forms of dummy first message.

Features:

Currently mb2md.pl has the following features:

converting the user's main mailbox that is referenced by the ENV variable $MAIL
converting a single mailbox into corresponding maildir
converting multiple mailboxes in a directory into corresponding maildirs
recursive operation on a given directory to convert the complete mail storage of one user
replaces all occurrences of dots ('.') in a mailbox name by underscores ('_')
is able to handle spaces in mailbox names
is able to handle leading/trailing spaces in mailbox names
converts mbox files in DOS format (CRLF) to Unix file format
can strip an extension (e.g. ".mbx") from a mailbox name prior to converting
uses the Content-Length: header if told to
removal of dummy message that a couple of IMAP servers (e.g. UW-IMAPD) put at the beginning of a mailbox
removal of the "From " line that starts every mail in a mailbox
removal of the special escape character (>) that the MTA puts in front of lines starting with "From " other than the first line
setting the file date of a converted message according to the date found in the "From " line of the original mail
setting the flags F,R,S,T (flagged, replied, seen, deleted) on the filename of the converted message according to the flags found in "Status:"/"X-Status:"/"X-Mozilla-Status:"/"X-Evolution:" headers of the original mail
should work on all *nix-like OS

Usage:

Run this as the user of the mailboxes, not as root.

mb2md -h
mb2md [-c] -m [-d destdir]
mb2md [-c] -s sourcefile [-d destdir]
mb2md [-c] -s sourcedir [-l wu-mailboxlist] [-R|-f somefolder] [-d destdir] [-r strip_extension]

-c use the Content-Length: headers (if present) to find the
beginning of the next message
Use with caution! Results may be unreliable. I recommend to do
a run without "-c" first and only use it if you are certain,
that the mbox in question really needs the "-c" option

-m If this is used then the source will
be the single mailbox at /var/spool/mail/blah for
user blah and the destination mailbox will be the
"destdir" mailbox itself.

-s source Directory or file relative to the user's home directory,
which is where the the "somefolders" directories are located.
Or if starting with a "/" it is taken as a
absolute path, e.g. /mnt/oldmail/user

A single mbox file which will be converted to
the destdir.

-R If defined, do not skip directories found in a mailbox
directory, but runs recursively into each of them,
creating all wanted folders in Maildir.
Incompatible with '-f'

-f somefolder Directories, relative to "sourcedir" where the Mbox files
are. All mailboxes in the "sourcedir"
directory will be converted and placed in the
"destdir" directory. (Typically the Inbox directory
which in this instance is also functioning as a
folder for other mailboxes.)

The "somefolder" directory
name will be encoded into the new mailboxes' names.
See the examples below.

This does not save an UW IMAP dummy message file
at the start of the Mbox file. Small changes
in the code could adapt it for looking for
other distinctive patterns of dummy messages too.

Don't let the source directory you give as "somefolders"
contain any "."s in its name, unless you want to
create subfolders from the IMAP user's point of
view. See the example below.

Incompatible with '-R'

-d destdir Directory where the Maildir format directories will be created.
If not given, then the destination will be ~/Maildir .
Typically, this is what the IMAP server sees as the
Inbox and the folder for all user mailboxes.
If this begins with a '/' the path is considered to be
absolute, otherwise it is relative to the users
home directory.

-r strip_ext If defined this extension will be stripped from
the original mailbox file name before creating
the corresponding maildir. The extension must be
given without the leading dot ("."). See the example below.

-l UW-file File containing the list of subscribed folders. If
migrating from UW-IMAP the list of subscribed folders will
be found in the file called .mailboxlist in the users
home directory. This will convert all subscribed folders
for a single user:
/bin/mb2md -s mail -l .mailboxlist -R -d Maildir
and for all users in a directory as root you can do the
following:
for i in *; do echo $i;su - $i -c "/bin/mb2md -s mail -l .mailboxlist -R -d Maildir";done

Example
=======

We have a bunch of directories of Mbox mailboxes located at
/home/blah/oldmail/

/home/blah/oldmail/fffff
/home/blah/oldmail/ggggg
/home/blah/oldmail/xxx/aaaa
/home/blah/oldmail/xxx/bbbb
/home/blah/oldmail/xxx/cccc
/home/blah/oldmail/xxx/dddd
/home/blah/oldmail/yyyy/huey
/home/blah/oldmail/yyyy/duey
/home/blah/oldmail/yyyy/louie

With the UW IMAP server, fffff and ggggg would have appeared in the root
of this mail server, along with the Inbox. aaaa, bbbb etc, would have
appeared in a folder called xxx from that root, and xxx was just a folder
not a mailbox for storing messages.

We also have the mailspool Inbox at:

/var/spool/mail/blah

To convert these, as user blah, we give the first command:

mb2md -m

The main Maildir directory will be created if it does not exist.
(This is true of any argument options, not just "-m".)

/home/blah/Maildir/

It has the following subdirectories:

/home/blah/Maildir/tmp/
/home/blah/Maildir/new/
/home/blah/Maildir/cur/

Then /var/spool/blah file is read, split into individual files and
written into /home/blah/Maildir/cur/ .

Now we give the second command:

mb2md -s oldmail -R

This reads recursively all Mbox mailboxes and creates:

/home/blah/Maildir/.fffff/
/home/blah/Maildir/.ggggg/
/home/blah/Maildir/.xxx/
/home/blah/Maildir/.xxx.aaaa/
/home/blah/Maildir/.xxx.bbbb/
/home/blah/Maildir/.xxx.cccc/
/home/blah/Maildir/.xxx.aaaa/
/home/blah/Maildir/.yyyy/
/home/blah/Maildir/.yyyy.huey/
/home/blah/Maildir/.yyyy.duey/
/home/blah/Maildir/.yyyy.louie/

The result, from the IMAP client's point of view is:

Note that although ~/Maildir/.xxx/ and ~/Maildir/.yyyy may appear
as folders to the IMAP client the above commands to not generate
any Maildir folders of these names. These are simply elements
of the names of other Maildir directories. (if you used '-R', they
whill be able to act as normal folders, containing messages AND folders)

With a separate run of this script, using just the "-s" option
without "-f" nor "-R", it would be possible to create mailboxes which
appear at the same location as far as the IMAP client is
concerned. By having Mbox mailboxes in some directory:
~/oldmail/nnn/ of the form:

/home/blah/oldmail/nn/xxxx
/home/blah/oldmail/nn/yyyyy

then the command:

mb2md -s oldmail/nn

will create two new Maildirs:

/home/blah/Maildir/.xxx/
/home/blah/Maildir/.yyyy/

Then what used to be the xxx and yyyy folders now function as
mailboxes too. Netscape 4.77 needed to be put to sleep and given ECT
to recognise this - deleting the contents of (Win2k example):

C:\Program Files\Netscape\Users\uu\ImapMail\aaa.bbb.ccc\

where "uu" is the user and "aaa.bbb.ccc" is the IMAP server

I often find that deleting all this directory's contents, except
"rules.dat", forces Netscape back to reality after its IMAP innards
have become twisted. Then maybe use File > Subscribe - but this
seems incapable of subscribing to folders.

For Outlook Express, select the mail server, then click the
"IMAP Folders" button and use "Reset list". In the "All"
window, select the mailboxes you want to see in normal
usage.

This script did not recurse subdirectories or delete old mailboxes, before addition of the '-R' parameter :)

Be sure not to be accessing the Mbox mailboxes while running this
script. It does not attempt to lock them. Likewise, don't run two
copies of this script either.

Trickier usage . . .
====================

If you have a bunch of mailboxes in a directory ~/oldmail/doors/
and you want them to appear in folders such as:

~/Maildir/.music.bands.doors.Jim
~/Maildir/.music.bands.doors.John

etc. so they appear in an IMAP folder:

Then you could rename the source directory to:

~/oldmail/music.bands.doors/

then use:

mb2md -s oldmail -f music.bands.doors

Or simply use '-R' switch with:
mb2md -s oldmail -R

Stripping mailbox extensions:
=============================

If you want to convert mailboxes that came for example from
a Windows box than you might want to strip the extension of
the mailbox name so that it won't create a subfolder in your
mail clients view.

Example:
You have several mailboxes named Trash.mbx, Sent.mbx, Drafts.mbx
If you don't strip the extension "mbx" you will get the following
hierarchy:

Inbox
|
- Trash
| | mbx
|
- Sent
| | mbx
|
- Drafts
| mbx

This is more than ugly!
Just use:
mb2md -s oldmail -r mbx

Note: don't specify the dot! It will be stripped off
automagically ;)

Download:

Version: mb2md-3.20 (requires TimeDate perl library!)