mailfront imapfront-auth with dovecot 1.2.16 : SUCCESS

[Giam Teck Choon]

Hi,

This is just to share with the list members that upon couple hours of
researching, I have successfully run mailfront v1.17 imapfront-auth with
dovecot v1.2.16 which doesn't require dovecot service to be running.

I know http://mij.oltrelinux.com/net/dovecot-qmail-vmailmgr/ and that
requires dovecot service running and I can't use with imapfront-auth. I
prefer to use mailfront + cvm (pop3d/pop3sd with pop3front-auth and
imapd/imapsd with imapfront-auth) so that all related mail services are
running under daemontools + ucspi-tcp.

Below is the wrapper /usr/bin/mailfront-imapfront-auth-dovecot or name
whatever you like and chmod +x /usr/bin/mailfront-imapfront-auth-dovecot:
--------------------8<----------8<----------8<----------8<---------------------
#!/bin/sh

if [ -n "$MAIL" ] ; then
export MAIL=maildir:`echo $MAIL | sed 's@:@::@g'`
fi
exec "$@"
--------------------8<----------8<----------8<----------8<---------------------

Below is example imapd/run file which works for courier-imap and dovecot:
--------------------8<----------8<----------8<----------8<---------------------
#!/bin/sh

DOVECOTWRAPPER=""
# Default to support dovecot 1.0.x
if [ -x "/usr/libexec/dovecot/imap" ] ; then
IMAPD=/usr/libexec/dovecot/imap
if [ -x "/usr/bin/mailfront-imapfront-auth-dovecot" ] ; then
DOVECOTWRAPPER=/usr/bin/mailfront-imapfront-auth-dovecot
elif [ -x "/usr/local/bin/mailfront-imapfront-auth-dovecot" ] ; then
DOVECOTWRAPPER=/usr/local/bin/mailfront-imapfront-auth-dovecot
fi
elif [ -x "/usr/local/bin/imapd" ] ; then
IMAPD=/usr/local/bin/imapd
elif [ -x "/usr/lib/courier-imap/bin/imapd" ] ; then
IMAPD=/usr/lib/courier-imap/bin/imapd
else
IMAPD=/usr/bin/imapd
fi

if [ -x "/usr/bin/imapfront-auth" ] ; then
IMAPFRONT_AUTH=/usr/bin/imapfront-auth
else
IMAPFRONT_AUTH=/usr/local/bin/imapfront-auth
fi

ulimitdata=500000000
concurrency=40

CVM_SASL_PLAIN=cvm-local:/tmp/.cvm-vmailmgr
CVM_SASL_LOGIN=cvm-local:/tmp/.cvm-vmailmgr
export CVM_SASL_PLAIN
export CVM_SASL_LOGIN

exec \
softlimit -m $ulimitdata \
tcpserver -DHRUvX -c "$concurrency" -l "`head -n 1 /var/qmail/control/me`" \
-x /etc/tcpcontrol/imap.cdb 0 imap \
$IMAPFRONT_AUTH $DOVECOTWRAPPER \
$IMAPD 2>&1
--------------------8<----------8<----------8<----------8<---------------------

C wrapper source can be found at:

http://choon.net/mailfront-imapfront-auth-dovecot.php

Many thanks to MIJ at http://mij.oltrelinux.com as I copied one of his
source code function. Sorry, I am not good in C at all and only code in
C occasionally.

Hope the above information is useful to someone ;)

Thanks.

Kindest regards,
Giam Teck Choon

[Bruce Guenter]

Post by Giam Teck Choon
This is just to share with the list members that upon couple hours of
researching, I have successfully run mailfront v1.17 imapfront-auth with
dovecot v1.2.16 which doesn't require dovecot service to be running.
Below is the wrapper /usr/bin/mailfront-imapfront-auth-dovecot or name
--------------------8<----------8<----------8<----------8<---------------------
#!/bin/sh
if [ -n "$MAIL" ] ; then
fi

Well, if that's all that's needed, I can add that right into
imapfront-auth itself.

Please try the appended patch. Simply set SETUP_DOVECOT=1 (or =
anything) before running imapfront-auth, and it should set $MAIL to an
appropriate string. I am working on testing it myself, but the test CVM
I am currently using doesn't actually set a mail fact.
--
Bruce Guenter <***@untroubled.org> http://untroubled.org/

diff --git a/imapfront-auth.c b/imapfront-auth.c
index 3f85d21..eecb658 100644
--- a/imapfront-auth.c
+++ b/imapfront-auth.c
@@ -245,15 +245,37 @@ void cmd_capability(void)
respond(0, "OK CAPABILITY completed");
}

+static int setup_env(void)
+{
+ const char* maildir;
+ const char* colon;
+
+ if (cvm_fact_mailbox != 0
+ && getenv("SETUP_DOVECOT")) {
+ /* Use cmd for temporary storage of the substituted mailbox name */
+ if (!str_copys(&cmd, "maildir:"))
+ return 0;
+ maildir = cvm_fact_mailbox;
+ while ((colon = strchr(maildir, ':')) != 0) {
+ if (!str_catb(&cmd, maildir, colon-maildir)
+ || !str_catb(&cmd, "::", 2))
+ return 0;
+ maildir = colon + 1;
+ }
+ if (!str_cats(&cmd, maildir))
+ return 0;
+ cvm_fact_mailbox = cmd.s;
+ }
+ return cvm_setenv()
+ && setenv("IMAPLOGINTAG", tag.s, 1) == 0
+ && setenv("AUTHENTICATED", cvm_fact_username, 1) == 0;
+}
+
void do_exec(void)
{
if (!cvm_setugid())
respond(0, "NO Internal error: could not set UID/GID");
- else if (!cvm_setenv() ||
- (cvm_fact_mailbox != 0 &&
- setenv("MAILDIR", cvm_fact_mailbox, 1) == -1) ||
- setenv("IMAPLOGINTAG", tag.s, 1) == -1 ||
- setenv("AUTHENTICATED", cvm_fact_username, 1) == -1)
+ else if (!setup_env())
respond(0, "NO Internal error: could not set environment");
else {
alarm(0);

[Giam Teck Choon]

Post by Bruce Guenter

Post by Giam Teck Choon
This is just to share with the list members that upon couple hours of
researching, I have successfully run mailfront v1.17 imapfront-auth with
dovecot v1.2.16 which doesn't require dovecot service to be running.
Below is the wrapper /usr/bin/mailfront-imapfront-auth-dovecot or name
--------------------8<----------8<----------8<----------8<---------------------
#!/bin/sh
if [ -n "$MAIL" ] ; then
fi

Well, if that's all that's needed, I can add that right into
imapfront-auth itself.
Please try the appended patch. Simply set SETUP_DOVECOT=1 (or =
anything) before running imapfront-auth, and it should set $MAIL to an
appropriate string. I am working on testing it myself, but the test CVM
I am currently using doesn't actually set a mail fact.

Yes, the patch works! I should ask you first before spending time to
code the wrapper :p

Below is the imapd/run file:

--------------------8<----------8<----------8<----------8<---------------------
#!/bin/sh

# Default to support dovecot
if [ -x "/usr/libexec/dovecot/imap" ] ; then
IMAPD=/usr/libexec/dovecot/imap
export SETUP_DOVECOT=1
elif [ -x "/usr/local/bin/imapd" ] ; then
IMAPD=/usr/local/bin/imapd
elif [ -x "/usr/lib/courier-imap/bin/imapd" ] ; then
IMAPD=/usr/lib/courier-imap/bin/imapd
else
IMAPD=/usr/bin/imapd
fi

if [ -x "/usr/bin/imapfront-auth" ] ; then
IMAPFRONT_AUTH=/usr/bin/imapfront-auth
else
IMAPFRONT_AUTH=/usr/local/bin/imapfront-auth
fi

ulimitdata=500000000
concurrency=40

CVM_SASL_PLAIN=cvm-local:/tmp/.cvm-vmailmgr
CVM_SASL_LOGIN=cvm-local:/tmp/.cvm-vmailmgr
export CVM_SASL_PLAIN
export CVM_SASL_LOGIN

exec \
softlimit -m $ulimitdata \
tcpserver -DHRUvX -c "$concurrency" -l "`head -n 1 /var/qmail/control/me`" \
-x /etc/tcpcontrol/imap.cdb 0 imap \
$IMAPFRONT_AUTH \
$IMAPD 2>&1
--------------------8<----------8<----------8<----------8<---------------------

Thanks.

Kindest regards,
Giam Teck Choon

[Bruce Guenter]

Post by Giam Teck Choon

Post by Bruce Guenter
Please try the appended patch. Simply set SETUP_DOVECOT=1 (or =
anything) before running imapfront-auth, and it should set $MAIL to an
appropriate string.

Yes, the patch works!

Excellent. Is the choice of $SETUP_DOVECOT a reasonable name, or should
I use something else (like $DOVECOT_MAIL etc)?

I see by the wiki that this works for version 2 as well, and also
supports mbox files. I will add a stat to the imapfront-auth code to
handle both cases.

[Giam Teck Choon]

Post by Bruce Guenter

Post by Giam Teck Choon

Post by Bruce Guenter
Please try the appended patch. Simply set SETUP_DOVECOT=1 (or =
anything) before running imapfront-auth, and it should set $MAIL to an
appropriate string.

Yes, the patch works!

Excellent. Is the choice of $SETUP_DOVECOT a reasonable name, or should
I use something else (like $DOVECOT_MAIL etc)?

If you ask me about appropriate name, I think $DOVECOT_MAIL_LOCATION or
$DOVECOT_MAILLOCATION since we are passing the MailLocation to dovecot
imap directly. Anyway, whatever name you like doesn't matter as long as
it works :)

I think if your imapfront-auth can handle dovecot different
mail_location format would be great. Yes, read your comment about
adding stat to handle both cases.... ... :) So how you stat the two
format? Just curious. By using the $SETUP_DOVECOT variable? Like
SETUP_DOVECOT=mbox|maildir? Or simply stat the $MAIL to see it is a
directory or a file to differential maildir or mbox respectively before
modifying $MAIL to maildir or mbox etc. to dovecot imap?

http://wiki.dovecot.org/MailLocation

mbox and maildir are well know standards I guess but dovecot has their
own called dbox (http://wiki.dovecot.org/MailboxFormat/dbox). Do you
plan to support such? Not a requirement since making mailfront + cvm
natively support dovecot is primary goal without excessive
hacking/patching will benefit other users :)

Post by Bruce Guenter
I see by the wiki that this works for version 2 as well, and also
supports mbox files. I will add a stat to the imapfront-auth code to
handle both cases.

That is my next task to test for version 2 actually :p

If their version 2 are using $MAIL for mail_location (same as their
version 1 which I guess should be) and with double colons "::" escape
feature in their version 2 as well, I don't see why won't work :p

If you are interested, I can report back my progress about the test for
their version 2?

Many thanks for your prompt response!

Kindest regards,
Giam Teck Choon

[Bruce Guenter]

Post by Giam Teck Choon
So how you stat the two
format? Just curious. By using the $SETUP_DOVECOT variable? Like
SETUP_DOVECOT=mbox|maildir? Or simply stat the $MAIL to see it is a
directory or a file to differential maildir or mbox respectively before
modifying $MAIL to maildir or mbox etc. to dovecot imap?

That's exactly what I've done now. Directory is assumed to be maildir,
anything else (including missing file) is assumed to be mbox.

Post by Giam Teck Choon
mbox and maildir are well know standards I guess but dovecot has their
own called dbox (http://wiki.dovecot.org/MailboxFormat/dbox). Do you
plan to support such?

Not planned, no, though I could be persuaded.

Post by Giam Teck Choon
If you are interested, I can report back my progress about the test for
their version 2?

Definitely, please do.

[Giam Teck Choon]

Post by Bruce Guenter

Post by Giam Teck Choon
So how you stat the two
format? Just curious. By using the $SETUP_DOVECOT variable? Like
SETUP_DOVECOT=mbox|maildir? Or simply stat the $MAIL to see it is a
directory or a file to differential maildir or mbox respectively before
modifying $MAIL to maildir or mbox etc. to dovecot imap?

That's exactly what I've done now. Directory is assumed to be maildir,
anything else (including missing file) is assumed to be mbox.

Ok.

Post by Bruce Guenter

Post by Giam Teck Choon
mbox and maildir are well know standards I guess but dovecot has their
own called dbox (http://wiki.dovecot.org/MailboxFormat/dbox). Do you
plan to support such?

Not planned, no, though I could be persuaded.

Post by Giam Teck Choon
If you are interested, I can report back my progress about the test for
their version 2?

Definitely, please do.

Things not what we expected. Their version 2.0.7 not honouring $MAIL at
all. I will keep on testing various environment variables and report
back if I find out more.

Thanks.

Kindest regards,
Giam Teck Choon

[Giam Teck Choon]

SUCCESS with v2.0.7... Many thanks to Timo from dovecot.org :)

Things not what we expected. Their version 2.0.7 not honouring $MAIL at
all. I will keep on testing various environment variables and report
back if I find out more.

Wrapper below for dovecot v2.0.7

--------------------8<----------8<----------8<----------8<---------------------
#!/bin/sh

if [ -n "$MAIL" ] ; then
export MAIL=maildir:`echo $MAIL | sed 's@:@::@g'`
fi
exec "$@" -O -o mail_location=$MAIL
--------------------8<----------8<----------8<----------8<---------------------

Thanks.

Kindest regards,
Giam Teck Choon

[Bruce Guenter]

Post by Giam Teck Choon
SUCCESS with v2.0.7... Many thanks to Timo from dovecot.org :)

Things not what we expected. Their version 2.0.7 not honouring $MAIL at
all. I will keep on testing various environment variables and report
back if I find out more.

So you could replace the $IMAPD part of the original script with:

sh -c "exec $IMAPD -O -o mail_location=\$MAIL"

so it would end up running:

tcpserver [options] \
imapfront-auth \
sh -c 'exec /usr/libexec/dovecot/imap -O -o mail_location=$MAIL'

making sh do the substitution of $MAIL. No wrapper script should be
needed for that.

[Giam Teck Choon]

Post by Bruce Guenter

Post by Giam Teck Choon
SUCCESS with v2.0.7... Many thanks to Timo from dovecot.org :)

Things not what we expected. Their version 2.0.7 not honouring $MAIL at
all. I will keep on testing various environment variables and report
back if I find out more.

sh -c "exec $IMAPD -O -o mail_location=\$MAIL"
tcpserver [options] \
imapfront-auth \
sh -c 'exec /usr/libexec/dovecot/imap -O -o mail_location=$MAIL'
making sh do the substitution of $MAIL. No wrapper script should be
needed for that.

If v1.2.16 not support the options? I haven't test that in depth but
mostly will not. I asked Timo and still waiting for reply. If v1 not
supporting those options, a wrapper is needed.

On my choon.net server:
# /usr/libexec/dovecot/imap -O -o mail_location=maildir:/tmp/haha
* PREAUTH [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID
ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS MULTIAPPEND
UNSELECT IDLE CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1
CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH
LIST-STATUS] Logged in as root
* BYE Server shutting down.
Info: Server shutting down. bytes=0/332
# /usr/sbin/dovecot --version
2.0.7

On my client server:
# /usr/libexec/dovecot/imap -O -o mail_location=maildir:/tmp/haha
imap(root): Error: Ambiguous mail location setting, don't know what to
do with it: /root/Maildir/ (try prefixing it with mbox: or maildir:)
imap(root): Fatal: Mail storage creation failed with mail_location:
/root/Maildir/
# /usr/sbin/dovecot --version
1.0.15

So I guess a wrapper is needed. I need a generic imapd/run script which
support both dovecot versions (v1 and v2) and courier-imap. If just for
a specific imap server like dovecot v2 then wrapper not needed like what
you stated.

Wrapper as below to support dovecot v1 and v2:
--------------------8<----------8<----------8<----------8<---------------------
#!/bin/sh

if [ -n "$MAIL" ] ; then
export MAIL=maildir:`echo $MAIL | sed 's@:@::@g'`
fi
# we need to know dovecot is v1 or v2
# as v1 might not be supporting the options
# -O -o mail_location=... ...
DOVECOTIMAPOPTIONS=""
if [ -x "/usr/sbin/dovecot" ] ; then
dovecotversion=`/usr/sbin/dovecot --version | cut -f1 -d '.'`
if [ "x$dovecotversion" == "x2" ] ; then
DOVECOTIMAPOPTIONS="-O -o mail_location=$MAIL"
fi
fi
exec "$@" $DOVECOTIMAPOPTIONS
--------------------8<----------8<----------8<----------8<---------------------

Thanks.

Kindest regards,
Giam Teck Choon

[Bruce Guenter]

Post by Giam Teck Choon
If v1.2.16 not support the options? I haven't test that in depth but
mostly will not. I asked Timo and still waiting for reply. If v1 not
supporting those options, a wrapper is needed.

Right, if you want to support both version 1 and 2 with the same script,
some kind of wrapper will be needed to autodetect the version. I was
thinking of just fixing on one version and saying the other is
unsupported.

[Giam Teck Choon]

Post by Bruce Guenter

Post by Giam Teck Choon
If v1.2.16 not support the options? I haven't test that in depth but
mostly will not. I asked Timo and still waiting for reply. If v1 not
supporting those options, a wrapper is needed.

Right, if you want to support both version 1 and 2 with the same script,
some kind of wrapper will be needed to autodetect the version. I was
thinking of just fixing on one version and saying the other is
unsupported.

Ok. I am going to update my C wrapper which also intend to support your
$USE_DOVECOT setting. Are you intend to release an update release if
you are going to use $USE_DOVECOT or other name?

I read at your github source at
https://github.com/bruceg/mailfront/blob/master/imapfront-auth.c which I
can't find such commit.

Thanks.

Kindest regards,
Giam Teck Choon

[Bruce Guenter]

Post by Giam Teck Choon
Ok. I am going to update my C wrapper which also intend to support your
$USE_DOVECOT setting. Are you intend to release an update release if
you are going to use $USE_DOVECOT or other name?

I am planning to put out a new release with this feature shortly. I am
still trying to figure how exactly to switch this feature on. I'm
leaning towards setting something like SETUP_ENV=dovecot, which would
leave the option for setting up a different kind of environment for some
other IMAP back-end.

Post by Giam Teck Choon
I read at your github source at
https://github.com/bruceg/mailfront/blob/master/imapfront-auth.c which I
can't find such commit.

I haven't pushed it out yet as I don't consider it finalized yet.

[Giam Teck Choon]

Post by Bruce Guenter

Post by Giam Teck Choon
Ok. I am going to update my C wrapper which also intend to support your
$USE_DOVECOT setting. Are you intend to release an update release if
you are going to use $USE_DOVECOT or other name?

I am planning to put out a new release with this feature shortly. I am
still trying to figure how exactly to switch this feature on. I'm
leaning towards setting something like SETUP_ENV=dovecot, which would
leave the option for setting up a different kind of environment for some
other IMAP back-end.

Ok. I have already updated my C wrapper. Instead of checking what name
($USE_DOVECOT or $SETUP_ENV) you use, I will just check $MAIL to see
whether ^maildir: or ^mbox: instead. If found, modifying $MAIL will be
skipped.

Post by Bruce Guenter

Post by Giam Teck Choon
I read at your github source at
https://github.com/bruceg/mailfront/blob/master/imapfront-auth.c which I
can't find such commit.

I haven't pushed it out yet as I don't consider it finalized yet.

Oh ok. From your bumped version to 1.18 message, I can read your
intention :)

Thanks.

Kindest regards,
Giam Teck Choon

[Giam Teck Choon]

For the benefit of others, I will just update the list members as
below. Sorry in advance if my posting pissed off anyone (if any).

Why version 2 needs -O -o mail_location=... ... options? See:

http://dovecot.org/pipermail/dovecot/2010-November/054780.html

If v1.2.16 not support the options? I haven't test that in depth but
mostly will not. I asked Timo and still waiting for reply. If v1 not
supporting those options, a wrapper is needed.

Already tested. Their v1.2.16 not support those options and has been
confirmed by Timo from dovecot.org (thanks to Timo for the
confirmation/reply):

http://dovecot.org/pipermail/dovecot/2010-November/054784.html

So I guess a wrapper is needed. I need a generic imapd/run script which
support both dovecot versions (v1 and v2) and courier-imap. If just for
a specific imap server like dovecot v2 then wrapper not needed like what
you stated.

Updated sh wrapper to support v1 and v2 and also see whether $MAIL is
prefixed with maildir or mbox before modifying $MAIL accordingly. This
wrapper also check $MAIL to see whether it is a directory to assume it
is maildir or otherwise assume it is mbox:
--------------------8<----------8<----------8<----------8<---------------------
#!/bin/sh

DOVECOTIMAPOPTIONS=""
if [ -n "$MAIL" ] ; then
# test to see whether maildir or mbox is prefixed in $MAIL
checkmaildir=`grep -E '^maildir:' $MAIL | grep -v grep`
checkmbox=`grep -E '^mbox:' $MAIL | grep -v grep`
if [ ! -n "$checkmaildir" ] && [ ! -n "$checkmbox" ] ; then
# if mailbox is a directory, assume it is maildir
# otherwise assume it is mbox
if [ -d "$MAIL" ] ; then
export MAIL=maildir:`echo $MAIL | sed 's@:@::@g'`
else
export MAIL=mbox:`echo $MAIL | sed 's@:@::@g'`
fi
fi
# we need to know dovecot is v1 or v2
# as v1 is not supporting the options
# -O -o mail_location=... ...
if [ -x "/usr/sbin/dovecot" ] ; then
dovecotversion=`/usr/sbin/dovecot --version | cut -f1 -d '.'`
if [ "x$dovecotversion" == "x2" ] ; then
DOVECOTIMAPOPTIONS="-O -o mail_location=$MAIL"
fi
fi
fi
exec "$@" $DOVECOTIMAPOPTIONS
--------------------8<----------8<----------8<----------8<---------------------

My C wrapper also updated and can be found at:

http://choon.net/mailfront-imapfront-auth-dovecot.php

Hopefully the above information is useful to someone ;)

Thanks.

Kindest regards,
Giam Teck Choon

[Giam Teck Choon]

Post by Giam Teck Choon
Updated sh wrapper to support v1 and v2 and also see whether $MAIL is
prefixed with maildir or mbox before modifying $MAIL accordingly. This
wrapper also check $MAIL to see whether it is a directory to assume it

Sorry, below is the updated sh wrapper as I should not grep $MAIL
variable without echo it as it isn't a file:

--------------------8<----------8<----------8<----------8<---------------------
#!/bin/sh

DOVECOTIMAPOPTIONS=""
if [ -n "$MAIL" ] ; then
# test to see whether maildir or mbox is prefixed in $MAIL
checkmaildir=`echo $MAIL | grep -E '^maildir:'`
checkmbox=`echo $MAIL | grep -E '^mbox:'`
if [ ! -n "$checkmaildir" ] && [ ! -n "$checkmbox" ] ; then
# if mailbox is a directory, assume it is maildir
# otherwise assume it is mbox
if [ -d "$MAIL" ] ; then
export MAIL=maildir:`echo $MAIL | sed 's@:@::@g'`
else
export MAIL=mbox:`echo $MAIL | sed 's@:@::@g'`
fi
fi
# we need to know dovecot is v1 or v2
# as v1 is not supporting the options
# -O -o mail_location=... ...
if [ -x "/usr/sbin/dovecot" ] ; then
dovecotversion=`/usr/sbin/dovecot --version | cut -f1 -d '.'`
if [ "x$dovecotversion" == "x2" ] ; then
DOVECOTIMAPOPTIONS="-O -o mail_location=$MAIL"
fi
fi
fi
exec "$@" $DOVECOTIMAPOPTIONS
--------------------8<----------8<----------8<----------8<---------------------

Thanks.

Kindest regards,
Giam Teck Choon