When trying to handle inbound email to roundup running in a docker container, there are
a few ways to do it.

One way is to mount the host's mail delivery directory (e.g. /var/mail, /var/spool/mail)
into the container. then devise some sort of cron command inside the container to process
the mbox file.

This isn't great as:
  1) processing time is delayed by the cron schedule
  2) other mailboxes on the host are exposed to the container
  3) manipulating an mbox file requires a lock so mail isn't lost. Locking can be
     an issue when crossing docker/host boundaries. (flock(2) should work, will dot locking?)

My thought was to spin up a simple smtp server using aiosmtpd 
(https://pypi.org/project/aiosmtpd/). It has a handler
called Mailbox, it only writes output in Maildir
https://docs.python.org/2.7/library/mailbox.html#maildir format.

Adding maildir format mbox to roundup-mailgw should be a nice self contained
bit of work. The Python mailbox module supports Maildir.

There are a few ways this could work:


  1 a python script running the aiosmtpd server triggers processing of all messages
    in a maildir folder when one is received

    Locking of the file is only required between roundup-mailgw processes. aiosmtpd
    will only create new files atomically in the Maildir folder named "new".

    Note that the lock/unlock methods for mailbox.Maildir are no-ops, so some locking
    will be needed to prevent 2 roundup-mailgw from processing the same file. This could
    be in aiosmtpd's triggering code if it tracks the exit status of its one and only
    child roundup-mailgw.

    This cleans up any abandoned emails the next time it is invoked, unless one of the
    emails causes roundup-mailgw to crash.

  2 a python script runs aiosmtpd with a handler to call roundup-mailgw
    feeding it the message on stdin (pipe mode). (a pipeto handler)

    This removes the need for locking since the roundup-mailgw process will only
    see one message. But this can be more complicated as the aiosmtpd has to monitor/
    interact with the child.

    A roundup-mailgw crash could be returned to the sender. So this might be better at
    exposing issues. Also this requires no change to roundup-mailgw. The existing pipe
    mechanism should work.

    I do not expect a file to be written by aiosmtpd, so there is nothing to clean up.

  3 use aiosmtpd to trigger roundup-mailgw to process and delete a specific new message
    in the 'new' maildir subfolder.

    This could leave behind abandoned files in the Maildir directory.

The mail spool for aiosmtpd's Maildir can be local and ephemeral (e.g. /var/tmp/Maildir)
or backed from a volume.

The tracker home directory would need to be imported into the roundup-mailgw container.
Also aiosmtp just delivers all the email to a single location. So multiple
aiosmtp/roundup-mailgw dockers would have to be configured as the routing endpoints for a single
multi-tracker docker installation. (This might not be true with sufficient programming of 
aiohttpd, but is true for naive use.) 

For a single instance tracker, an implementation of this might be small enough to work
in the main docker container. If an admin doesn't want it, just don't expose the port, or
place a flag: .nomail file in the tracker home??

Supporting Maildir by itself would be useful for other MTA's that store their results
in Maildir format (e.g courier or qmail).

On Sun, Nov 19, 2023 at 05:35:37AM +0000, John Rouillard wrote:

> My thought was to spin up a simple smtp server using aiosmtpd 
> (https://pypi.org/project/aiosmtpd/). It has a handler
> called Mailbox, it only writes output in Maildir
> https://docs.python.org/2.7/library/mailbox.html#maildir format.

How about using an IMAP port external to the container for retrieving
mail? That can either be in its own container or be the mailserver of
the organisation, or a cloud, ...

Ralf
-- 
Dr. Ralf Schlatterbeck                  Tel:   +43/2243/26465-16
Open Source Consulting                  www:   www.runtux.com
Reichergasse 131, A-3411 Weidling       email: office@runtux.com

Hi Ralf:

In message <20231119105314.pkrr3v6drdyjyt5x@runtux.com>,
Ralf Schlatterbeck writes:
>On Sun, Nov 19, 2023 at 05:35:37AM +0000, John Rouillard wrote:
>How about using an IMAP port external to the container for retrieving
>mail? That can either be in its own container or be the mailserver of
>the organisation, or a cloud, ...

Yup, using imap or pop are some of the options I mention at:

  https://roundup-tracker.org/docs/installation.html#docker-support

But how do you trigger the call to roundup-mailgw? Some service has to
be running to schedule/trigger roundup-mailgw. Running cron as root
inside the container is problematic as is running cron as non-root.

You can use cron on the docker host (assuming it's not just a
dedicated docker/kubernetes host environment) to run:

   docker exec roundup-tracker roundup-mailgw tracker \
         imaps  ...

periodically, but...

Neither of those runs roundup-mailgw in response to an arriving email
event.

So I thought why not run an smtp frontend that drives the
roundup-mailgw on email arrival. The docker container hosting the smtp
server (really receiver) acts similar to a kubernetes sidecar.

On Sun, Nov 19, 2023 at 04:56:27PM +0000, John Rouillard wrote:
> But how do you trigger the call to roundup-mailgw? Some service has to
> be running to schedule/trigger roundup-mailgw. Running cron as root
> inside the container is problematic as is running cron as non-root.

> You can use cron on the docker host (assuming it's not just a
> dedicated docker/kubernetes host environment) to run:
> 
>    docker exec roundup-tracker roundup-mailgw tracker \
>          imaps  ...
> 
> periodically, but...

Yes, this is the method I'm using in one of the containerized instances
I'm running.

> Neither of those runs roundup-mailgw in response to an arriving email
> event.

Yes, but this is the same for other imap installations.

Running the mailgw inside the delivery chain is problematic since
maintenance of the system is harder. And I don't know about
infrastructures where this is possible anymore, most companies (even
small ones) have a dedicated mail server and will not be able to deliver
some mailboxes via a dedicated transport.

> So I thought why not run an smtp frontend that drives the
> roundup-mailgw on email arrival. The docker container hosting the smtp
> server (really receiver) acts similar to a kubernetes sidecar.

Could be possible but this is not a use-case I see in my installations.
So we're usually stuck with polling the mail system via imap (or even
pop3).

But if this is a use-case you're seeing it's certainly a good idea to
make it work...

Ralf
-- 
Dr. Ralf Schlatterbeck                  Tel:   +43/2243/26465-16
Open Source Consulting                  www:   www.runtux.com
Reichergasse 131, A-3411 Weidling       email: office@runtux.com