Roundup Tracker - Issues

Message7860

Author rouilj
Recipients rouilj
Date 2023-11-19.05:35:37
Message-id <1700372137.69.0.91034709461.issue2551301@roundup.psfhosted.org>
In-reply-to
When trying to handle inbound email to roundup running in a docker container, there are
a few ways to do it.

One way is to mount the host's mail delivery directory (e.g. /var/mail, /var/spool/mail)
into the container. then devise some sort of cron command inside the container to process
the mbox file.

This isn't great as:
  1) processing time is delayed by the cron schedule
  2) other mailboxes on the host are exposed to the container
  3) manipulating an mbox file requires a lock so mail isn't lost. Locking can be
     an issue when crossing docker/host boundaries. (flock(2) should work, will dot locking?)

My thought was to spin up a simple smtp server using aiosmtpd 
(https://pypi.org/project/aiosmtpd/). It has a handler
called Mailbox, it only writes output in Maildir
https://docs.python.org/2.7/library/mailbox.html#maildir format.

Adding maildir format mbox to roundup-mailgw should be a nice self contained
bit of work. The Python mailbox module supports Maildir.

There are a few ways this could work:


  1 a python script running the aiosmtpd server triggers processing of all messages
    in a maildir folder when one is received

    Locking of the file is only required between roundup-mailgw processes. aiosmtpd
    will only create new files atomically in the Maildir folder named "new".

    Note that the lock/unlock methods for mailbox.Maildir are no-ops, so some locking
    will be needed to prevent 2 roundup-mailgw from processing the same file. This could
    be in aiosmtpd's triggering code if it tracks the exit status of its one and only
    child roundup-mailgw.

    This cleans up any abandoned emails the next time it is invoked, unless one of the
    emails causes roundup-mailgw to crash.

  2 a python script runs aiosmtpd with a handler to call roundup-mailgw
    feeding it the message on stdin (pipe mode). (a pipeto handler)

    This removes the need for locking since the roundup-mailgw process will only
    see one message. But this can be more complicated as the aiosmtpd has to monitor/
    interact with the child.

    A roundup-mailgw crash could be returned to the sender. So this might be better at
    exposing issues. Also this requires no change to roundup-mailgw. The existing pipe
    mechanism should work.

    I do not expect a file to be written by aiosmtpd, so there is nothing to clean up.

  3 use aiosmtpd to trigger roundup-mailgw to process and delete a specific new message
    in the 'new' maildir subfolder.

    This could leave behind abandoned files in the Maildir directory.

The mail spool for aiosmtpd's Maildir can be local and ephemeral (e.g. /var/tmp/Maildir)
or backed from a volume.

The tracker home directory would need to be imported into the roundup-mailgw container.
Also aiosmtp just delivers all the email to a single location. So multiple
aiosmtp/roundup-mailgw dockers would have to be configured as the routing endpoints for a single
multi-tracker docker installation. (This might not be true with sufficient programming of 
aiohttpd, but is true for naive use.) 

For a single instance tracker, an implementation of this might be small enough to work
in the main docker container. If an admin doesn't want it, just don't expose the port, or
place a flag: .nomail file in the tracker home??

Supporting Maildir by itself would be useful for other MTA's that store their results
in Maildir format (e.g courier or qmail).
History
Date User Action Args
2023-11-19 05:35:37rouiljsetrecipients: + rouilj
2023-11-19 05:35:37rouiljsetmessageid: <1700372137.69.0.91034709461.issue2551301@roundup.psfhosted.org>
2023-11-19 05:35:37rouiljlinkissue2551301 messages
2023-11-19 05:35:37rouiljcreate