Roundup Tracker - Issues

Issue 2551309

classification
Built-in mailbox polling scheduler
Type: behavior Severity: normal
Components: Mail interface Versions: devel, 2.4.0
process
Status: new
:
: : asavchuk, rouilj
Priority: :

Created on 2023-12-14 23:10 by asavchuk, last changed 2023-12-15 00:29 by rouilj.

Messages
msg7887 Author: [hidden] (asavchuk) Date: 2023-12-14 23:10
Hello! I'm also interested in polling IMAP mailboxes when using containerized Roundup. Documentation says the only way is using cron tasks. 

I think I have an idea. It would be possible to implement built-in scheduler using native Python sched module. In this case, all the necessary values can be stored in the Roundup configuration file (or as secrets) and the roundup.mailgw module will be launched directly by the main Roundup process.

This could solve the permissions issues and the like.

What do you think?
msg7888 Author: [hidden] (rouilj) Date: 2023-12-15 00:29
Hi Anton:

In message <1702595412.18.0.970424296848.issue2551309@roundup.psfhosted.org>,
Anton Savchuk writes:
>Hello! I'm also interested in polling IMAP mailboxes when using
>containerized Roundup. Documentation says the only way is using cron tasks. 
>
>I think I have an idea. It would be possible to implement built-in
>scheduler using native Python sched module.

How would sched be invoked? Out of the main loop in the server, a
separate thread, a subprocess started from the main server?

Running roundup-server in the current containerization was a "get
something running" attempt.  For real deployments (50+
connections/sec) running Roundup via wSGI under gunicorn (as is done
with bugs.python.org) is the "production" deployment method.

Will using the sched module work if there is no master roundup-server?
IIUC wSGI instances basically sit idle until triggered from
upstream. I'm not sure how only one instance of sched can be run in
such an environment.

Also roundup-server can run in either forking or threaded mode. By
default we chose forking (multiprocess). In the future this may be
extended to include a subinterpreter method rather than a fork. Can
sched work in a threaded environment? (There is a report that threads
respond faster (2x+) under light load, but the response times increase
faster than the forking model under load. Why I'm not sure.)

I'd like to use subinterpreters in roundup-server when it is
implemented in a future Python. It will make roundup-server scale
better (not to gunicorn levels but) and be usable in more scenarios.
But it is too early to understand how sched and subinterpreter will
co-exist.

https://tonybaloney.github.io/posts/sub-interpreter-web-workers.html
https://realpython.com/python312-subinterpreters/
https://peps.python.org/pep-0734/

Hopefully subinterpreter will be like multiprocess.

Thoughts?

Have a great weekend.
History
Date User Action Args
2023-12-15 00:29:59rouiljsetnosy: + rouilj
messages: + msg7888
2023-12-14 23:10:12asavchukcreate