Roundup Tracker - Issues

Issue 2551224

classification
Replace dbm db for sessions and otks when using sqlite
Type: resource usage Severity: normal
Components: Database Versions: 2.3.0, 2.2.0
process
Status: fixed fixed
:
: rouilj : rouilj, schlatterbeck
Priority: normal :

Created on 2022-07-25 01:04 by rouilj, last changed 2022-07-26 22:46 by rouilj.

Messages
msg7618 Author: [hidden] (rouilj) Date: 2022-07-25 01:04
Issue 729889 mentions that dbm access relies on a global tracker lock. In issue 2551036
on rest rate limiting, I mention a 10% non-record rate possibly caused by failure
to get a lock or other issue.

So it seems we have a performance issue with DBM. SQLite used to be used for sessions/otks
db's. However there is only one handle to the database. So flushing CSRF keys when used
caused data to commit when it shouldn't have (issue2550955). On MySQL/PostgreSQL opening
a second connection to the db for session/otks soved the issue. That's not possible with
SQLite. At that point, session_dbm was paired with back_sqlite to handle the otks/sessions
database.

I have a working implementation (its passing the test suite) that uses sqlite
databases db-otk and db-session for the respective storage.

Will be testing for performance issues on my system.
Working on improving the sessions test suite but that will be the subject of 
issue2551223.
msg7619 Author: [hidden] (rouilj) Date: 2022-07-25 04:32
Question do we need a migration plan for the data from the dbm files to sqlite?
The session and otk data is ephemeral so I am inclined to not provide a way to migrate.
msg7622 Author: [hidden] (rouilj) Date: 2022-07-26 22:40
See: changeset:   6808:375d40a9e730
msg7623 Author: [hidden] (rouilj) Date: 2022-07-26 22:42
Sorry, previous msg pointed to wrong commit:

    changeset:   6804:25d08e15e3b4

is the right changeset.

The commit in the previous msg completed testing
on the BasicDatabase underlying otks/sessions.
History
Date User Action Args
2022-07-26 22:46:05rouiljsetversions: + 2.2.0
2022-07-26 22:42:41rouiljsetmessages: + msg7623
2022-07-26 22:40:18rouiljsetstatus: open -> fixed
resolution: fixed
messages: + msg7622
versions: + 2.3.0, - 2.2.0
2022-07-25 04:32:06rouiljsetnosy: + schlatterbeck
messages: + msg7619
2022-07-25 01:05:06rouiljsetstatus: new -> open
2022-07-25 01:04:56rouiljcreate