Figured out how to test this. My original script using add-issue must
have been broken. I used the following:

#! /bin/sh
#set -xv 
for i in `seq 1 5`; do
    for j in `seq 1 5`; do
        PYTHONPATH=$PWD python scripts/add-issue demo urgent "title $i.$j" &
    done
    wait
done

to get the following exception:

Traceback (most recent call last):
  File "scripts/add-issue", line 53, in <module>
    messages=messages)
  File
"/home/rouilj/develop/roundup.dev/roundup.hg/roundup/backends/rdbms_common.py",
line 1524, in create
    newid = self.create_inner(**propvalues)
  File
"/home/rouilj/develop/roundup.dev/roundup.hg/roundup/backends/rdbms_common.py",
line 1668, in create_inner
    self.db.addnode(self.classname, newid, propvalues)
  File
"/home/rouilj/develop/roundup.dev/roundup.hg/roundup/backends/rdbms_common.py",
line 966, in addnode
    self.sql(sql, vals)
  File
"/home/rouilj/develop/roundup.dev/roundup.hg/roundup/backends/rdbms_common.py",
line 221, in sql
    cursor.execute(sql, args)
sqlite3.IntegrityError: UNIQUE constraint failed: _issue.id

which I think is the 2016 version of the original 2011 error.  I got
that exception 8 times running the script above (25 processes total).

After my patch, no failures using the loop above.

Run the script after increasing the number of processes and loops to
10.  The printing the last 20 issues sorted by creation date produces:

   title 9.8
   title 9.1
   title 9.5
   title 9.9
   title 9.3
   title 9.7
   title 9.4
   title 9.2
   title 9.10
   title 9.6
   title 10.4
   title 10.10
   title 10.1
   title 10.2
   title 10.5
   title 10.9
   title 10.8
   title 10.3
   title 10.6
   title 10.7

so all entries are present and accounted for, the start order is not
preserved, but I am ok with that. The default lock timeout set in
config.ini is 30 seconds, but I could set it to 10 seconds and get
each batch of 10 to complete. setting it to 9 produces at least 1

  sqlite3.OperationalError: database is locked

exception.

If the default timeout of 30 seconds needs to be increased (set in
config.RDBMS_SQLITE_TIMEOUT) because of timeout exceptions the user
probably needs to move to a better database with better concurrent
access (postgresql, mysql).

I was able to use a BEGIN IMMEDIATE transaction (rather than BEGIN
EXCLUSIVE in the attached patch). The immediate transaction allows
reads from the database but prevents another BEGIN IMMEDIATE or write
from occurring.  Since all reads for the newid occur inside an
IMMEDIATE transaction, other reads of the newid table can occur (if
there are any) that are outside of reserving/generating a newid.

Using immediate transactions seems to have reduced the impact by a
small amount over exclusive transactions. But the full test_sqlite
test suite takes 40m50s now compared to 31m5s without the transaction
code. Probably because of having to recreate the database for every
test that hits the newid code heavily. Note that it appears the
slowdown is due to i/o wait. A better disk subsystem should help. But
at least now it's correct.

Committed this patch as: 9cebf686c552