Roundup Tracker - Issues

Issue 2551053

classification
REST-API: Regular expressions used as keys in dictionary
Type: Severity: normal
Components: Versions:
process
Status: closed fixed
:
: schlatterbeck : rouilj, schlatterbeck, tttech-klonner
Priority: :

Created on 2019-08-12 16:37 by schlatterbeck, last changed 2019-08-13 07:49 by schlatterbeck.

Messages
msg6590 Author: [hidden] (schlatterbeck) Date: 2019-08-12 16:37
The __route_map dictionary takes regular expressions as keys of a
dictionary. Now the regular expression implementation (at least in
python2) has a cache, so usually returns the same object for the same
regex (plus options) for calls to re.compile.

But this doesn't seem to work in all cases, we're observing a dictionary
with multiple entries for the *same* regular expression with different
methods in a WSGI process in apache, e.g.

^rest/data/([\\w.\\-~!$&'()*+,;=:\\%%]+)$ ['POST', 'GET']
^rest/data/([\\w.\\-~!$&'()*+,;=:\\%%]+)$ ['OPTIONS', 'DELETE']

(this prints the .pattern of the compiled regex plus the options stored
under that key)

The result is that we sometimes get an error message "Method not
allowed" for a call to PUT (or GET) on a path that should allow this method.

A simple test that bypasses the regex cache (by clearing it between
invocations) shows that compiled regular expression objects should not
be used as keys in a dictionary (at least not in Python2):

>>> import re
>>> r1 = re.compile ('a')
>>> re.purge ()
>>> r2 = re.compile ('a')
>>> r1 == r2
False
>>> d = dict (r1 = r1, r2 = r2)
>>> d
{'r1': <_sre.SRE_Pattern object at 0x7f4a0aa19d78>, 'r2':
<_sre.SRE_Pattern object at 0x7f4a0aa19e00>}

Thanks to Robert Klonner for debugging it so far that I could make this
bug-report. We're working on a fix, too, so no need to look into this
too soon.
msg6591 Author: [hidden] (schlatterbeck) Date: 2019-08-13 07:49
Fixed in commit 167ef847fcdf
Now the key in the dictionary is the uncompiled pattern.
The dict value is now a tuple with the compiled regex as the first item,
the second tuple item is the previous method dictionary.
History
Date User Action Args
2019-08-13 07:49:29schlatterbecksetstatus: new -> closed
nosy: + rouilj
resolution: fixed
messages: + msg6591
2019-08-13 06:17:20tttech-klonnersetnosy: + tttech-klonner
2019-08-12 16:37:54schlatterbeckcreate