Roundup Tracker - Issues

Issue 651409

classification
Indexer doesn't cope with batches
Type: Severity: normal
Components: Database Versions:
process
Status: closed fixed
:
: richard : richard, rochecompaan
Priority: normal :

Created on 2002-12-10 12:53 by rochecompaan, last changed 2002-12-12 09:33 by richard.

Messages
msg531 Author: [hidden] (rochecompaan) Date: 2002-12-10 12:53
The roundup indexer doesn't seem to cope with batch
submissions. A common example of when issues will be
submitted as a batch would be when I create a whole
bunch of issues while I am working offline and submit
all of them at once the next time I go online.

I get 2 common tracebacks:

Traceback1:

----  traceback of failure  ----
Traceback (most recent call last):
  File
"/usr/lib/python2.1/site-packages/roundup/mailgw.py",
line 248, in handle_Message
    return self.handle_message(message)
  File
"/usr/lib/python2.1/site-packages/roundup/mailgw.py",
line 821, in handle_message
    self.db.commit()
  File
"/usr/lib/python2.1/site-packages/roundup/backends/back_anydbm.py",
line 584, in commit
    self.indexer.save_index()
  File
"/usr/lib/python2.1/site-packages/roundup/indexer.py",
line 311, in save_index
    os.chmod(filename, 0664)
OSError: [Errno 2] No such file or directory:
'/home/httpd/roundup/checkins/db/indexes/index.db8'


Traceback 2:

----  traceback of failure  ----
Traceback (most recent call last):
  File
"/usr/lib/python2.1/site-packages/roundup/mailgw.py",
line 248, in handle_Message
    return self.handle_message(message)
  File
"/usr/lib/python2.1/site-packages/roundup/mailgw.py",
line 821, in handle_message
    self.db.commit()
  File
"/usr/lib/python2.1/site-packages/roundup/backends/back_anydbm.py",
line 581, in commit
    self.getclass(classname).index(nodeid)
  File
"/usr/lib/python2.1/site-packages/roundup/backends/back_anydbm.py",
line 1953, in index
    mime_type)
  File
"/usr/lib/python2.1/site-packages/roundup/indexer.py",
line 76, in add_text
    self.load_index()
  File
"/usr/lib/python2.1/site-packages/roundup/indexer.py",
line 259, in load_index
    pickle_str = zlib.decompress(f.read())
error: Error -5 while decompressing data

msg532 Author: [hidden] (richard) Date: 2002-12-11 01:53
Logged In: YES 
user_id=6405

How are you performing the batch submissions? Could you provide a code 
snippet? 
 
msg533 Author: [hidden] (rochecompaan) Date: 2002-12-11 08:16
Logged In: YES 
user_id=354804

I simply add a number of issues with my mail client while I
am offline. When I go online again all the issues in my
outbox are sent all at once. I guess the same would happen
if multiple users submit issues at the same time.

This error should be reproducable if you submit about 10 or
more issues from your mail client all at once.
msg534 Author: [hidden] (richard) Date: 2002-12-11 09:48
Logged In: YES 
user_id=6405

Sounds like a locking problem. Yucch. I guess the instance-lock has just got 
more important. 
 
 
msg535 Author: [hidden] (richard) Date: 2002-12-12 09:33
Logged In: YES 
user_id=6405

I've implemented whole-instance locking which should fix this problem. 
History
Date User Action Args
2002-12-10 12:53:47rochecompaancreate