Roundup Tracker - Issues

Issue 980314

classification
full text search failure after import
Type: Severity: normal
Components: None Versions:
process
Status: closed fixed
:
: : qual, richard
Priority: normal :

Created on 2004-06-26 15:54 by qual, last changed 2004-06-28 23:03 by richard.

Files
File name Uploaded Description Edit Remove
rdbms_common.py_patch_against_075 qual, 2004-06-26 22:32
rdbms_common.py.diff qual, 2004-06-27 21:24
Messages
msg1322 Author: [hidden] (qual) Date: 2004-06-26 15:54
scenario:
crreated two identical trackers (backend sqlite)

1) created one issue with two messages in the first tracker
2) exported first tracker
3) import of the export in step 2) into the second tracker
4) added a second issue into second tracker 

a full text search now doesn't find words in the first
issue.
words occuring in the second issue are found during 
full text search

the command reindex issued in roundup-admin doesn't
change this scenario.

if you make a dump (after step 3 ) of the db file
via the sqlite command and compare the contents one gets
the following significant difference:

the db of the second tracker misses the following lines:

INSERT INTO __textids VALUES('msg',1,'summary',32);
INSERT INTO __textids VALUES('msg',1,'content',33);
INSERT INTO __textids VALUES('issue',1,'title',34);
INSERT INTO __textids VALUES('msg',2,'summary',35);
INSERT INTO __textids VALUES('msg',2,'content',36);

INSERT INTO __words VALUES('RGEFRDFEG',32.0);
INSERT INTO __words VALUES('RGEFRDFEG',33.0);
INSERT INTO __words VALUES('TTT',34.0);
INSERT INTO __words VALUES('SUCHEN',35.0);
INSERT INTO __words VALUES('EINS',36.0);
INSERT INTO __words VALUES('SUCHEN',36.0);
INSERT INTO __words VALUES('ZWEI',36.0);
msg1323 Author: [hidden] (qual) Date: 2004-06-26 22:32
Logged In: YES 
user_id=337146

here is a quick and dirty patch borrowing some stuff from
inner_create
msg1324 Author: [hidden] (qual) Date: 2004-06-27 16:10
Logged In: YES 
user_id=337146

reindex command:
if you apply the following diff:
2350c2350,2352
<             if isinstance(propclass, String) and
propclass.indexme:
---
>             if isinstance(propclass, String) \
>                    and ( propclass.indexme or
>                          ( self.classname == "msg" and
prop == content")):

the reindex command will index the contents of the messages.
Either I don't understand the purpose of the indexme
attribute or
there is something broken (other properties should be reindexed 
also, not only issues.title )
msg1325 Author: [hidden] (qual) Date: 2004-06-27 21:24
Logged In: YES 
user_id=337146

some changes after some more insights into the hyperdb.
this patch fixes the  missing index creation during import
and the "failing" reindex command.
a better mechanism for the failing reindex command would be a
change of the msg object in dbinit.py instead of the changes
of the
FileClass object.
I 
msg1326 Author: [hidden] (richard) Date: 2004-06-28 23:03
Logged In: YES 
user_id=6405

Thanks for chasing this down! 
 
Added, with a slight change so that we don't index all 
non-text/plain file content, so on the FileClass:: 
 
    def index(self, nodeid): 
        '''Add (or refresh) the node to search indexes. 
 
        Pass on the content-type property. 
        ''' 
        # find all the String properties that have indexme 
        for prop, propclass in self.getprops().items(): 
            if not isinstance(propclass, String) or not 
propclass.indexme: 
                continue 
 
            if prop == 'content': 
                mime_type = self.get(itemid, 'type') 
                if not mime_type: 
                    mime_type = self.default_mime_type 
            else: 
                mime_type = 'text/plain' 
            self.db.indexer.add_text((self.classname, nodeid, prop), 
                str(self.get(nodeid, prop)), mime_type) 
 
msg1327 Author: [hidden] (richard) Date: 2004-06-28 23:12
Logged In: YES 
user_id=6405

(the actual implementation was cleaner than this - see the 
eventual source if you're keen ;) 
History
Date User Action Args
2004-06-26 15:54:48qualcreate