Roundup Tracker - Issues

Issue 1195739

Title: search in russian does not work
Type: Severity: normal
Components: Web interface Versions:
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: a1s, ber, richard
Priority: low Keywords: patch

Created on 2005-05-05 08:24 by anonymous, last changed 2016-06-27 03:17 by rouilj.

File name Uploaded Description Edit Remove
roundup.diff anonymous, 2005-05-05 08:24
msg1946 Author: [hidden] (anonymous) Date: 2005-05-05 08:24
Search functionality does not work in russian. It is 
possible to correct this by using unicode features of 
python. See enclosed difference.
msg1947 Author: [hidden] (a1s) Date: 2005-05-06 06:43
Logged In: YES 

confirmed: current HEAD checkout does not find russian words

the patch fixes this problem for rdbms backend only, dbm and
xapian indexers are not affected.  perhaps i could port this
change to dbm backend.

more serious problem is that this patch contains hard-coded
charset, i.e. all database contents are assumed to be in
utf-8.  if we apply this patch, we lose the last chance to
ever support databases with different text encodings. (the
chances were not high anyway.)

Richard, do you think this is acceptable?
msg1948 Author: [hidden] (richard) Date: 2005-05-18 05:43
Logged In: YES 

This patch is as good as we're going to get for now. 
msg1949 Author: [hidden] (a1s) Date: 2005-05-22 18:32
Logged In: YES 

the patch is applied to HEAD and maint-0-8.

dbm indexer needs a bit more work: if i change text_splitter
to use unicode in a way similar to indexer_rdbms, dbm
indexer fails in .save_index() because there are no slots
for russian letters.

Richard, could you update docs, please?  i think we should
mention that from now on the database and all templates
*must* be in utf-8; other character sets are not and will
never be supported.
msg1950 Author: [hidden] (richard) Date: 2005-06-24 07:10
Logged In: YES 

Where would you recommend that I put that information? 
Have you tried to index utf-8 with the Xapian indexer? If that 
works, then I'd be happy to say that if people wish to index utf-8 
then they can't use the built-in dbm indexer, but need to use 
the Xapian indexer. 
Having said that, I actually don't have a problem with just telling 
people they need to use one of the RDBMS backends. The dbm 
backend is really just for playing around, not serious work... 
msg1951 Author: [hidden] (a1s) Date: 2005-07-05 06:58
Logged In: YES 

no, i didn't look at Xapian at all.

as for documentation update, it would be best to add a note
to three documents:
 * upgrade
 * html template editing
 * database setup
msg4614 Author: [hidden] (ber) Date: 2012-08-21 08:55
Reading up on the history. I believe this issues would need a retest.
Date User Action Args
2016-06-27 03:17:17rouiljsetkeywords: + patch
2012-08-21 08:55:36bersetassignee: richard ->
messages: + msg4614
nosy: + ber
2005-05-05 08:24:26anonymouscreate