Issue 2550743
Created on 2012-01-30 15:43 by telsch, last changed 2016-07-09 18:26 by rouilj.
msg4492 |
Author: [hidden] (telsch) |
Date: 2012-01-30 15:43 |
|
When we try to reindex our roundup instance we get the error 'MySql
Server has gone away (2006)'
We found a quick fix, if we set in the configuration from mysql:
/etc/my.conf
[mysqld]
max_allowed_packet = 1M
the 'max_allowed_packet' value from '1M' to '64M'
i hope it is possible to reduce the transaction size, so we don't have
to configure the mysql for the roundup reindex.
|
msg5794 |
Author: [hidden] (rouilj) |
Date: 2016-07-09 18:26 |
|
Sorry this has taken so long to address.
Do you have large attached files that could be indexed?
In looking through the code (specifically
backends/indexer_rdbms.py::add_text, I can see getting
this error if you are indexing attachment contents.
It looks like it tries to insert all words in a document at a
single time using:
# for each word, add an entry in the db
sql = 'insert into __words (_word, _textid) values (%s, %s)'%(a, a)
words = [(word, id) for word in words]
self.db.cursor.executemany(sql, words)
While that could be split up into single execute commands or
smaller batches of X number of words, performance would suffer
as the python mysql driver generates a large insert with values
command to the server if I understand:
https://dev.mysql.com/doc/connector-python/en/connector-python-api-mysqlcursor-executemany.html
correctly.
We may accept a patch to make the number of inserts
configurable using a setting in the [rdbms] section of the
tracker config file. So if you develop one, please feel free
to open an issue and attach the patch. The default should
be an unlimited number of inserts.
If you can't increase the packet size, maybe some other search
engine (in 1.6.0 we support whoosh as well as xapian) would work.
I have added this problem to the documentation in mysql.txt under the
header: Other Configuration so others can benefit from your knowledge.
Thanks for the report and sorry this took so long.
-- rouilj
|
|
Date |
User |
Action |
Args |
2016-07-09 18:26:33 | rouilj | set | status: new -> closed resolution: wont fix messages:
+ msg5794 nosy:
+ rouilj |
2012-01-31 10:03:57 | telsch | set | type: crash |
2012-01-30 15:43:25 | telsch | create | |
|