Hi Ralf:

In message <20190220094433.72uymum546o72qsu@runtux.com>,
Ralf Schlatterbeck writes:
>On Tue, Feb 19, 2019 at 08:45:09PM -0500, John P. Rouillard wrote:
>> 
>> Does anybody know how to fix the Travis CI errors with mysql?
>> Jerrykan/John K do you have any ideas? It would be good to release the
>> current head as roundup 2.0.a1 once it passes CI to pypi.
>
>Do we have an issue for this, I have just looked and haven't found one.

We should as soon as this email hits the tracker 8-). I removed the
roundup-devel mailing list as the creation of the ticket will be
forwarded there and I didn't want to chance having the ML on the nosy
list.

>That said: I looked into the travis errors and investigated this and had
>written some emails about it so I would have to dig this out of my mail
>logs.
>
>The failing tests seem to be a configuration issue with mysql that
>should be fixable by changing the mysql config.

Yeah, I think the errors were happening during tear down. A quick
search indicated that the error could be caused by an oversized
packet. So the question is why did it all of sudden start, and how do
you change it in travis?

Have a great week.

In order to be able to help it would be nice to have links to the
current findings (ci, and Ralf's previous mails).

I was unable to find Ralf's emails in the list other than one note that
it should be fixable by changing mysql's my.cnf.

Looking at doc/mysql.txt, a fix for this is condition to change the
max_allowed_packet size setting. See also:

  
https://stackoverflow.com/questions/7942154/mysql-error-2006-mysql-server-has-gone-away

which indicates it could also be an out of memory condition, connection
limit or a timeout being hit.
The interactive_timeout and wait_timeout values are by default: 28800
seconds or 8 hours and the entire run
takes only 20 minutes. There are no settings for these vars in my.cnf
that would reduce the value.
Max_connections is 151 by default, again I don't think this is an issue.

I tried to edit /etc/mysql/my.cnf change that to 500M (from 16M) and
restart the server. This seems to have been done correctly according to
the log:

 https://travis-ci.org/roundup-tracker/roundup/builds/504322490

see lines: 5851 and 5927

but the 14 errors are still there.

Looking at the errors, some look like an example of issue2550743 as
these failing tests are indexer based:

test/test_indexer.py::mysqlIndexerTest::test_basics FAILED             
 [ 28%]
test/test_indexer.py::mysqlIndexerTest::test_casesensitity FAILED      
 [ 28%]
test/test_indexer.py::mysqlIndexerTest::test_change FAILED             
 [ 28%]
test/test_indexer.py::mysqlIndexerTest::test_clear FAILED              
 [ 28%]
test/test_indexer.py::mysqlIndexerTest::test_extremewords FAILED       
 [ 28%]
test/test_indexer.py::mysqlIndexerTest::test_manyresults FAILED        
 [ 28%]
test/test_indexer.py::mysqlIndexerTest::test_stopwords FAILED          
 [ 28%]
test/test_indexer.py::mysqlIndexerTest::test_wordsplitting FAILED      
 [ 28%]

but these failing tests/errors:

test/test_mysql.py::mysqlSessionTest::testDestroy FAILED               
 [ 65%]
test/test_mysql.py::mysqlSessionTest::testGetAll FAILED                
 [ 65%]
test/test_mysql.py::mysqlSessionTest::testList FAILED                  
 [ 66%]
test/test_mysql.py::mysqlSessionTest::testSetSession FAILED            
 [ 66%]
test/test_mysql.py::mysqlSessionTest::testUpdateSession FAILED         
 [ 66%]
test/test_mysql.py::mysqlSpecialActionTestCase::testInnerMain FAILED   
 [ 66%]
test/test_mysql.py::mysqlSpecialActionTestCase::testInnerMain ERROR    
 [ 66%]

aren't AFAIK indexer based.

The errors always come from:

    def sql_close(self):
        self.log_info('close')
        try:
>           self.conn.close()
E           OperationalError: (2006, '')
roundup/backends/back_mysql.py:589: OperationalError

So I am stumped.

I rolled back the mysqlclient installed by pip from 1.3.14 (breaks test) 
to 1.3.13 (tests pass).

I don't know what in 1.3.14 causes mysql to exit with a 2006 error on 
close. Nothing in the changelog stood out as a possible cause/change.

I reverted the pin to version 1.3.13 and we still have the same 
failure. Plus some new rest erro case failures. I have reinstalled
the pin.

I thought we may be double closing the connection, but by attempt to 
ignore the second close call if the handle was already dead didn;t make 
a difference.

I submitted this question to stack overflow to see if we can get some 
help.

https://stackoverflow.com/questions/58359938/error-2006-when-closing-
database-using-version-1-3-14-but-not-1-3-13-of-mysqlcli

Using the github tracker for this sort of question seems unwanted by 
the developer.

If anybody has an idea here I would appreciate it.

Hmm, Using:

https://github.com/PyMySQL/mysqlclient-
python/compare/1.3.13...1.3.14#diff-
b86fdc88461d69279e84018bca8ea267L118-L119

it looks like  a new case was added that results in 
CR_SERVER_GONE_ERROR which is what error 2006 is IIUC.

I wonder if I was on the right path and we are leaking a double close 
of the database through.

Looking at the sql_close code, it looks like ProgrammingError used to 
be raised with a connection closed message.

So I am trying to emulate the same for OperationalError with:

         self.log_info('close')                                                 
         try:                                                                   
             self.conn.close()                                                  
+        # issue2551025: with revision 1.3.14 of mysqlclient.                   
+        # It looks like you can get an OperationalError 2006                   
+        # raised for closing a closed handle.                                  
+        except MySQLdb.OperationalError as message:                            
+            print str(message)                                                 
+            if str(message) != '':                                             
+                raise                                                          
         except MySQLdb.ProgrammingError as message:                            
             if str(message) != 'closing a closed connection':                  
                 raise                                                          

I hope this will allow me to figure out what message to filter
on and suppress the exception.

Comments??

mysql tests are passing with mysqlclient 1.4.4

I ended up using:
 
   except MySQLdb.OperationalError as message:
       if str(message) != "(2006, '')":  # close connection                
           raise

Not sure if this is 100% correct but it appears to do the trick.

Marking issue pending. Anybody got feedback?

-- rouilj

No feedback on my patch. Closing as fixed.