Created on 2022-06-07 16:20 by marcus.priesch, last changed 2023-02-20 00:19 by rouilj.
|failed_submission_email.txt||marcus.priesch, 2022-06-07 16:20|
|message_to_mailgw.txt||marcus.priesch, 2022-06-07 16:20|
|msg7563||Author: [hidden] (marcus.priesch)||Date: 2022-06-07 16:20|
as requested per issue2551203, here are the requested email messages the problematic characters dropped into the mail by copy/paste from the devtools browser window. however, maybe the header line: Content-Type: text/plain; charset=UTF-8; format=flowed should have triggered conversion to utf-8 instead of ascii. when opening message_to_mailgw.txt in python with ascii codec, you get exactly the same error message: >>> open ("message_to_mailgw.txt","r",encoding="ascii").read() Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/usr/lib/python-exec/python3.9/../../../lib/python3.9/encodings/ascii.py", line 26, in decode return codecs.ascii_decode(input, self.errors) UnicodeDecodeError: 'ascii' codec can't decode byte 0xef in position 1817: ordinal not in range(128) thanks, marcus.
|msg7564||Author: [hidden] (marcus.priesch)||Date: 2022-06-07 16:25|
maybe next bug ... ;) there is only one attachment sent with the initial email although i added two attachments in the web-interface. regards, marcus.
|msg7565||Author: [hidden] (rouilj)||Date: 2022-06-07 20:24|
This is an interesting issue. Bern, Ralf this ticket has a message that when process by the mail gateway resulted in a UunicodeDecodeError with python 2.7 and roundup 1.6.1 (run as issues.roundup-tracker.org). I think this bug is fixed in the current development release as I don't get an error message when I run a current dev tracker under python 2 or 3. Does this look like a reasonable way to test. I generated the test message using: (echo From firstname.lastname@example.org; curl https://issues.roundup- tracker.org/file1789/message_to_mailgw.txt ) | sed -e 's/\[issue[0-9]*]/[issue1]/' -e 'email@example.comfirstname.lastname@example.org/' > unencode.msg where email@example.com is the address for my admin user. This creates an mbox style mail spool file that I used as: python3 roundup/scripts/roundup_mailgw.py trackers/issue.tracker mailbox unencode.msg I did the same with python2 and a python2 tracker homedir. I verified that the characters that should cause the error were stored in db/msg/msgXX by running 'od -c' on the file and verifying the 3 digit non-ascii character codes on the line before the answer: header. It might be worth looking at https://issues.roundup-tracker.org/msg7562 for background and https://issues.roundup-tracker.org/issue2550740 https://issues.roundup-tracker.org/issue2550540 for other still open encoding issues that sadly do not have reproduction steps. Do you think I should try to set up a 1.6.1 install to test with to verify that my test method can cause the issue? If you guys think this is a valid test, I'll close this ticket out as not-reproducable.
|msg7566||Author: [hidden] (ber)||Date: 2022-06-08 06:34|
> Does this look like a reasonable way to test. I don't know just by reading. Thus > Do you think I should try to set up a 1.6.1 install to test with > to verify that my test method can cause the issue? Seems the prudent approach which also is the fastest to produce a reliable result.
|msg7575||Author: [hidden] (rouilj)||Date: 2022-06-08 18:35|
Well this is interesting. Bern, I spun up 1.6.1 under python2 on my test machine and was unable to produce the error. I did need to add a date to the "From " mbox header line (From firstname.lastname@example.org Wed Jun 5 01:06:54 2022). The message was properly added each time (although I did make the message id unique otherwise the message was ignored). So I logged into issues.roundup-tracker.org and running the same test (using my email and a date stamp) did produce the error, however I also saw: sudo -u roundup /srv/roundup/env/bin/roundup-mailgw /srv/roundup/trackers/roundup mailbox unencode.msg /srv/roundup/env/local/lib/python2.7/site-packages/psycopg2/__init__.py:144: UserWarning: The psycopg2 wheel package will be renamed from release 2.8; in order to keep installing from binary please use "pip install psycopg2-binary" instead. For details see: <http://initd.org/psycopg/docs/install.html#binary-install-from- pypi>. """) No handlers could be found for logger "detector" the handler issue was unexpected. The rsswriter.py detector module to generate an rss feed from the tracker was one that I copied from my sysadmin tracker. It opened a logger for "detector" which wasn't defined. So I changed the code to open the "roundup" logger instead (and also changed an extension to do the same). Now when running the same command I can successful submit the same message (after changing the message id) multiple times without an error being emailed and the message shows up in the web interface (with the non-ascii chars represented by a glyph). So this looks like an exception or other error was generated in a reactor and it triggered the error email but also aborted the finalization of the message in the database somehow. I am not quite sure how an error in a detector can do that. Sadly trying that same thing on my test instances isn't producing the problem. I have a different version of python2 so that may be an issue.
|msg7576||Author: [hidden] (ber)||Date: 2022-06-09 06:44|
Hi Marcus, so far we cannot reproduce the problem. There must be something different in your setup than in John's. (Could be some different in Python version or version settings inherited from the general operating system or something done by a set of reactors or other modules.) Can you reproduce this in a plain, fresh minimal setup? (Then you and John could compare.) Regards Bernhard
|msg7577||Author: [hidden] (marcus.priesch)||Date: 2022-06-09 08:29|
Hi Bernhard, it happended when submitting to roundup-tracker.org - so, sorry, i cannot ... regards, marcus.
|msg7578||Author: [hidden] (rouilj)||Date: 2022-06-09 13:31|
Hi Bern: There appears to be a misunderstanding. In message <email@example.com>, Bernhard Reiter writes: >so far we cannot reproduce the problem. This is true for my test instance. However the problem Marcus' describes is on the https://issues.roundup-tracker.org tracker when he submitted the attached message. It's not one of Marcus' trackers. I was able to reproduce the problem on the issues.roundup-tracker.org tracker. My "fix" to the logging.getLogger() calls to eliminate the error seems to have fixed it on issues.roundup-tracker.org. Howvwer I can't explain why that made the error go away. Also I can't reproduce that failure in my test code to understand why that failure caused the behavior Marcus described. >There must be something different [...] (Could be some different in >Python version or version settings inherited from the general >operating system or something done by a set of reactors or other >modules.) That's what I am assuming. However I am using the tracker from mercurial that should be the same as what is deployed on issues.roundup-tracker.org. Work continues to verify that assumption. -- rouilj
|msg7720||Author: [hidden] (rouilj)||Date: 2023-02-20 00:19|
I had no luck verifying this. Since I can't reproduce the error after the fix on issues.roundup-tracker.org (IRO), I am closing this. Also I don't get error emails from IRO and since BPO is in read-only mode, they are not monitoring it as much anymore. I do have access to the error log, but I don't check it on a regular basis.
|2023-02-20 00:19:16||rouilj||set||status: open -> fixed|
messages: + msg7720
|2022-06-09 13:31:49||rouilj||set||messages: + msg7578|
|2022-06-09 08:29:07||marcus.priesch||set||messages: + msg7577|
|2022-06-09 06:44:14||ber||set||messages: + msg7576|
|2022-06-08 18:35:25||rouilj||set||messages: + msg7575|
|2022-06-08 06:34:23||ber||set||messages: + msg7566|
|2022-06-07 20:24:30||rouilj||set||status: new -> open|
messages: + msg7565
nosy: + schlatterbeck, ber, rouilj
|2022-06-07 16:25:31||marcus.priesch||set||messages: + msg7564|