Roundup Tracker - Issues

Issue 2550534

classification
bounce_message fails
Type: crash Severity: critical
Components: Mail interface Versions: 1.4
process
Status: closed
:
: : ajaksu2, ajaksu_test, pefu, richard, schlatterbeck, yyz
Priority: : patch

Created on 2009-04-01 15:58 by yyz, last changed 2009-07-20 07:35 by schlatterbeck.

Files
File name Uploaded Description Edit Remove
roundup_bounce_message.patch yyz, 2009-04-01 15:58 Roundup r4212 patch of mailer.py
errormail schlatterbeck, 2009-04-02 18:27
roundupmailer.patch yyz, 2009-04-02 20:00 Smaller more focused patch
roundupmailer.patch yyz, 2009-04-02 23:46
Messages
msg3672 Author: [hidden] (yyz) Date: 2009-04-01 15:58
This is basically a copy of the report I sent to roundup-users.  With a little extra information 
before the traceback.

I'm running some tests with Roundup 1.4.8 and I was having a lot of
problems with the bounce_message function from mailer.py.
    
It looks like fallout from transitioning to the email package.  For one the documentation of 
the error parameter is incorrect.  error is passed in a list.  Looking through the svn history, it 
used to get '\n'.join()-ed so I did that, but that just exposed an error in the email creation.  
bounce_message attempted to attach files to a plain/text message.  So I changed it to use a 
MIMEMultipart.

I've attached the output of svn.diff

Specifically the error I was getting (at first) was:

Traceback (most recent call last):
  File "/u/testreq/bin/roundup-mailgw", line 3, in <module>
    run()
  File "/u/testreq/lib/python/roundup/scripts/roundup_mailgw.py", line
210, in run
    sys.exit(main(sys.argv))
  File "/u/testreq/lib/python/roundup/scripts/roundup_mailgw.py", line
153, in main
    return handler.do_pipe()
  File "/u/testreq/lib/python/roundup/mailgw.py", line 554, in do_pipe
    self.main(s)
  File "/u/testreq/lib/python/roundup/mailgw.py", line 716, in main
    return self.handle_Message(Message(fp))
  File "/u/testreq/lib/python/roundup/mailgw.py", line 773, in
handle_Message
    self.mailer.bounce_message(message, [sendto[0][1]], m)
  File "/u/testreq/lib/python/roundup/mailer.py", line 147, in
bounce_message
    part = MIMEText(error)
  File "/lusr/opt/python-2.5.4/lib/python2.5/email/mime/text.py", line
30, in __init__
    self.set_payload(_text, _charset)
  File "/lusr/opt/python-2.5.4/lib/python2.5/email/message.py", line
220, in set_payload
    self.set_charset(charset)
  File "/lusr/opt/python-2.5.4/lib/python2.5/email/message.py", line
260, in set_charset
    cte(self)
  File "/lusr/opt/python-2.5.4/lib/python2.5/email/encoders.py", line
73, in encode_7or8bit
    orig.encode('ascii')
AttributeError: 'list' object has no attribute 'encode'
msg3674 Author: [hidden] (ajaksu2) Date: 2009-04-02 18:11
Testing this locally...
msg3675 Author: [hidden] (ajaksu2) Date: 2009-04-02 18:11
2009/4/2 Daniel Diniz <issues@roundup-tracker.org>:
>
> Daniel Diniz <ajaksu@gmail.com> added the comment:
>
> Testing this locally...
>

And replying by mail :)
msg3676 Author: [hidden] (schlatterbeck) Date: 2009-04-02 18:27
I'm getting this too when sending to roundup tracker, see attached
bounce-mail.
msg3677 Author: [hidden] (yyz) Date: 2009-04-02 20:00
I think I have the minimal patch that will solve this issue.  The other patch included me mucking 
with mailer.get_standard_message, and while I think that's still correct, it's not necessary to fix 
the issue at hand.

So, here is a smaller patch that just touches mailer.bounce_message.

I'd love to help write unit tests for this, but I'm 1) a novice at that, and 2) currently too busy 
(transitioning to roundup as it happens).  Sorry I can't be of more help.
msg3678 Author: [hidden] (ajaksu2) Date: 2009-04-02 22:33
Sorry for taking so long to reply.

ISTM this will only happen when a mail delivery really fails, so mocking
should be used for testing. So, if that failure does happen, we will
certainly see the bug in the Python tracker.

This part of the patch is patently correct:
-        part = MIMEText(error)
+        part = MIMEText('\n'.join(error))

(unless there's a chance that error is a string, but I see no way to
achieve from current callers of mailer.bounce_message)

The change to multipart to allow attachments looks good to me too, but
I'm not sure if it's the only/best way to do it (nothing against it,
just lack of knowledge on my part).

This bit below I don't understand:
         else:
-            body.write(bounced_message.fp.read())
-        part = MIMEText(bounced_message.fp.read())
-        part['Content-Disposition'] = 'attachment'
-        for header in bounced_message.headers:
-            part.write(header)
+            part = MIMEText(bounced_message.fp.read())
         message.attach(part)

Again, not that I know it's wrong, I just can't say it's correct because
it's not obvious. I can't even understand it before the patch :/

Looks like a good patch that I must find a way to test (and soon!), so 
don't pay attention if I register a new user and set his email to
something invalid :)
msg3679 Author: [hidden] (yyz) Date: 2009-04-02 23:46
thus spake Daniel Diniz <issues@roundup-tracker.org>: 
> The change to multipart to allow attachments looks good to me too, but
> I'm not sure if it's the only/best way to do it (nothing against it,
> just lack of knowledge on my part).

Since the code appeared to me to want to attach the original message
to the new bounce message that already contained the error (created as
a MIMEText), then it seemed you either go Multipart and attach them,
or concatenate them as text.  The code appeared to support the first
option more strongly so I went with that.

> This bit below I don't understand:
>          else:
> -            body.write(bounced_message.fp.read())
> -        part = MIMEText(bounced_message.fp.read())
> -        part['Content-Disposition'] = 'attachment'
> -        for header in bounced_message.headers:
> -            part.write(header)
> +            part = MIMEText(bounced_message.fp.read())
>          message.attach(part)
> 
> Again, not that I know it's wrong, I just can't say it's correct because
> it's not obvious. I can't even understand it before the patch :/

I have to admit this is me guessing based on the subversion history of
mailer.py (thank you svn blame).

As it had been written it wouldn't work at all, but I found this
in svn -r1866:

        body.write('\n')
        try:
            bounced_message.rewindbody()
        except IOError, message:
            body.write("*** couldn't include message body: %s ***"
                       % bounced_message)
        else:
            body.write(bounced_message.fp.read())

Oh, darn it. You know what, I don't think this part of my patch is entirely correct.
It shouldn't crash, so in that way it's better than before, but it
doesn't preserve the original desired behavior, which I believe is to
send along the headers as well...

New patch attached. Sorry for all this churn.
msg3692 Author: [hidden] (ajaksu2) Date: 2009-04-18 00:41
LGTM.
msg3803 Author: [hidden] (schlatterbeck) Date: 2009-07-15 13:14
Fixed in the repository (with a variant of the last patch that avoids
repeated string-concatenation). Also added a regression test. Can
somebody test this?
msg3808 Author: [hidden] (pefu) Date: 2009-07-17 12:10
Hi,
this morning I tried to send a reply to issue2550564 (tracker  
here), which failed on roundup@psf.upfronthosting.co.za with
the following traceback:

Final-Recipient: rfc822; roundup+roundup@psf.upfronthosting.co.za
Original-Recipient: rfc822;issues@roundup-tracker.org
Action: failed
Status: 5.3.0
Diagnostic-Code: x-unix; Traceback (most recent call last):   File
    "/home/roundup/roundup/bin/roundup-mailgw", line 6, in ?     run() 
 File
   
"/home/roundup/roundup/lib/python2.4/site-packages/roundup/scripts/roundup_mailgw.py",
    line 210, in run     sys.exit(main(sys.argv))   File
   
"/home/roundup/roundup/lib/python2.4/site-packages/roundup/scripts/roundup_mailgw.py",
    line 153, in main     return handler.do_pipe()   File
   
"/home/roundup/roundup/lib/python2.4/site-packages/roundup/mailgw.py", line
    554, in do_pipe     self.main(s)   File
   
"/home/roundup/roundup/lib/python2.4/site-packages/roundup/mailgw.py", line
    716, in main     return self.handle_Message(Message(fp))   File
   
"/home/roundup/roundup/lib/python2.4/site-packages/roundup/mailgw.py", line
    773, in handle_Message     self.mailer.bounce_message(message,
    [sendto[0][1]], m)   File
   
"/home/roundup/roundup/lib/python2.4/site-packages/roundup/mailer.py", line
    151, in bounce_message     part = MIMEText(error)   File
    "email/MIMEText.py", line 28, in __init__   File "email/Message.py",
line
    218, in set_payload   File "email/Message.py", line 258, in set_charset
    File "email/Encoders.py", line 63, in encode_7or8bit AttributeError:
'list'
    object has no attribute 'encode'

This looks like the same issue here.  Am I right?

Peter.
msg3811 Author: [hidden] (schlatterbeck) Date: 2009-07-17 15:56
On Fri, Jul 17, 2009 at 12:10:29PM +0000, Peter Funk wrote:
> 
> Hi,
> this morning I tried to send a reply to issue2550564 (tracker  
> here), which failed on roundup@psf.upfronthosting.co.za with
> the following traceback:
> 
>     File "email/Encoders.py", line 63, in encode_7or8bit AttributeError:
> 'list'
>     object has no attribute 'encode'
> 
> This looks like the same issue here.  Am I right?

Yes. You probably sent the email from an address not registered with the
tracker you were addressing. When roundup tried to tell you that you
aren't a registered user the traceback was hit.

The bug is fixed in the roundup repository.

Ralf
-- 
Dr. Ralf Schlatterbeck                  Tel:   +43/2243/26465-16
Open Source Consulting                  Fax:   +43/2243/26465-23
Reichergasse 131                        www:   http://www.runtux.com
A-3411 Weidling                         email: office@runtux.com
osAlliance member                       email: rsc@osalliance.com
msg3820 Author: [hidden] (richard) Date: 2009-07-20 06:17
Ralf, may this issue be closed?
msg3823 Author: [hidden] (schlatterbeck) Date: 2009-07-20 07:35
On Mon, Jul 20, 2009 at 06:17:38AM +0000, Richard Jones wrote:
> 
> Ralf, may this issue be closed?

Works for me -- I've received no test reports, though.
History
Date User Action Args
2009-07-20 07:35:55schlatterbecksetstatus: open -> closed
messages: + msg3823
2009-07-20 06:17:38richardsetnosy: + richard
messages: + msg3820
2009-07-17 15:56:58schlatterbecksetmessages: + msg3811
2009-07-17 12:10:29pefusetnosy: + pefu
messages: + msg3808
2009-07-15 13:14:28schlatterbecksetstatus: new -> open
messages: + msg3803
2009-04-18 00:41:34ajaksu2setmessages: + msg3692
2009-04-18 00:29:44ajaksu_testsetnosy: + ajaksu_test
2009-04-02 23:46:36yyzsetfiles: + roundupmailer.patch
messages: + msg3679
2009-04-02 22:33:15ajaksu2setmessages: + msg3678
2009-04-02 20:00:24yyzsetfiles: + roundupmailer.patch
messages: + msg3677
2009-04-02 18:27:50schlatterbecksetfiles: + errormail
messages: + msg3676
2009-04-02 18:23:03schlatterbecksetnosy: + schlatterbeck
2009-04-02 18:11:45ajaksu2setmessages: + msg3675
2009-04-02 18:11:01ajaksu2setnosy: + ajaksu2
messages: + msg3674
2009-04-01 15:58:52yyzcreate