Roundup Tracker - Issues

Issue 614072

classification
^M in mails
Type: Severity: normal
Components: None Versions:
process
Status: closed fixed
:
: : ber, richard, sheilaking, tobias
Priority: normal :

Created on 2002-09-24 22:04 by tobias, last changed 2002-10-09 09:20 by ber.

Messages
msg363 Author: [hidden] (tobias) Date: 2002-09-24 22:04
From http://issues.fresco.org/bug35:

For some reason, every email I get from roundup
with a comment entered via the web form displays
in mutt like:

I've got it all designed and half implemented; ^M
just need to implement one more piece, and then^MI
don't know why -- I've never seen Mutt display a
message like that before.

I do know that control-M is the CR in DOS-style
CRLF line terminators, and I do know that roundup
is doing the slightly odd trick of sending mail with
  Content-Transfer-Encoding: quoted-printable
in the headers, and then coding all the ^M's as
the string "=0D", which is weird.

I have a guess: web browsers send form text back
with CRLF EOLs (I don't know this for sure),
Roundup doesn't realize this and interprets
messages the Unix way (LF EOLs), and quotes
messages the Unix way (CR as a special character).
 Then, the MTAs (which are supposed to handle EOL
conversion, I think?) don't notice the CR's
because of the quoting, and are unable to strip
them when delivering to my Unix machine.  (I don't
know what the messages come out like on Windows,
where they probably end up as CRCRLF.)

If this is the case, then Roundup just needs to be
taught that HTML forms may use CRLF line
terminators, and convert them appropriately... but
someone who knows more about these protocols is
welcome to correct me :-)
msg364 Author: [hidden] (richard) Date: 2002-09-25 05:40
Logged In: YES 
user_id=6405

I'm not sure what the solution here is.  I'd really rather not filter 
messages for those linebreaks unless I really have to... perhaps there's 
a version of mutt that's a little less picky about its line endings? 
 
msg365 Author: [hidden] (ber) Date: 2002-10-08 18:27
Logged In: YES 
user_id=113859

Please open that bug again.
I've also been bitten by it.

It definately is not a good bevaviour
as there is one CR to much. Tobias is right in that it
should be 
transformed in a real line ending for sending under unix.
So convert the line endind you get from HTML forms to proper
line feeds on the platfrom roundup is running.
msg366 Author: [hidden] (richard) Date: 2002-10-08 22:27
Logged In: YES 
user_id=6405

As far as the CRLF transformation goes:   
1. do I do the same transformation on Windows?  
2. do I perhaps do the reverse transformation on Windows, turning LF  
from unix clients to CRLF (and similarly, CR from Macs to CRLF)?  
3. what do I do on Mac OS?  
4. should I always convert line endings to plain LF regardless of OS? 
Would this confuse Windows users just as the current situation is 
confusing Unix users? 
5. should the quoted-printable encoding be quoting the CRs? 
 
Or, as already stated: do I just encourage users to get reasonable  
email clients that can handle the line endings correctly? 
 
msg367 Author: [hidden] (sheilaking) Date: 2002-10-08 22:36
Logged In: YES 
user_id=467955

I responded to this comment yesterday and sent my response
to the developers list. Apparently no one saw it?

Here it is again:

=======================================
This is not strictly a Mutt issue. I have encountered the same
problem on my GypsyMail form handler script.

My resolution was, to run the following function on any
input from an HTML form that was supposed to be sent via
SMTP as an email:

text = string.replace(text, '\r\n', '\n')

I put this "patch" into my cgi script as a "fix" for an issue
with Outlook (Express) mail client.

Technically, the above "fix" doesn't address line endings for
Macintosh clients. A complete fix would do the following:

text = string.replace(text, '\r\n', '\n')
text = string.replace(text, '\r', '\n')

The above two statements work as follows:

(1) If '\r\n' occurs anywhere within the message, it is a line
ending introduced by a windows client. Replace with the standard
'\n'.
(2) After completing the first statement, if there are any
remaining occurences of '\r' in the text string, these are line
endings inserted by a Macintosh client. Replace those with '\n'.

Here is a corroborating discussion of this phenomenon in
another scripting situation:

http://www.aota.net/forums/showthread.php?postid=47373#post47373

--
Sheila King
http://www.thinkspot.net/sheila/
http://www.k12groups.org
========================================

The above pasted response does give a "fix" for this
problem, as far as I know.

--Sheila King
msg368 Author: [hidden] (richard) Date: 2002-10-08 23:52
Logged In: YES 
user_id=6405

Sorry, Shiela, I lost track of your response on the users list. I'll respond 
to that on the devel mailing list to move the discussion there (I despise 
this tracker for discussing problems ;) 
 
msg369 Author: [hidden] (richard) Date: 2002-10-09 00:52
Logged In: YES 
user_id=6405

OK, added code to mangle the newlines. 
 
msg370 Author: [hidden] (ber) Date: 2002-10-09 09:20
Logged In: YES 
user_id=113859

For the sake of recorded information:
I'm pretty sure that this is not a MUA related problem.
You can assume mutt behaviour regarding mail standards being
correct
most of the time. (Outlook usually does bad relating to
standards.)

For  any application you  have to norm newlines when text
comes in
from a source. For sending out email you have to use the
known standard way.
History
Date User Action Args
2002-09-24 22:04:38tobiascreate