From the mailing list,  opening an issue as requested by ralf.

- HTML attachments should not be served as text/html, see discussion on
  roundup-users under the title "Spam attack, observations, how to
  repair". I've committed a partial fix but this needs more work:

  Browsers seem to be interpreting *any* content-type without a '/' as
  html. This is actively used by spammers. Since we don't currently have
  nofollow set for attachments, search engines happily index these
  pages.

  There is no issue yet, if someone has time and wants to contribute,
  making an issue for this would be a welcome contribution. If you do
  this, please set me on the nosy list.

The email thread Ralf referred to is at:

http://permalink.gmane.org/gmane.comp.bug-tracking.roundup.user/10733

FWIW we have been using this detector to force text/plain on (x)html
documents:
http://hg.python.org/tracker/python-dev/file/6f1b863bd1d8/detectors/no_texthtml.py

> Browsers seem to be interpreting *any* content-type without a '/' as 
> html. This is actively used by spammers.

I'm not sure if our detector covers this case, but so far it's been
working fine for us.

Thanks John for creating the issue.

On Sun, Jul 20, 2014 at 07:44:24AM +0000, Ezio Melotti wrote:
> 
> FWIW we have been using this detector to force text/plain on (x)html
> documents:
> http://hg.python.org/tracker/python-dev/file/6f1b863bd1d8/detectors/no_texthtml.py
> 
> > Browsers seem to be interpreting *any* content-type without a '/' as 
> > html. This is actively used by spammers.
> 
> I'm not sure if our detector covers this case, but so far it's been
> working fine for us.

Your detector will *not* work for the case we were discussing. Browsers
accept any content-type without a '/' as html. And search engines will
happily index stuff as html. So as soon as someone sets the type to
'anything' or 'wrzlbrmft' your detector will fail and the file will be
interpreted as html.

Note the the current fix is very similar to your detector and I didn't
know browsers are doing this until it came up here.

We are shipping text/html attachments as application/octet-stream unless
a config-option is set. This needs to be extended to content types that
don't contain a '/'. A whitelist feature would be even better
(configurable list of content-types that are *not* mangled when shipping
via the web server).

Ralf
-- 
Dr. Ralf Schlatterbeck                  Tel:   +43/2243/26465-16
Open Source Consulting                  www:   http://www.runtux.com
Reichergasse 131, A-3411 Weidling       email: office@runtux.com
allmenda.com member                     email: rsc@allmenda.com

This is a release blocker.

There have been discussions on this on December 2014 on the devel ml.

Ralf wrote in the end:
"""
So we should
- check for valid mime-types on incoming attachments (either via
  web-interface or via mail)
  Can be realized as an auditor so that users can change the policy
  here. We should only rewrite clearly invalid mime-types at that point.
- have a whitelist of attachments that can safely be shipped to the
  browser. All mime-types not in the whitelist are shipped as
  application/octet-stream. My tests indicate that browsers will not
  display these attachments with this content-type, they only offer to
  download the file. The original code by Richard attempted this but
  failed on invalid mime-types for reasons indicated above.

I think the hardest part is coming up with a decent whitelist that
doesn't miss too many content-types in use out there.
But users can reconfigure the whitelist (and give feedback) so we can
converge to something usable.
"""

Should be make seperate issues out of this?

On Sun, Jul 20, 2014 at 6:37 AM, John Rouillard
<issues@roundup-tracker.org> wrote:
>
> I've committed a partial fix but this needs more work:

Where is the commit(s) for the history?

>   Browsers seem to be interpreting *any* content-type without a '/' as
>   html.

Browser Security Handbook confirms this:
https://code.google.com/p/browsersec/wiki/Part2#Survey_of_content_sniffing_behaviors

Hi Anatoly:

In message <CAPkN8xJ91nCz_OW-Z3mo4-u2-Q3cuQBHFcETJ5BrwPs-xv406A@mail.gmail.com>
 <CAPkN8xJ91nCz_OW-Z3mo4-u2-Q3cuQBHFcETJ5BrwPs-xv406A@mail.gmail.com>,
anatoly techtonik writes:
>On Sun, Jul 20, 2014 at 6:37 AM, John Rouillard
><issues@roundup-tracker.org> wrote:
>>
>> I've committed a partial fix but this needs more work:
>
>Where is the commit(s) for the history?

I opened the issue on Jul 20, 2014 at Ralf's request.

So the commit comment originated from Ralf not me. Maybe check the
mercurial log for his commits rather than looking for my name will
turn up the commit?

>>> I've committed a partial fix but this needs more work:
>>Where is the commit(s) for the history?

Found it. 48d93e98be7b or
http://sourceforge.net/p/roundup/code/ci/48d93e98be7b3428785e1087495be7ec2ee81512/

Committing a better fix now.

Commit 63c31b18b955 fixes this issue:
http://sourceforge.net/p/roundup/code/ci/63c31b18b95593865fd8bbd932b0030d0e2110be/

It adds whitelist composed from analysis of attached file and
https://mail.python.org/pipermail/tracker-discuss/2015-January/003988.html

The whitelist is hardcoded, because adding another option requires a major
version bump, because it will lead to removal of allow_html_file, new
upgrading.txt docs etc. etc. and due to limited time that we all have I don't
want to delay a release.

Creating issue2550897 for tracking a better solution,
closing the urgend issue here, because it seems to be resolved.
(Testing reports appreciated.)