Roundup Tracker - Issues

Issue 2550668

classification
Can't edit Change Note when run under nginx proxy
Type: behavior Severity: major
Components: Web interface Versions: 1.4
process
Status: closed
:
: : pefu, richard, rouilj, roy
Priority: :

Created on 2010-09-29 22:44 by roy, last changed 2016-04-09 04:16 by rouilj.

Messages
msg4130 Author: [hidden] (roy) Date: 2010-09-29 22:44
I'm running Roundup 1.4.15 on Ubuntu Lucid (kernel 2.6.32.16-linode28).  Backend is MySQL 5.1.  Python 2.6.5 with MySQLdb 1.2.2.

If I run in standalone mode and connect directly to the server, everything works fine.  But, if I go through a nginx proxy, 
everything works with the exception that I cannot edit any Change Notes.  Whenever I enter any text into a Change Note field, 
when I click the submit button, the server throws a "RuntimeError: response has not been started" exception.  Everything else 
works fine behind the proxy -- I can create issues, change the other fields, and so on.

I discussed this a bit on the user list.  Here's the threads, which might provide some additional details I've left out here.

http://sourceforge.net/mailarchive/forum.php?thread_name=1285799017.8311.86.camel%40santogold&forum_name=roundup-users
http://sourceforge.net/mailarchive/message.php?msg_name=1285694356.8311.28.camel%40santogold
msg4131 Author: [hidden] (roy) Date: 2010-09-30 15:55
OK, we've got this figured out.  The solution was to add:

proxy_set_header  X-Forwarded-For $remote_addr;

to our nginx proxy configuration for this service.  This gets us up and 
running, but somebody who knows the roundup code should figure out what's 
going on here, and especially why this only affects the Change Note 
field.  At a minimum, a note should be added to the documentation 
alerting people to do this if they're running nginx.
msg4132 Author: [hidden] (roy) Date: 2010-09-30 20:43
OK^2, now we've really got this figured out :-)

This is fundamentally a file-system permissions problem.  The reason it 
only affected Change Notes is because a change note ends up being 
written out to a "msg" file on the file system.  When running in 
standalone mode (started by init.d), the python process was running as 
root.  When running under apache with mod_wsgi, it was running as www-
data and didn't have write permission on the msg directory.

The X-Forward-For stuff was an entirely different problem, which just 
added confusion to the analysis.
msg4133 Author: [hidden] (richard) Date: 2010-10-01 01:04
Great, I'm glad it's all figured out now!

I wonder whether there's something that might be altered in Roundup to better-handle this 
situation? I assume some error was being raised due to the permission problem that was 
confusing WSGI?
msg4134 Author: [hidden] (roy) Date: 2010-10-01 01:18
The biggest problem in diagnosing it was that the only error message we got was 
"RuntimeError: response has not been started".  I'm assuming that the orignal exception was 
something like

IOError: [Errno 13] Permission denied: '/opt/roundup/trackers/songza/msg/msg23'

If we had seen that, it probably would have been trivial to sort out, but it looks like something 
caught that and replaced it with the (uninformative and generic) RuntimeError.
msg4135 Author: [hidden] (roy) Date: 2010-10-01 01:22
PS, we discussed this a bit in the office, and we're a little surprised it writes these bits of text 
out to the file system instead of storing them in the database.  Why was it done that way?
msg4136 Author: [hidden] (pefu) Date: 2010-10-01 07:38
> we're a little surprised it writes these bits of text 
> out to the file system instead of storing them in the database.  
> Why was it done that way?
messages and attached files are of arbitrary size.  The design decision
to keep these files in the file system was made by the original author
Ka-Ping Yee over 10 years ago.  The roundup distribution still contains
the paper written by him as the file doc/original_overview.html . 
Unfortunately it doesn't discuss this particular decision.  I can only
guess it was one of the following two reasons: Some of the datebases 
which should be supported by the general hyper database interface are
not very good in storing arbitrary sized blobs of data.  Or getting rid
of old and no longer used attachments and messages without having
rewrite the whole database is a lot easier to implement.  But as I said:
These are wild guesses of somebody not knowing better.

I wonder why you ran into this problem in the first place?
Usually the subdirectory 'db/' of your tracker instance and all
files in it should have the same owner/group and permissions.
If writing to the database succeeds and storing the message 
raises a Permission denied exception then the only plausible
cause seems to be that my assumption above was not true in 
your case.  How comes?
msg4137 Author: [hidden] (richard) Date: 2010-10-04 03:28
On Fri, Oct 1, 2010 at 5:38 PM, Peter Funk <issues@roundup-tracker.org> wrote:
> Peter Funk <pefu@users.sourceforge.net> added the comment:
>> we're a little surprised it writes these bits of text
>> out to the file system instead of storing them in the database.
>> Why was it done that way?
> messages and attached files are of arbitrary size.  The design decision
> to keep these files in the file system was made by the original author
> Ka-Ping Yee over 10 years ago.

Ping's original motivation was to make the text searchable using grep.
A number of aspects of Roundup have changed since he made that
decision (and I implemented it).
msg5507 Author: [hidden] (rouilj) Date: 2016-04-09 04:16
Closing. There may be an action item to better return errors
to the client/user. But this is 5+ years old so chances are
the code paths have changed.
History
Date User Action Args
2016-04-09 04:16:12rouiljsetstatus: new -> closed
nosy: + rouilj
messages: + msg5507
2010-10-04 03:28:47richardsetmessages: + msg4137
2010-10-01 07:38:56pefusetmessages: + msg4136
2010-10-01 07:19:47pefusetnosy: + pefu
2010-10-01 01:22:05roysetmessages: + msg4135
2010-10-01 01:18:04roysetmessages: + msg4134
2010-10-01 01:04:21richardsetnosy: + richard
messages: + msg4133
2010-09-30 20:43:01roysetmessages: + msg4132
2010-09-30 15:55:15roysetmessages: + msg4131
2010-09-29 22:44:54roycreate