Currently the REST-API doesn't log anything when an error occurs. I'm currently dealing with a problem where the lag between retries and me receiving the problem report is on the order of weeks. From the apache log (roundup runs as WSGI under apache in that application) I see that they receive a 400 from apache. Now I'd really like to know what they sent.

I propose to add logging to failed requests: Currently all errors are handled in the function _data_decorator in rest.py. So we could add a log there centrally and log the parameters of the request and the error message they receive.

I'm unsure about the severity of the log. Since this is the normal operation (and errors are not errors in roundup) I'd suggest to log this as INFO or lower. Or should this be a DEBUG log? I would run this always-on in production as it is normal operation that I have to debug a client web service on my end :-) So I tend to make this INFO.

What do you think?

Info sounds reasonable.

Ralf, do you have patches for this?

Ralf:

What do you think about a config.ini setting:


  rest_logging = [none|debug|info|...]

so you can enable/disable rest logging and set the log level?

As an aside, should we have more section in config.ini, e.g. [rest]?

I would need to figure out how aliases work and move some items around if
we were to add a new section.

Hi John, thanks for looking into this!

On Wed, Aug 09, 2023 at 05:22:49AM +0000, John Rouillard wrote:
> 
> Ralf:
> 
> What do you think about a config.ini setting:
> 
> 
>   rest_logging = [none|debug|info|...]
> 
> so you can enable/disable rest logging and set the log level?

Good idea! This would cover my needs to run this continuously and I
could make my log-level higher (to hide other things). Better than
having a hard-coded log-level.

> As an aside, should we have more section in config.ini, e.g. [rest]?
Hmm, for now I wouldn't do this we currently don't have too many
rest-specific items I think.

Ralf
-- 
Dr. Ralf Schlatterbeck                  Tel:   +43/2243/26465-16
Open Source Consulting                  www:   www.runtux.com
Reichergasse 131, A-3411 Weidling       email: office@runtux.com

Ralf did you do anything for this ticket?

I've just pushed a first stab at this.

I would really like to also log the parameters of the REST call (maybe also configurable with an option). For a GET request this is typically a FieldStorage object while for POST or PUT it typically is a dictionary (serialized from JSON). For logging this it would require some serialization. I could live with a multi-line JSON object (serializing the dictionary for POST or PUT) but FieldStorage cannot currently be easily serialized (although serializing this to a nested dictionary should be doable).

Now since FieldStorage is already deprecated in python (going away in 3.13) we might introduce some replacement in the future? It might be nice if everything was a nested dictionary...

Oh and for file uploads and other huge objects we probably want to limit the amount being logged.

> Now since FieldStorage is already deprecated in python (going away in 3.13) 

It's not just FieldStorage that is deprecated but all of cgi.py.

I have already vendored cgi.py under roundup/anypy/vendored/cgi.py so Roundup can
run under 3.13. This works (tested using a 3.13 alpha docker image).

Alternatives are discussed at:

  https://discuss.python.org/t/alternative-function-for-deprecated-cgi/21960/11

which addresses mostly wsgi (not CGI) use a, vendored cgi.py works.

When we drop python2 support, replacing cgi.py would be a more tractable project,
but the glib solutions to use the email module for parsing put/post bodies etc. are not
trivial replacements.