Issue 2550651
Created on 2010-05-10 10:22 by tonimueller, last changed 2017-10-02 23:17 by rouilj.
Messages | |||
---|---|---|---|
msg4055 | Author: [hidden] (tonimueller) | Date: 2010-05-10 10:22 | |
Copied from debian:540626 : Experimental evidence suggests that HEAD requests will result in Roundup generating the entire page, discard it, then send the headers to the client. This is very wasteful, particularly on large pages. Making requests to bugs.darcs.net *from* bugs.darcs.net (to avoid network latency), we can see that HEAD and GET take about the same amount of time: $ time nc bugs.darcs.net www <<<$'HEAD /status1 HTTP/1.1\nHost: bugs.darcs.net\n\n' | wc -l 7 real 0m18.549s user 0m0.004s sys 0m0.004s $ time nc bugs.darcs.net www <<<$'GET /status1 HTTP/1.1\nHost: bugs.darcs.net\n\n' | wc -l 3117 real 0m18.324s user 0m0.004s sys 0m0.004s This issue has practical implications for me. I maintain a script to interact with roundup's mailgw, and I wanted to valid status IDs before sending emails: if ! curl -fsIo/dev/null http://bugs.example.net/status$N then error "$N is not a valid status ID!" fi Currently this request can take deciseconds, and so is far too slow to use. I've talked to both Eric Kow of darcs.net to get 1.4.13 installed there, and to the user, who has confirmed that the problem is still present in the latest version of roundup. |
|||
msg4056 | Author: [hidden] (tonimueller) | Date: 2010-05-10 10:24 | |
Ouch, someone has apparently disabled the Debian-BTS linking. The original bug can be viewed here: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=540626 |
|||
msg4094 | Author: [hidden] (richard) | Date: 2010-07-12 04:16 | |
I'm not sure how this situation could be improved. It would be quite difficult in most cases to determine what HEAD should be testing for a "last modified" date for pages. |
|||
msg4098 | Author: [hidden] (ber) | Date: 2010-07-20 09:14 | |
I'd say that the original use case should be solved differently. Or what are other use cases for HEAD that we should support? |
|||
msg5509 | Author: [hidden] (rouilj) | Date: 2016-04-09 04:51 | |
Is there a way in TAL to tell if a HEAD has been requested and abort the generation of the tal page. Something like: <tal:dontprocess tal:condition="python: request.client.env['REQUEST_METHOD'] != 'HEAD'" >' as the very first line of the page.html file should prevent the page generation/tal processing right? Will that have any effect on the expires time? |
|||
msg5823 | Author: [hidden] (rouilj) | Date: 2016-07-10 23:22 | |
Probably too late to make any difference, but another possibility occurred to me. Rather than using: curl -fsIo/dev/null http://bugs.example.net/status$N how about creating a template that is empty. Touch tracker/html/issue.empty.html then use: curl -fsIo/dev/null 'http://localhost:9017/demo/issue10400?@template=empty'; echo $? 22 curl -fsIo/dev/null 'http://localhost:9017/demo/issue1?@template=empty'; echo $? 0 where issue1 exists and the other one doesn't. This is probably as close as you can get at the moment. It does go into the template code, but does very little work. Alternatively you could use the xmlrpc interface. |
|||
msg6026 | Author: [hidden] (rouilj) | Date: 2017-10-02 23:17 | |
Since roundup is no longer distributed as a debian package, this sort of takes on less importance. However I did two experiments: Change page.html. I put: <tal:dontprocess tal:condition="python:request.client.env['REQUEST_METHOD'] != 'HEAD'" >' right before the <body> tag in page.html. Then I put: </tal:dontprocess> right after the </body> tag. When curling I used the following framework: time sh -c 'for i in `seq 1 100`; do curl -sk -o/dev/null -w"%{time_total}\n" https://rouilj.dynamic-dns.net/demo/issue3 ; done' (wordwrapped manually). Running this takes total time of: 0m45.816s Using the same as above but with --head added to the curl args, it takes total time of: 0m26.533s For anybody who can divide by 100, the following will not be shocking. In the GET case, curl reports average total time of 0.446s with median of .447 and a range of 0.422, 0.456 The HEAD case returns average: 0.253s median: .253 and range 0.236, 0.259. So we can see that there is a significant ~50% decrease in time. The test env is running via a proxy web server (hiawatha 10.6) against a daemonized roundup with a sqlite back end on an ASUS EB1036-B0534 Desktop with xubuntu 14.04.5. YMMV if you are using it as a cgi. Given the original report that deciseconds was too slow this doesn't help much as best case is still 2.5 deciseconds. Creating an empty issue.empty.html file in the html subdir and running the above loop with a url of: "https://rouilj.dynamic-dns.net/demo/issue3?@template=empty" (note the quotes to protect the ?) returns get case: avg: 0.15966 med: 0.160 range: 0.153, 0.171 returns head case: avg: 0.15968 med: 0.160 range: 0.154, 0.169 which are pretty much indistinguishable. However still in 1.5 decisecond range. I think just the work of accessing the db (to validate that the issue exists) and setting up an html response is what we are seeing. But this does get it down to 25% of the original request time. Note that using get on issue999 which generates a 404 error from the server runs in 45ish seconds. Probably because it is rendering _generic.404.html with the page border and all the rest. Running just a head runs in 26 seconds which seems to support that supposition since it looks similar to what was obtained with the original modified page.html. So again slower than requested. Putting the tal to detect a HEAD request at the top and bottom of the _generic.404.page will probably cut that back to something similar to (but higher than) the empty issue.empty.html template. This is left as an exercise for the reader. Also note there is no last-modified header generated for issues, only attached files. So Richard's note that HEAD would not include a last modified date is moot since I don't think any issue page has a last modified date. -- rouilj |
History | |||
---|---|---|---|
Date | User | Action | Args |
2017-10-02 23:17:45 | rouilj | set | title: HEAD requests are very slow. (fix in comments) -> HEAD requests are very slow. (fix to take 1/4 of original time in comments) |
2017-10-02 23:17:17 | rouilj | set | status: new -> closed assignee: rouilj resolution: wont fix messages: + msg6026 title: HEAD requests arer very slow. -> HEAD requests are very slow. (fix in comments) |
2016-07-10 23:22:26 | rouilj | set | messages: + msg5823 |
2016-04-09 04:51:23 | rouilj | set | nosy:
+ rouilj messages: + msg5509 |
2010-07-20 09:14:34 | ber | set | nosy:
+ ber messages: + msg4098 |
2010-07-20 09:13:18 | ber | set | title: HEAD is ridiculously slow. -> HEAD requests arer very slow. |
2010-07-12 04:16:57 | richard | set | nosy:
+ richard messages: + msg4094 |
2010-05-10 10:24:11 | tonimueller | set | messages: + msg4056 |
2010-05-10 10:22:45 | tonimueller | create |