Created on 2021-07-05 02:23 by rouilj, last changed 2021-07-26 01:54 by rouilj.
|http_compression1.patch||rouilj, 2021-07-05 02:23||First compression patch|
|msg7298||Author: [hidden] (rouilj)||Date: 2021-07-05 02:23|
Roundup doesn't compress (Content-Encoding) assets when returning them to the client. This leads to poor scores on google core web vitals. Roundup behind nginx or apache can compress using gzip. Also running with uwsgi can do the same. Neither seems to compress with brotli. Using gnunicorn as a wsgi server won't compress data. This patch adds compression with: gzip, brotli, zstd to assets served by roundup. Note that 404 and other error pages are sent without compression. Also the response to a range request by the client is not compressed. Still to be implemented: Implement priority order for Accept-Encoding. Right now the first matching entry in the list: zstd, br, gzip is used (1). So an Accept-Encoding: br;q=0.5, gzip;q=0.75, zstd;q=0.25 will return zstd encoded data but it should return gzip. Allow admin to enable/disable on the fly compression and choose preferred ordering of compression algorithms if client doesn't specify a preference. E.G. brotli may be too expensive on a small system and the admin should be able to disable it. (Precompressed assets will still be delivered with compression.) Consider adding an option to not compress content smaller that a certain size. E.G. under 100 bytes is uncompressed. The default compression settings should be sufficient, so I do not plan on exposing quality/compression levels in the config. The code compresses dynamic output (expansion of web page templates) on the fly. It will also compress attachments accessed via /fileN, /msgN on the fly. It will compress files accessed via /@@files on the fly. However for these files we can improve compression and reduce cpu load and response time by precompressing the files. If you have style.css and generate: style.css.zstd style.css.br style.css.gzip it will return the file that matches acceptable compression using the order in (1) above. So if zstd is in Accept-Encoding it will read and return style.css.zstd if the client requests the url: https://example.com/@@file/style.css Note the file will always be compressed if the client supports it since roundup will compress on the fly. Using pre-compressed assets just reduces server computation time and improves response time. Also compression uses a middle of the road setting for all compression methods. E.G. for br it doesn't use quality of 11 (the default) but 4 which gets ~70% of the compression in a fraction of the time. By precompressing the assets you can squeeze the most space saving out of the file without having to wait a second for a response. Client.precompressed_mime_types is a list of mime types that should not be compressed. It includes image/png and image/jpeg. So these files will not be compressed when set to the client. I believe this can be changed using interfaces.py in the tracker to add/remove mime types. The python standard library provides gzip. To get brotli or zstd support, you have to install brotli or zstd via pip. Plan is to apply this post 2.1.0 release in mid July.
|msg7301||Author: [hidden] (rouilj)||Date: 2021-07-26 01:54|
First pass in changeset: 6458:8f1b91756457 Had a couple of fixes to the tests afterward. Changes from original description: 404 pages may be compressed if > 100 bytes in size. No option to set size limit. Will see if anybody requests it. No option to choose which methods can be used for on the fly compression. Use of virtualenv and installing just the compression modules you want to use is suggested. Preferred order is hardcoded (first to last) zstd, brotli gzip. Admin/config settings are limited: turn on/off dynamic compression. Default on. enable/disable looking for precompressed static assets. Default off since admin needs to compress the assets. Default of off means there is no search penalty by default. Any precompressed asset that can be handled by the client will prevent on the fly compression of that asset. This happens even if the on-the-fly compression is a better choice. E.G. if the file foo.css.gzip exists, a request for @@file/foo.css with brotli or gzip encoding will send the gzip encoding. It will not upgrade to brotli encoding because a pre-compressed foo.css.br does not exist. If foo.css.gzip did not exist, foo.css would be served and brotli encoding would be done on the fly. If the client can't handle gzip, foo.css would be served with brotli. This way pure dynamic content can use brotli, but static assets can use only the acceptable provided pre-compressed assets. Marking as pending and if no updates in a couple of weeks will close.
|2021-07-26 01:54:56||rouilj||set||status: open -> pending|
messages: + msg7301
|2021-07-20 17:10:09||rouilj||set||title: Enable compression of assets in roundup -> Enable compression of http responses in roundup|
|2021-07-11 03:06:59||rouilj||set||status: new -> open|