Roundup Tracker - Issues

Message7739

Author rouilj
Recipients rouilj
Date 2023-03-06.04:13:45
Message-id <1678076025.99.0.183794276633.issue2551187@roundup.psfhosted.org>
In-reply-to
Another solution I am testing is to use a docker image of imgproxy 
(https://github.com/imgproxy/imgproxy).

My use case assumes I display an image attached to a message/issue in the issue or
message item view.

In the Roundup generated web page replace the img src="url" of:

   https://example.com/issues/file14/face.jpg 

with

   https://example.com/imgproxy/security_hash/rs:fit:300:200:0:0/
           filename:face:expires:XXXXX/plain/local:///0/file14

This url resizes the image to 300x200. It will serve up WEBP or AVIF if imgproxy
is configured to do so. The download filename produced will be face.jpg or face.webp
or some other extension depending on what imgproxy serves up. imgproxy is configured
with direct access to the db/files/file tree and serves the image from there.

This appears to work well serving up images 1/4 or less of the size of the original.
For image heavy pages it reduces the time to finish loading the page (even with lazy
loading).

The problem is imgproxy has no way to know if it should serve up a file or not.
It can't access the authentication and authorization info that the tracker has
for an image.

To keep this somewhat secure, I use the expires unix epoch timestamp and the
security_hash. The security_hash prevents all modifications to the URL unless you
know the salt and key used to generate the hash. So you can't change the file that
is accessed, add the raw directive to access some other non-image file, modify the
expiration (expires) time etc.

I set the expires value to a few seconds (say 5) from the time the page was generated.
If the URL isn't used within 5 seconds of its generation it's useless. This attempts
to make the access valid only for including the image in the Roundup issue/msg page.

The headers returned from imgproxy allow a the file to be cached only until the
expires time is reached. This does mean that upstream caches are of limited use.
If you  know the file will be displayed to the anonymous user without any limitations,
you could set the expires header far in the future to make use of an http cache.

Downloading directly from Roundup is preferred if you are viewing the fileNN page.
This should download using the normal URL: /issues/file14/face.jpg for 2 reasons:

  1) access controls are applied
  2) the original uploaded file is downloaded without any modifications

Number 2 can be important since imgproxy strips EXIF and other info. Also depending
on why the image was attached, modifying the format can reduce detail etc.

See: https://github.com/imgproxy/imgproxy/issues/1126 for more details.

If I deploy this, I will write up something on the wiki and probably close out
this ticket as I think this is a better solution:

  1) original file is preserved
  2) no wasted disk space for files that will never be displayed
  2) will run faster (imgproxy is a go binary)

I also took a brief look at https://github.com/thumbor/thumbor (written in python) which
has a similar signature method to prevent DOS. But it i missing the equivalent of the expires
directive. So there is no way to limit the time the image URL is available.
Some of the PRO options in imgproxy are available in thumbor: watermark, algorithm tuning
.... for free. Also it has an internal cache.
History
Date User Action Args
2023-03-06 04:13:45rouiljsetmessageid: <1678076025.99.0.183794276633.issue2551187@roundup.psfhosted.org>
2023-03-06 04:13:45rouiljsetrecipients: + rouilj
2023-03-06 04:13:45rouiljlinkissue2551187 messages
2023-03-06 04:13:45rouiljcreate