Roundup Tracker - Issues

Issue 2550677

classification
Interception of Class/FileClass methods
Type: behavior Severity: normal
Components: Database Versions: 1.4
process
Status: closed fixed
:
: rouilj : bnordgren, rouilj
Priority: normal : patch

Created on 2010-10-25 16:58 by bnordgren, last changed 2018-06-01 23:59 by rouilj.

Files
File name Uploaded Description Edit Remove
interceptor.py bnordgren, 2010-10-25 16:58
Messages
msg4182 Author: [hidden] (bnordgren) Date: 2010-10-25 16:58
I made a generic ClassInterceptor designed to be mixed in with Class or
FileClass at runtime as directed by schema.py (hence supporting all
backends). The two initial uses of this class are: 1] compression of
"content" (for messages primarily composed of text); and 2] suppression
of duplicate file entries (handy for people who have "image signatures"
setup in their email client.) It could also be used for encryption of
file content on disk.

The attached file contains the code. Feel free to post on the wiki, as
the wiki's permissions are still setup to prohibit postings by this
registered user. Also feel free to include in the standard distribution.
Use of the file in schema.py is as follows: 

from interceptor import interceptor_factory, GzipFileClass, UniqueFileClass

# Make a message file class which compresses the content before storage.
MessageFileClass =
interceptor_factory('MessageFileClass',GzipFileClass,FileClass)

# Make a file class which only stores unique files.
DataFileClass =
interceptor_factory('DataFileClass',UniqueFileClass,FileClass)
msg5666 Author: [hidden] (rouilj) Date: 2016-06-28 01:01
This is pretty cool.

To use this put in the tracker's lib subdirectory (create if missing).

Then add at the top of schema.py:
  from interceptor import interceptor_factory, GzipFileClass,
UniqueFileClass

  # Make a message file class which compresses the content before storage.
  MessageFileClass =
interceptor_factory('MessageFileClass',GzipFileClass,FileClass)

  # Make a file class which only stores unique files.
  DataFileClass
=interceptor_factory('DataFileClass',UniqueFileClass,FileClass)

then later in schema.py replace FileClass:

  # FileClass automatically gets this property in addition to the Class
ones:
  #   content = String()    [saved to disk in <tracker home>/db/files/]
  #   type = String()       [MIME type of the content, default 'text/plain']
  msg = MessageFileClass(db, "msg",
                author=Link("user", do_journal='no'),
                recipients=Multilink("user", do_journal='no'),
                date=Date(),
                summary=String(),
                files=Multilink("file"),
                messageid=String(),
                inreplyto=String())

  file = DataFileClass(db, "file",
                name=String())

as shown above.

I have not found any issues with the gzip of messages. The gui
displays msg files on disk in both uncompressed (before schema change)
and compressed (after schema change).

Using "zcat db/fles/msgs/0/msgs3" will show the contents of the
compressed message.

DataFileClass is interesting but has the following UI issues:

   When attaching the same file with a different name, the
     new file doesn't show up. Understandable since it is already
     attached. However it just looks like nothing happened. It
     should raise a message saying that the file already exists
     attached to the ticket under the other name.

   Assume we have two files with the same contents: a.txt and b.txt.
     attach a.txt (designated file1) to issue1 first then attach b.txt
     to issue2.  On issue2, a link is added to file1 reusing the
     exiting uploaded file so only one copy is kept on disk. The
     filename displayed on issue2 however is a.txt (the name of the
     first file to be attached). Not sure how to solve this. This
     could allow data leakage. Also a reference in issue2 to "b.txt"
     will be confusing.

This certainly merits posting on the wiki and I think the compressed
file idea is great.

I wonder is something similar could be done for those people who want
to put files as blobs in the database.
msg6078 Author: [hidden] (rouilj) Date: 2018-06-01 23:59
Added to wiki at:

http://www.roundup-tracker.org/cgi-bin/moin.cgi/MixinClassFileClass

I am not going to add it to the core because of the UI issues
with DataFileClass.

Closing.
History
Date User Action Args
2018-06-01 23:59:45rouiljsetpriority: normal
status: new -> closed
resolution: fixed
messages: + msg6078
2016-06-28 01:01:06rouiljsetassignee: rouilj
messages: + msg5666
nosy: + rouilj
2016-06-28 00:25:56rouiljsetkeywords: + patch
2010-10-25 16:58:28bnordgrencreate