Created on 2010-10-25 16:58 by bnordgren, last changed 2016-06-28 01:01 by rouilj.
|interceptor.py||bnordgren, 2010-10-25 16:58|
|msg4182||Author: [hidden] (bnordgren)||Date: 2010-10-25 16:58|
I made a generic ClassInterceptor designed to be mixed in with Class or FileClass at runtime as directed by schema.py (hence supporting all backends). The two initial uses of this class are: 1] compression of "content" (for messages primarily composed of text); and 2] suppression of duplicate file entries (handy for people who have "image signatures" setup in their email client.) It could also be used for encryption of file content on disk. The attached file contains the code. Feel free to post on the wiki, as the wiki's permissions are still setup to prohibit postings by this registered user. Also feel free to include in the standard distribution. Use of the file in schema.py is as follows: from interceptor import interceptor_factory, GzipFileClass, UniqueFileClass # Make a message file class which compresses the content before storage. MessageFileClass = interceptor_factory('MessageFileClass',GzipFileClass,FileClass) # Make a file class which only stores unique files. DataFileClass = interceptor_factory('DataFileClass',UniqueFileClass,FileClass)
|msg5666||Author: [hidden] (rouilj)||Date: 2016-06-28 01:01|
This is pretty cool. To use this put in the tracker's lib subdirectory (create if missing). Then add at the top of schema.py: from interceptor import interceptor_factory, GzipFileClass, UniqueFileClass # Make a message file class which compresses the content before storage. MessageFileClass = interceptor_factory('MessageFileClass',GzipFileClass,FileClass) # Make a file class which only stores unique files. DataFileClass =interceptor_factory('DataFileClass',UniqueFileClass,FileClass) then later in schema.py replace FileClass: # FileClass automatically gets this property in addition to the Class ones: # content = String() [saved to disk in <tracker home>/db/files/] # type = String() [MIME type of the content, default 'text/plain'] msg = MessageFileClass(db, "msg", author=Link("user", do_journal='no'), recipients=Multilink("user", do_journal='no'), date=Date(), summary=String(), files=Multilink("file"), messageid=String(), inreplyto=String()) file = DataFileClass(db, "file", name=String()) as shown above. I have not found any issues with the gzip of messages. The gui displays msg files on disk in both uncompressed (before schema change) and compressed (after schema change). Using "zcat db/fles/msgs/0/msgs3" will show the contents of the compressed message. DataFileClass is interesting but has the following UI issues: When attaching the same file with a different name, the new file doesn't show up. Understandable since it is already attached. However it just looks like nothing happened. It should raise a message saying that the file already exists attached to the ticket under the other name. Assume we have two files with the same contents: a.txt and b.txt. attach a.txt (designated file1) to issue1 first then attach b.txt to issue2. On issue2, a link is added to file1 reusing the exiting uploaded file so only one copy is kept on disk. The filename displayed on issue2 however is a.txt (the name of the first file to be attached). Not sure how to solve this. This could allow data leakage. Also a reference in issue2 to "b.txt" will be confusing. This certainly merits posting on the wiki and I think the compressed file idea is great. I wonder is something similar could be done for those people who want to put files as blobs in the database.
|2016-06-28 01:01:06||rouilj||set||assignee: rouilj|
messages: + msg5666
nosy: + rouilj
|2016-06-28 00:25:56||rouilj||set||keywords: + patch|