Roundup Tracker - Issues

Issue 2550976

classification
Make CSV import/export compatible across Python versions
Type: rfe Severity: normal
Components: Infrastructure Versions: 1.6
process
Status: fixed fixed
:
: : joseph_myers
Priority: : python3

Created on 2018-07-25 19:41 by joseph_myers, last changed 2018-09-02 23:52 by joseph_myers.

Messages
msg6132 Author: [hidden] (joseph_myers) Date: 2018-07-25 19:41
We need to ensure that the CSV format used for export/import is properly
compatible across Python versions.  UTF-8 strings are included in
exports with \x escapes for the individual bytes of the UTF-8
representation (because that's how Python 2 does repr of a str object).
 Special care will be needed to keep this compatible for an export with
one Python version to be importable with another Python version.
msg6210 Author: [hidden] (joseph_myers) Date: 2018-08-19 12:11
When dealing with single strings, in Python 3, if eval of the value from
the CSV file produces a str value s (which in Python 2 is the string
wanted) then s.encode('iso-8859-1').decode('utf-8') is the actual string
wanted in Python 3; for the reverse direction, where repr(s) is
appropriate in Python 2, repr(s.encode('utf-8'))[1:] will give the
representation in Python 3.

However, the CSV files for journals also use repr / eval on dict objects
which contain strings as values.  So to keep those compatible, after
calling eval I suppose you need to do the
s.encode('iso-8859-1').decode('utf-8') for every str value in the
resulting dict, while maybe the code needs to replicate the effects of
repr on a dict itself in order to do the repr(s.encode('utf-8'))[1:] on
every value in the dict.  (Or you could just replace the dict's str
values with s.encode('utf-8') in Python 3 (call
roundup.anypy.strings.s2b unconditionally) before calling repr on the
dict; that would mean the values appeared as b'...' in the CSV file
generated with Python 3, but Python 2 would still be able to read those
values OK.)
msg6230 Author: [hidden] (joseph_myers) Date: 2018-09-02 23:52
I've committed a fix for this issue.
History
Date User Action Args
2018-09-02 23:52:23joseph_myerssetstatus: new -> fixed
resolution: fixed
messages: + msg6230
2018-08-19 12:11:02joseph_myerssetmessages: + msg6210
2018-07-26 10:42:09joseph_myerssetkeywords: + python3
2018-07-25 19:41:49joseph_myerscreate