Issue 2550976
Created on 2018-07-25 19:41 by joseph_myers, last changed 2018-09-02 23:52 by joseph_myers.
msg6132 |
Author: [hidden] (joseph_myers) |
Date: 2018-07-25 19:41 |
|
We need to ensure that the CSV format used for export/import is properly
compatible across Python versions. UTF-8 strings are included in
exports with \x escapes for the individual bytes of the UTF-8
representation (because that's how Python 2 does repr of a str object).
Special care will be needed to keep this compatible for an export with
one Python version to be importable with another Python version.
|
msg6210 |
Author: [hidden] (joseph_myers) |
Date: 2018-08-19 12:11 |
|
When dealing with single strings, in Python 3, if eval of the value from
the CSV file produces a str value s (which in Python 2 is the string
wanted) then s.encode('iso-8859-1').decode('utf-8') is the actual string
wanted in Python 3; for the reverse direction, where repr(s) is
appropriate in Python 2, repr(s.encode('utf-8'))[1:] will give the
representation in Python 3.
However, the CSV files for journals also use repr / eval on dict objects
which contain strings as values. So to keep those compatible, after
calling eval I suppose you need to do the
s.encode('iso-8859-1').decode('utf-8') for every str value in the
resulting dict, while maybe the code needs to replicate the effects of
repr on a dict itself in order to do the repr(s.encode('utf-8'))[1:] on
every value in the dict. (Or you could just replace the dict's str
values with s.encode('utf-8') in Python 3 (call
roundup.anypy.strings.s2b unconditionally) before calling repr on the
dict; that would mean the values appeared as b'...' in the CSV file
generated with Python 3, but Python 2 would still be able to read those
values OK.)
|
msg6230 |
Author: [hidden] (joseph_myers) |
Date: 2018-09-02 23:52 |
|
I've committed a fix for this issue.
|
|
Date |
User |
Action |
Args |
2018-09-02 23:52:23 | joseph_myers | set | status: new -> fixed resolution: fixed messages:
+ msg6230 |
2018-08-19 12:11:02 | joseph_myers | set | messages:
+ msg6210 |
2018-07-26 10:42:09 | joseph_myers | set | keywords:
+ python3 |
2018-07-25 19:41:49 | joseph_myers | create | |
|