Roundup Tracker - Issues

Issue 2551385

classification
Use locale aware conversions for string -> float.
Type: behavior Severity: normal
Components: Database Versions:
process
Status: new
:
: : rouilj
Priority: low : Effort-High

Created on 2025-01-01 19:23 by rouilj, last changed 2025-01-01 19:38 by rouilj.

Messages
msg8247 Author: [hidden] (rouilj) Date: 2025-01-01 19:23
Issue1182919 (support range syntax for numbers), has a discussion of representing
a floating point number using "12345,24" where the comma is the decimal separator.

This localized representation doesn't seem to work as
hyperdb.py::Number.from_raw(self, value, *kw)
uses float() to convert value to a number. Float isn't locale aware.

  >>> string_number = "1234,56"
  >>> float(string_number)
  Traceback (most recent call last):
    File "<stdin>", line 1, in <module>
  ValueError: could not convert string to float: '1234,56'

Also it doesn't support ',' or '.' as thousand separators (but does support '_' for
thousand separators by python 3.8.

This conversion needs to use something like:

  >>> import locale
  >>> locale.setlocale(locale.LC_ALL, "en_DK.utf8")
  'en_DK.utf8'
  >>> string_number = "1.234,56"
  >>> float_number = locale.atof(string_number)
  >>> print(float_number)
  1234.56

which does support alternate thousand separators and decimal separators according
to the locale. (However thousand separators with exponent: locale.atof("1,234.56E3")
generates a ValueError in en_US but locale.atof("1.234,56E-2") generates (incorrectly)
1234.56 with en_HK (python 3.8.10).)

Currently the hyperdb is locale unaware. Also locale methods are not
thread safe (which might be an issue for roundup-server in thread mode). Some backends
like back_anydbm use float() directly for checks/conversions.

I am documenting the number input type in user_guide.txt as requiring a period
decimal separator, no thousands and supporting scientific notation.
History
Date User Action Args
2025-01-01 19:38:27rouiljsetpriority: low
2025-01-01 19:23:57rouiljcreate