Roundup Tracker - Issues

Issue 2551200

classification
Latest language tests in test_liveserver fail for me
Type: behavior Severity: normal
Components: Test Versions: devel
process
Status: closed fixed
:
: : marcus.priesch, rouilj, schlatterbeck
Priority: :

Created on 2022-05-02 09:31 by schlatterbeck, last changed 2022-05-03 15:01 by schlatterbeck.

Messages
msg7481 Author: [hidden] (schlatterbeck) Date: 2022-05-02 09:31
The latest tests for german locale in test/test_liveserver.py fail for me.

Looks like the computed LOCALE_DIRS in roundup/i18n.py does not find the locale of the checked-out directory.
I have the roundup checkout I'm testing with in
/nobackup/ralf/checkout/sourceforge/roundup.hg.tip
and LOCALE_DIRS contains:

(Pdb) p LOCALE_DIRS
['/usr/share/locale', '/nobackup/ralf/share/locale']

This results in all mofiles being None and the translator being set to the NullTranslator.
Note that when searching in LOCALE_DIRS the domain is set to 'roundup' and on that system no roundup locales are installed in /usr/share.

Any Idea on how to improve this? The regression tests should find the .mo files in the checked out directory and *not* in the installed roundup installation. I'm wondering why this works in the online regression-test framework?

This bit me when testing Marcus' modifications until I noticed that the test breaks without his changes, too.
msg7482 Author: [hidden] (schlatterbeck) Date: 2022-05-02 12:30
Looks like another locale test in test/test_dates.py also failed.
I've fixed both and revert the monkey patches to i18n after the tests in test_dates.

See latest commit 5be4f9104cf7

And commit da6c9050a79e which fixes an xfail test in test_liveserver that was failing due to modification of global data in test_cgi.

John, can you take a look at the tests and close this if the regression tests pass?

Thanks
Ralf
msg7483 Author: [hidden] (rouilj) Date: 2022-05-02 12:54
Hi Ralf:

I am not sure how things work in travis ci 8-(. I have been looking into how
to log into a test container to debug the xfail cache issue that you may have fixed with
changeset:   6651:da6c9050a79e.

Does /nobackup/ralf/share/locale exist? If so this might be breakage from
roundup trying to find the prefix and appending share/local at line 70
in i18n.py. This code is fragile and I am not happy with it but it's an attempt
to find the locale directory where setup placed the locale files.
 
I pushed both your changes to ci. We shall see how they do.
msg7484 Author: [hidden] (schlatterbeck) Date: 2022-05-02 13:18
On Mon, May 02, 2022 at 12:54:32PM +0000, John Rouillard wrote:

> I am not sure how things work in travis ci 8-(. I have been looking into how
> to log into a test container to debug the xfail cache issue that you may have fixed with
> changeset:   6651:da6c9050a79e.

The problem was reproduceable when running *all* tests. With all tests
(because the test_wsgi test ran before the one failing) the test you
marked xfail would fail. When only running test_liveserver it would not
fail (an uppercase X for the xfail test). Then I searched for all
strings that I thought could make a difference and hit on the Cache
stuff. Confirmed by running just the wsgi tests and test_liveserver
together.

> Does /nobackup/ralf/share/locale exist? If so this might be breakage
> from roundup trying to find the prefix and appending share/local at
> line 70 in i18n.py. This code is fragile and I am not happy with it
> but it's an attempt to find the locale directory where setup placed
> the locale files.

Hmm, the directory is not existing: It is not supposed to: I've never
installed roundup on that test-machine. So it is expected that the
directory doesn't exist.

Instead I copied the (unfortunately not quite working: I also fixed
that) trick from test_dates.py: It modifies the LOCALE_DIRS in i18n.py.
I also modified DOMAIN there so it would find the checked out (instead
of the installed) locales. (And I'm reverting the changes to i18n so
that we do not get different behaviour when all tests run vs. when only
one test runs, Marcus observed different behaviour which we traced to
the virtualenv he was using where roundup was installed before).

> I pushed both your changes to ci. We shall see how they do.
Thanks! That was quick :-)

Ralf
-- 
Dr. Ralf Schlatterbeck                  Tel:   +43/2243/26465-16
Open Source Consulting                  www:   www.runtux.com
Reichergasse 131, A-3411 Weidling       email: office@runtux.com
msg7485 Author: [hidden] (schlatterbeck) Date: 2022-05-02 15:46
OK, looks like all the tests pass:
https://app.travis-ci.com/github/roundup-tracker/roundup/builds/250113370

Closing this issue.
msg7486 Author: [hidden] (rouilj) Date: 2022-05-02 22:59
In message <20220502131811.i2pzm65xax2wzhkb@runtux.com>,
Ralf Schlatterbeck writes:
>Ralf Schlatterbeck added the comment:
>On Mon, May 02, 2022 at 12:54:32PM +0000, John Rouillard wrote:
>> Does /nobackup/ralf/share/locale exist? If so this might be breakage
>> from roundup trying to find the prefix and appending share/local at
>> line 70 in i18n.py. This code is fragile and I am not happy with it
>> but it's an attempt to find the locale directory where setup placed
>> the locale files.
>
>Hmm, the directory is not existing: It is not supposed to: I've never
>installed roundup on that test-machine. So it is expected that the
>directory doesn't exist.

That's wierd because the code reads:

path = __file__
for N in 1, 2:
    path = os.path.dirname(path)
    # path is /usr/local/lib/python3.10/site-packages
_ldir = os.path.join(path, sys.prefix[1:], 'share', 'locale')
if os.path.isdir(_ldir):
    LOCALE_DIRS.append(_ldir)
del _ldir

That should add _ldir to LOCALE_DIRS only if _ldir is a directory. It
takes the path of i18n.py (I assume:
/nobackup/ralf/checkout/sourceforge/roundup.hg.tip/i18n.py) and
reduces it to:

  /nobackup/ralf/checkout/sourceforge/roundup.hg.tip
  /nobackup/ralf/checkout/sourceforge

so I am not sure where /nobackup/ralf/share/locale even came from.

Hmph.
msg7487 Author: [hidden] (schlatterbeck) Date: 2022-05-03 08:04
On Mon, May 02, 2022 at 11:00:00PM +0000, John Rouillard wrote:
> >Hmm, the directory is not existing: It is not supposed to: I've never
> >installed roundup on that test-machine. So it is expected that the
> >directory doesn't exist.
> 
> That's wierd because the code reads:
[...]

I think it's the code starting line 48:

# compute mo location relative to roundup installation directory
# (prefix/lib/python/site-packages/roundup/msgfmt.py on posix systems,
# prefix/lib/site-packages/roundup/msgfmt.py on windows).
# locale root is prefix/share/locale.
if os.name == "nt":
    _mo_path = [".."] * 4 + ["share", "locale"]
else:
    _mo_path = [".."] * 5 + ["share", "locale"]
_mo_path = os.path.normpath(os.path.join(msgfmt.__file__, *_mo_path))
if _mo_path not in LOCALE_DIRS:
    LOCALE_DIRS.append(_mo_path)
del _mo_path

This subtracts 5 directory components from the filename on Linux and
then appends /share/locale

Ralf
-- 
Dr. Ralf Schlatterbeck                  Tel:   +43/2243/26465-16
Open Source Consulting                  www:   www.runtux.com
Reichergasse 131, A-3411 Weidling       email: office@runtux.com
msg7494 Author: [hidden] (rouilj) Date: 2022-05-03 14:45
In message <20220503080423.cyerdijzaogfwdmc@runtux.com>,
Ralf Schlatterbeck writes:
>
>
>Ralf Schlatterbeck added the comment:
>
>On Mon, May 02, 2022 at 11:00:00PM +0000, John Rouillard wrote:
>> >Hmm, the directory is not existing: It is not supposed to: I've never
>> >installed roundup on that test-machine. So it is expected that the
>> >directory doesn't exist.
>> 
>> That's wierd because the code reads:
>[...]
>
>I think it's the code starting line 48:
>
># compute mo location relative to roundup installation directory
># (prefix/lib/python/site-packages/roundup/msgfmt.py on posix systems,
># prefix/lib/site-packages/roundup/msgfmt.py on windows).
># locale root is prefix/share/locale.
>if os.name == "nt":
>    _mo_path = [".."] * 4 + ["share", "locale"]
>else:
>    _mo_path = [".."] * 5 + ["share", "locale"]
>_mo_path = os.path.normpath(os.path.join(msgfmt.__file__, *_mo_path))
>if _mo_path not in LOCALE_DIRS:
>    LOCALE_DIRS.append(_mo_path)
>del _mo_path
>
>This subtracts 5 directory components from the filename on Linux and
>then appends /share/locale

and appends it blindly. Shouldn't there be a check to see if _mo_path
exists and is a directory?
msg7496 Author: [hidden] (schlatterbeck) Date: 2022-05-03 15:01
On Tue, May 03, 2022 at 02:45:45PM +0000, John Rouillard wrote:
> >This subtracts 5 directory components from the filename on Linux and
> >then appends /share/locale
> 
> and appends it blindly. Shouldn't there be a check to see if _mo_path
> exists and is a directory?

I don't think this is a problem. When the directories are used to find
locales later on, each attempt that doesn't find a locale adds a None
object to the variable 'mofiles'.
This variable is later filtered and removes all the None objects.

So I'm fine with the current behaviour.

Ralf
-- 
Dr. Ralf Schlatterbeck                  Tel:   +43/2243/26465-16
Open Source Consulting                  www:   www.runtux.com
Reichergasse 131, A-3411 Weidling       email: office@runtux.com
History
Date User Action Args
2022-05-03 15:01:26schlatterbecksetmessages: + msg7496
2022-05-03 14:45:45rouiljsetmessages: + msg7494
2022-05-03 08:04:26schlatterbecksetmessages: + msg7487
2022-05-02 23:00:00rouiljsetmessages: + msg7486
2022-05-02 15:46:58schlatterbecksetpriority: normal ->
status: open -> closed
resolution: fixed
messages: + msg7485
2022-05-02 13:18:15schlatterbecksetmessages: + msg7484
2022-05-02 12:54:31rouiljsetpriority: normal
status: new -> open
messages: + msg7483
2022-05-02 12:30:23schlatterbecksetmessages: + msg7482
2022-05-02 09:32:55schlatterbecksetversions: + devel
2022-05-02 09:32:39schlatterbecksettype: behavior
components: + Test
2022-05-02 09:31:58schlatterbeckcreate