Roundup Tracker - Issues

Issue 2551143

classification
Title: Problem with installing external trackers due to changed location of share folder under Ubuntu 18.04
Type: behavior Severity: normal
Components: Installation Versions: devel
process
Status: new Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: rouilj, schlatterbeck, tttech-klonner
Priority: Keywords: Blocker, patch

Created on 2021-06-09 09:35 by tttech-klonner, last changed 2021-06-11 19:24 by rouilj.

Files
File name Uploaded Description Edit Remove
setup_install_under_prefix_or_slash_usr.patch rouilj, 2021-06-10 00:05 patch file to make setup put data files outside of egg install dir.
setup_install_under_prefix_or_slash_usr2.patch rouilj, 2021-06-11 19:24 second try using sysconfig.get_path('platlib') for default install prefix
Messages
msg7268 Author: [hidden] (tttech-klonner) Date: 2021-06-09 09:35
During installation of the latest devel version I recognized a problem 
during installation under Ubuntu 18.04 and Python 2.7.17. I am installing 
Roundup with "python setup.py install".

I want to install a external tracker and therefore I am copying the 
templates under /usr/local/share/roundup/templates.
With the latest version the share folder is now created under 
/usr/local/lib/python2.7/dist-packages/roundup-2.0.0-
py2.7.egg/share/roundup/templates

I then get the traceback because the package is not found:

roundup-admin install time-todo-abo postgresql

Templates:: responsive, devel, jinja2, minimal, classic
Select template [classic]: Traceback (most recent call last):
  File "/usr/local/bin/roundup-admin", line 11, in <module>
    load_entry_point('roundup==2.0.0', 'console_scripts', 'roundup-admin')
()
  File "/usr/local/lib/python2.7/dist-packages/roundup-2.0.0-
py2.7.egg/roundup/scripts/roundup_admin.py", line 50, in run
    sys.exit(tool.main())
  File "/usr/local/lib/python2.7/dist-packages/roundup-2.0.0-
py2.7.egg/roundup/admin.py", line 1788, in main
    ret = self.run_command(args)
  File "/usr/local/lib/python2.7/dist-packages/roundup-2.0.0-
py2.7.egg/roundup/admin.py", line 1657, in run_command
    return self.do_install(self.tracker_home, args)
  File "/usr/local/lib/python2.7/dist-packages/roundup-2.0.0-
py2.7.egg/roundup/admin.py", line 409, in do_install
    default='classic')
  File "/usr/local/lib/python2.7/dist-packages/roundup-2.0.0-
py2.7.egg/roundup/admin.py", line 499, in _get_choice
    argument = my_input('%s [%s]: ' % (prompt, default))
EOFError: EOF when reading a line
The command '/bin/sh -c roundup-admin install time-todo-abo 
postgresql' returned a non-zero code: 1


This is not as convenient as before as I now have to know the installation 
path and Roundup version for scripted installation.

The problem also persists if I manually create the non existing folder 
/usr/local/share/roundup/templates and put the templates there so the path 
seems also not to be included in the search path.

The last working version I had was from 7.4.2021: 
http://hg.code.sf.net/p/roundup/code/rev/4e48a6a40bfe

I know that the version from 18.05.2021 
http://hg.code.sf.net/p/roundup/code/rev/dbacf6bf2a2f and 6.6.2021 
http://hg.code.sf.net/p/roundup/code/rev/ff4ab763f47c have this issue.

I tried to have a look at the diff to my last working version but I did 
not find the change that caused this installation change:
https://github.com/roundup-
tracker/roundup/compare/7e955485d7e30ef5aeaabe8ebaa972687326cd89..6117dcd

I also shortly discussed with Ralf and we decided to do a bug report.

BR, Robert
msg7269 Author: [hidden] (rouilj) Date: 2021-06-09 14:13
Hi Robert:

In message <1623231327.12.0.257353362144.issue2551143@roundup.psfhosted.org>,
Robert Klonner writes:
>New submission from Robert Klonner:
>I want to install a external tracker and therefore I am copying the 
>templates under /usr/local/share/roundup/templates.
>With the latest version the share folder is now created under 
>/usr/local/lib/python2.7/dist-packages/roundup-2.0.0-
>py2.7.egg/share/roundup/templates

I agree, you should never have to deal with /dist-packages/....

I am unable to look till this afternoon. Can you check to see if the
change from distutils to setuptools occured between the working and
non-working versions.

AFAIK, /usr/local/share/roundup/templates was never a valid directory
for templates.

Does /usr/share/roundup/templates exist?

John K. any ideas here?
msg7270 Author: [hidden] (schlatterbeck) Date: 2021-06-09 15:54
On Wed, Jun 09, 2021 at 02:13:50PM +0000, John Rouillard wrote:
> In message <1623231327.12.0.257353362144.issue2551143@roundup.psfhosted.org>,
> Robert Klonner writes:
> >With the latest version the share folder is now created under 
> >/usr/local/lib/python2.7/dist-packages/roundup-2.0.0-
> >py2.7.egg/share/roundup/templates
> 
> I agree, you should never have to deal with /dist-packages/....
> 
> AFAIK, /usr/local/share/roundup/templates was never a valid directory
> for templates.
> 
> Does /usr/share/roundup/templates exist?

Hmm, depending on which --prefix option you install with (I usually use
--prefix=/usr/local) I would think that /usr/local/share should be fine.
Imo it definitely should *not* install to /usr/share if --prefix is
/usr/local.

But I'm not a setuptools expert :-)

Ralf
-- 
Dr. Ralf Schlatterbeck                  Tel:   +43/2243/26465-16
Open Source Consulting                  www:   www.runtux.com
Reichergasse 131, A-3411 Weidling       email: office@runtux.com
msg7271 Author: [hidden] (rouilj) Date: 2021-06-09 16:48
In message <20210609155446.ptsnq6zwldw6inh6@runtux.com>,
Ralf Schlatterbeck writes:
>On Wed, Jun 09, 2021 at 02:13:50PM +0000, John Rouillard wrote:
>> In message <1623231327.12.0.257353362144.issue2551143@roundup.psfhosted.org>,
>> Robert Klonner writes:
>> >With the latest version the share folder is now created under 
>> >/usr/local/lib/python2.7/dist-packages/roundup-2.0.0-
>> >py2.7.egg/share/roundup/templates
>> 
>> I agree, you should never have to deal with /dist-packages/....
>> 
>> AFAIK, /usr/local/share/roundup/templates was never a valid directory
>> for templates.
>> 
>> Does /usr/share/roundup/templates exist?
>
>Hmm, depending on which --prefix option you install with (I usually use
>--prefix=/usr/local) I would think that /usr/local/share should be fine.
>Imo it definitely should *not* install to /usr/share if --prefix is
>/usr/local.

Yes, if you use --prefix=/usr/local then /usr/local/share/... should work.
I missed that you were using a different prefix.

Still no idea why it's going into dist-packages not site-packages.
msg7272 Author: [hidden] (rouilj) Date: 2021-06-10 00:05
Hi Robert:

Can you try this patch. This appears to be a known issue
with setuptools. It's not a great patch but it seems
to work when running setup directly and running 
pip install --target....

For those following at home, I deduce the prefix by
looking at sys.argv and parsing for --prefix. I recognize
--prefix either as a standalone arg in which case the
prefix directory is the next arg in the argv. Or I recognize
--prefix as a prefix and use the value after stripping the
first 9 characters (--prefix=).

Than I iterate over the data array where the data files are
and make them absolute using the prfix directory.
msg7279 Author: [hidden] (rouilj) Date: 2021-06-10 22:44
See also:

https://github.com/pypa/setuptools/issues/130#issuecomment-398189533

https://setuptools.readthedocs.io/en/latest/userguide/datafiles.html

'sudo pip install .' overrides setuptool settings needed to install
data files outside of the egg install directory. So man pages etc.
are all placed in the right locations.
msg7280 Author: [hidden] (tttech-klonner) Date: 2021-06-11 07:49
Hi John,

thanks for the fast patch. Now I have the following behavior:

1) setup.py + patch and cmd "python setup.py install":
   -> share/roundup goes to /usr/share (instead of egg path) but 
installation is under /usr/local
-> improvement

2) setup.py + patch and cmd "python setup.py install --
prefix=/usr/local":
   -> share/roundup and installation go to /usr/local
-> correct


Behavior 2) would be OK for me but I think when not specifying a prefix 
like in case 1) the installation and the placement of share/ should have 
the same prefix and the default seems to be /usr/local and this behavior 
we had until beginning of April.
Can we get the default prefix somehow into the patch?
msg7281 Author: [hidden] (rouilj) Date: 2021-06-11 19:24
Hi Robert:

Can you back out the other patch and try this new patch.

It uses sysconfig.get_path('platlib') to get the platform
library. Then I parse from the right and return the component
just before the /lib directory.

On python 2.7 it returns /usr/local and on python3.6 it returns
/usr which seem correct.
History
Date User Action Args
2021-06-11 19:24:20rouiljsetfiles: + setup_install_under_prefix_or_slash_usr2.patch
messages: + msg7281
2021-06-11 07:49:50tttech-klonnersetmessages: + msg7280
2021-06-10 22:44:53rouiljsetmessages: + msg7279
2021-06-10 01:36:56rouiljsetkeywords: + Blocker
2021-06-10 00:05:58rouiljsetfiles: + setup_install_under_prefix_or_slash_usr.patch
keywords: + patch
messages: + msg7272
2021-06-09 16:48:23rouiljsetmessages: + msg7271
2021-06-09 15:54:49schlatterbecksetmessages: + msg7270
2021-06-09 14:13:49rouiljsetmessages: + msg7269
2021-06-09 09:35:27tttech-klonnercreate