When choosing Jinja2 as the template and sqlite as the database 
during the tracker creation (roundup-admin install), the resulting 
config.ini file does not include the database backend declaration.  
As the user tries to initialize the tracker (roundup-admin 
initialize), they get an error:  

  File "/usr/bin/roundup-admin", line 3, in <module>
    run()
  File "/usr/lib/python2.7/site-
packages/roundup/scripts/roundup_admin.py", line 49, in run
    sys.exit(tool.main())
  File "/usr/lib/python2.7/site-packages/roundup/admin.py", line 
1635, in main
    ret = self.run_command(args)
  File "/usr/lib/python2.7/site-packages/roundup/admin.py", line 
1504, in run_command
    return self.do_initialise(self.tracker_home, args)
  File "/usr/lib/python2.7/site-packages/roundup/admin.py", line 
526, in do_initialise
    tracker = roundup.instance.open(tracker_home)
  File "/usr/lib/python2.7/site-packages/roundup/instance.py", line 
327, in open
    return Tracker(tracker_home, optimize=optimize)
  File "/usr/lib/python2.7/site-packages/roundup/instance.py", line 
92, in __init__
    self.backend = backends.get_backend(rdbms_backend)
  File "/usr/lib/python2.7/site-
packages/roundup/backends/__init__.py", line 42, in get_backend
    module = __import__(module_name, vars)
ImportError: No module named back_

The error goes away after adding the line to the tracker's 
config.ini:
# Database backend.
# Default:
backend = sqlite

The change is in the [rdbms] section.
Files affected in share/roundup/templates:
  devel/config.ini
  jinja2/config.ini
  responsive/config.ini
These have no config.ini
  classic/
  minimal/
Perhaps they should.

Recall that in roundup 1.5, and perhaps 1.6, the specification of the
database type was in the tracker's db/ directory in the file
backend_name. It was decided that it makes more sense to make it the
config parameter rdbms.backend. Changes to these config.ini files will
complete that change.

Trying to find a simple way to reproduce
and I could verify the problem.

Testing with rev:095db27e80644a2a547f6e66a58207981352323d 2019-03-13

== Reproduction with demo.py failed
python3  ./demo.py -t jinja2 -b sqlite nuke
# and python2

worked and had 
# Database backend.
# Default:
backend = sqlite
in ./demo/config.ini

== Reproduction with install
mkdir test-install/
cd roundup-hq
python3 setup.py install --prefix=../test-install
# setting PATH and PYTHONPATH with fishshell
set -l base /home/bern/roundup/test-install
set PATH $base/bin $PATH
set PYTHONPATH $base/test-install/lib/python3.5/site-packages/:$PYTHONPATH

mkdir test-tracker
roundup-admin -i test-tracker install jinja2 sqlite
roundup-admin -i test-tracker initialise 1234

hg:095db27e80644a2a547f6e66a58207981352323d 2019-03-10 maint-1.6
has the same problem.

hg bisec between maint-1.6 and 1.5.1 results in

The first bad revision is:
changeset:   5041:5251e97b1de0
user:        John Kristensen <john@jerrykan.com>
date:        Mon Dec 22 13:30:20 2014 +1100
summary:     Configure the database backend in the config.ini

John,
seems your change towards 1.6.0 broke the roundup-admin initalise
command. >:)

Ok I think I figured this out.

Bern can you try removing config.ini from the jinja2 template directory 
and ry to reproduce the problem.

I think the is that we supply a config.ini as part of the template. It 
gets copied into the tracker directory. Because it exists it stops the install command from creating a config.ini that matches the cnfig 
settings.

If you delete config.ini from the templates, the install command will 
create a config.ini with the correct setting for backend.

If this works, I suggest one of two paths:

 1) delete config.ini from the template
 2) force the install command to always overwrite the config.ini even
    if one exists.

I am partial to option 1 so we don't have to try to keep the template 
config.ini file in sync at all. Thoughts?

Looking at Tom's comment made me realize that classic/minimal templates 
never had config.ini. I was unable to reproduce crash using those 
templates.

I concluded the right thing was to delete config.ini from the 
templates.

This has been done on both 1.6_maint and trunk.

In addition on trunk, I enhanced the config code to validate
the value for the backend setting against available backends.

So backend = foo doesn't crash but emits and error. Similarly
backend = postgres will emit an error if postgres isn't available.

The comment for the keyword lists available backends.
The rdbms section comment now says that the backend setting is
used by other backends (e.g. if set to anydbm or a future mongodb).

Giving changeset:   5760:f0ca4daf6a18
a test run, like outlined in msg6388:

Result:
roundup-admin -i test-tracker initialise 1234
barfs with 

[..]
python3.5/site-packages/roundup/configuration.py", line 197, in get
    raise OptionUnsetError(self)
roundup.configuration.OptionUnsetError: MAIL_DOMAIN is not set and has
no default

which seems okay, I've now added the setting
vim  test-tracker/config.ini
domain = sometest.intevation.de"
web = 'http://localhost:8080/support/'

env LANG=C roundup-admin -i test-tracker initialise 1234
WARNING: The database is already initialised!
If you re-initialise it, you will lose all the data!
Erase it? Y/N:

This is surprising because the last call gave me a traceback,
but it still initialized the database it seems.

Starting the server does not lead to an index pages to be
rendered.
roundup-server support=test-tracker/

The browser just shows:

{{
  db.issue.renderWith('index',
    sort=[('-', 'activity')], group=[('+', 'priority')],
    filter=['status'],
    columns=['id','activity','title','creator','assignedto', 'status'],
    filterspec={'status':['-1','1','2','3','4','5','6','7']}
  )|u
}}
<!-- SHA: 9b2c4620b74452287d1429df204b0e9e84f6df21 -->

but not the resulting rendering.
So I am not sure if the two surprises about should be part of this issue
or not.

a) initialising tracebacks, but still has set the account (or more?)
b) There is no index page rendered

Additional info:
python3  ./demo.py -t jinja2 -b sqlite nuke
brings up a rendered page as it should

Going back to "open" because initialising of the jinja2 template
does not seems to work (and this is the main use case of this issue as
far as I understood it).

Okay, I can get something rendered when changing to

template_engine = jinja2

so the initialisation progess did not set the right template_engine.
Also missing are the static files needed, there is no css and no javascript:


127.0.0.1 - - [03/Jun/2019 14:33:59] "GET /support/ HTTP/1.1" 200 -
127.0.0.1 - - [03/Jun/2019 14:33:59] code 404, message
/support/@@file/style.css
127.0.0.1 - - [03/Jun/2019 14:33:59] "GET /support/@@file/style.css
HTTP/1.1" 404 -
127.0.0.1 - - [03/Jun/2019 14:33:59] code 404, message
/support/@@file/bootstrap.min.css
127.0.0.1 - - [03/Jun/2019 14:33:59] "GET
/support/@@file/bootstrap.min.css HTTP/1.1" 404 -
127.0.0.1 - - [03/Jun/2019 14:33:59] code 404, message
/support/@@file/bootstrap-responsive.min.css
127.0.0.1 - - [03/Jun/2019 14:33:59] "GET
/support/@@file/bootstrap-responsive.min.css HTTP/1.1" 404 -
127.0.0.1 - - [03/Jun/2019 14:33:59] code 404, message
/support/@@file/bootstrap.min.js
127.0.0.1 - - [03/Jun/2019 14:33:59] "GET
/support/@@file/bootstrap.min.js HTTP/1.1" 404 -
127.0.0.1 - - [03/Jun/2019 14:33:59] code 404, message
/support/@@file/jquery-1.9.0.min.js
127.0.0.1 - - [03/Jun/2019 14:33:59] "GET
/support/@@file/jquery-1.9.0.min.js HTTP/1.1" 404 -
127.0.0.1 - - [03/Jun/2019 14:33:59] code 404, message
/support/@@file/bootstrap.min.js
127.0.0.1 - - [03/Jun/2019 14:33:59] "GET
/support/@@file/bootstrap.min.js HTTP/1.1" 404 -
127.0.0.1 - - [03/Jun/2019 16:33:59] "GET /favicon.ico HTTP/1.1" 200 -

Hi Bern:

In message <1559572570.81.0.0716488299242.issue2551029@roundup.psfhosted.org>,
Bernhard Reiter writes:
>Okay, I can get something rendered when changing to
>
>template_engine = jinja2
>
>so the initialisation progess did not set the right template_engine.

Hmm, ok how to handle this? There is no way for install to know what
template engine should be used.

Should that be another NO DEFAULT setting? If so how is the user to
know which of the three settings (or possibly some hybrid) to use?

Maybe we neeed a config.ini.deploy that gets installed into the
tracker working directory, loaded and then deleted. The
config.ini.deploy would include only:

   template_engine = jinja2

or any other template specific settings.

Then the normal save operation for config.ini will generate an
properly updated config.ini?

Thoughts?

>Also missing are the static files needed, there is no css and no javascript:
>127.0.0.1 - - [03/Jun/2019 14:33:59] "GET /support/ HTTP/1.1" 200 -
>127.0.0.1 - - [03/Jun/2019 14:33:59] code 404, message
>/support/@@file/style.css
>127.0.0.1 - - [03/Jun/2019 14:33:59] "GET /support/@@file/style.css
>HTTP/1.1" 404 -
>127.0.0.1 - - [03/Jun/2019 14:33:59] code 404, message
>/support/@@file/bootstrap.min.css
>127.0.0.1 - - [03/Jun/2019 14:33:59] "GET
>/support/@@file/bootstrap.min.css HTTP/1.1" 404 -

Are there special directions for deploying the jinja2 template?  IIRC
bootstrap is under 3rdparty. Maybe that directory needs to be copied
into html as an extra step?

>127.0.0.1 - - [03/Jun/2019 14:33:59] code 404, message
>/support/@@file/bootstrap-responsive.min.css

Note there is a ticket that Christof is working for updating this
template to boostrap 4. Boostrap 4 doesn't have a separate responsive
css file. It's all in the base.

John,
the documented install command knows the template
(and I am not sure if templates came with some config or settings before 
or what shall be copied. It makes some sense, though.)

roundup-admin -i test-tracker install jinja2 sqlite

In message <1559575768.09.0.524450595549.issue2551029@roundup.psfhosted.org>,
Bernhard Reiter writes:
>Bernhard Reiter added the comment:
>
>the documented install command knows the template
>(and I am not sure if templates came with some config or settings before 
>or what shall be copied. It makes some sense, though.)
>
>roundup-admin -i test-tracker install jinja2 sqlite

The config.ini that was deployed with the templates prevented
roundup-admin install from generating an up to date config (including
the database backend) which is what triggered this ticket in the first
place.

If we can put a "seed" config.ini in place (think of the
initial_data.py file in a tracker) then the seed can define the
template language used to process the html/.* files for the template.

I think I like this solution. But I'll have to see if it is
implementable.

Here is my proposed solution:

 1) add a config_ini.ini file to the jinja2 template. AFAIK it is the
    only one that needs it.
 2) the file consists of (not indented):

     [main]
     template_engine = jinja2

    so it looks like a part of a valid ini file.
 3) change admin.py:AdminTool::do_install() with the attached patch.
    This patch loads config_ini.ini from the template directory
    if it exists. It then adds the settings found except HOME
    to override config file settings like rdbms_backend.
    It then passes these overrides to the init.install module.
    then it deleted the config_ini.ini from the tracker home
    if it exists.

This seems to work for me and created a valid config.ini file.
Bern can you try the patch and add config_ini.ini to your jinja2
template directory with contents as above.

Does the proposed solution also solve the installation of additional
files problem?

In message <1559642995.09.0.335706426922.issue2551029@roundup.psfhosted.org>,
Bernhard Reiter writes:
>
>
>Bernhard Reiter added the comment:
>
>Does the proposed solution also solve the installation of additional
>files problem?

I don't think so.

The installer just does a copytree of the template directory. So the
additional files should be part of the template. If they aren't that's
a separate issue.

Bern:

in config.ini try setting:

static_files = static

It looks like the files that are missing are under the 
tracker_home/static directory. This should allow @@files to find them.
I am pulling back the config.ini to see if there is something else 
missing as I create config_ini.ini files for responsive and jinja2
templates.

Hi Bern:

I pushed rev5762:b76be13e027e with the fix for this. Can you test and
validate that it works in your environment?

It should handle both the missing files (they weren't missing, just
in the static directory and roundup wasn't looking there). It
should also make sure that jinja2 is used as the engine for the
jinja2 template.

The responsive template also got a config_ini.ini file to set 
static_files. That seems to work in my testing.

If others can test as well and verify it handles the issues I would 
appreciate it.

-- rouilj

This works for me. Closing it. If somebody finds more errors we can 
reopen it.