Roundup Tracker - Issues

Issue 2550757

classification
Avoid duplicate relative tracker homes when reinitialising.
Type: behavior Severity: normal
Components: Documentation, Mail interface, Command-line interface Versions:
process
Status: fixed fixed
:
: rouilj : rouilj, wking
Priority: : patch

Created on 2012-05-15 18:11 by wking, last changed 2016-07-23 21:21 by rouilj.

Files
File name Uploaded Description Edit Remove
avoid-duplicate-relative-tracker-homes-when-reinitialising.patch wking, 2012-05-15 18:11
document-mailgw-help-and-confirmation-handling.patch wking, 2012-05-15 19:48
Messages
msg4558 Author: [hidden] (wking) Date: 2012-05-15 18:11
With an already-intialised roundup configured with:

  …
  [main]
  database: rel/db
  …

Calls to

  $ roundup-admin -i inst/home initialise

from `/cwd/` will remove

  /cwd/inst/home/rel/db/backend_name

and create

  /cwd/inst/home/inst/home/rel/db/backend_name

which is obviously broken, due to duplicate `inst/home`s.

The duplication occurs because `database` is a `FilePathOption`, so
`tracker.config.DATABASE` returns `inst/home/rel/db` because
`FilePathOption.get` prepends `self.config["HOME"]`.  In the original
implementation, `tracker_home` is passed to `write_select_db` as
`instance_home`, and then prepended to `dbdir`, so you get duplicate
home values.

This is not a problem when either the configured `database` path or
`tracker_home` are absolute, because then the value returned by
`tracker.config.DATABASE` will be absolute, and the additional
`os.path.join` prepend in `write_select_db` has no effect.
msg4559 Author: [hidden] (wking) Date: 2012-05-15 19:48
The email gateway currently handles two operations besides processing
new issues and messages associated with issues:

* responding to help requests
* processing user confirmation replies

This patch adds documentation for these operations to the user guide
and mailgw docstring.
msg4577 Author: [hidden] (wking) Date: 2012-06-28 15:59
On Thu, Jun 28, 2012 at 04:43:12PM +0200, Bernhard Reiter wrote:
> Thanks for sending the patches! They do not seem to have become attached to
> http://issues.roundup-tracker.org/issue2550757

Sorry, that's because my subjects:

  [issue2550757] [PATCH 0 of 3] …

Raised:

  Failed issue tracker submission
  …
	There were problems handling your subject line argument list:
  - not of form [arg=value,value,...;arg=value,value,...]

Browsing through mailgw.py, it looks like the problem has to do with
subject suffix parsing.  Perhaps I should revise my help patch (which
is accidentally attached to issue2550757)?  In general, I think
parse_subject() would benefit from having a more structured format
(with a BNF description?).  It's not entirely clear to me how I should
have altered the `[PATCH 0 of 3]` bit to avoid suffix processing.
Perhaps I should have made it `(PATCH 0 of 3)`?

Anyhow, I'll resend my patchbomb so it gets attached to the issue.
msg4578 Author: [hidden] (wking) Date: 2012-06-28 16:04
I was talking to Thomas on IRC and he suggested it would be easier to
close some bugs I opened if I sent along more comprehensive patches.
Well, here's a patchbomb for issue2550757.

Note that it doesn't address the mailgw help patch that I accidenally
attached to that bug via an inadvised “reply”.
msg4579 Author: [hidden] (wking) Date: 2012-06-28 16:05
# HG changeset patch
# User W. Trevor King <wking@tremily.us>
# Date 1340389836 14400
# Node ID 139a50e588d0052ee04cc75dcb78e05e8c16dedc
# Parent  45ac4cd1a3816199b188f969f5823301641c0b48
Add -f (force) option to roundup-admin.

This is necessary for the upcoming testAdminDuplicateInitialisation,
but it also makes it easier to mindlessly select defaults when you're
getting started.

While I was at it, I consolidated the "choose a value from a list"
logic into AdminTool._get_choice()

If this seems like a bad idea, we could keep the AdminTool.force
attribute, but not expose it through Getopt.

diff -r 45ac4cd1a381 -r 139a50e588d0 CHANGES.txt
--- a/CHANGES.txt	Sun Jun 17 20:29:57 2012 +0800
+++ b/CHANGES.txt	Fri Jun 22 14:30:36 2012 -0400
@@ -7,7 +7,10 @@

 Features:

-- ...
+- Add -f (force) option to roundup-admin.  This is necessary for
+  testAdminDuplicateInitialisation, but it also makes it easier to
+  mindlessly select defaults when you're getting started.  Like most
+  --force options, this should probably be used sparingly.

 Fixed:

diff -r 45ac4cd1a381 -r 139a50e588d0 roundup/admin.py
--- a/roundup/admin.py	Sun Jun 17 20:29:57 2012 +0800
+++ b/roundup/admin.py	Fri Jun 22 14:30:36 2012 -0400
@@ -72,6 +72,7 @@
         self.tracker_home = ''
         self.db = None
         self.db_uncommitted = False
+        self.force = None

     def get_class(self, classname):
         """Get the class - raise an exception if it doesn't exist.
@@ -113,6 +114,7 @@
  -i instance home  -- specify the issue tracker "home directory" to administer
  -u                -- the user[:password] to use for commands
  -d                -- print full designators not just class id numbers
+ -f                -- force potentially dangerous action, never prompt
  -c                -- when outputting lists of data, comma-separate them.
                       Same as '-S ","'.
  -S <string>       -- when outputting lists of data, string-separate them
@@ -379,36 +381,35 @@
         # check for both old- and new-style configs
         if list(filter(os.path.exists, [config_ini_file,
                 os.path.join(tracker_home, 'config.py')])):
-            ok = raw_input(_(
+            if not self.force:
+                ok = raw_input(_(
 """WARNING: There appears to be a tracker in "%(tracker_home)s"!
 If you re-install it, you will lose all the data!
 Erase it? Y/N: """) % locals())
-            if ok.strip().lower() != 'y':
-                return 0
+                if ok.strip().lower() != 'y':
+                    return 0

             # clear it out so the install isn't confused
             shutil.rmtree(tracker_home)

         # select template
         templates = self.listTemplates()
-        template = len(args) > 1 and args[1] or ''
-        if template not in templates:
-            print _('Templates:'), ', '.join(templates)
-        while template not in templates:
-            template = raw_input(_('Select template [classic]: ')).strip()
-            if not template:
-                template = 'classic'
+        template = self._get_choice(
+            list_name=_('Templates:'),
+            prompt=_('Select template'),
+            options=templates,
+            argument=len(args) > 1 and args[1] or '',
+            default='classic')

         # select hyperdb backend
         import roundup.backends
         backends = roundup.backends.list_backends()
-        backend = len(args) > 2 and args[2] or ''
-        if backend not in backends:
-            print _('Back ends:'), ', '.join(backends)
-        while backend not in backends:
-            backend = raw_input(_('Select backend [anydbm]: ')).strip()
-            if not backend:
-                backend = 'anydbm'
+        backend = self._get_choice(
+            list_name=_('Back ends:'),
+            prompt=_('Select backend'),
+            options=backends,
+            argument=len(args) > 2 and args[2] or '',
+            default='anydbm')
         # XXX perform a unit test based on the user's selections

         # Process configuration file definitions
@@ -457,6 +458,21 @@
 }
         return 0

+    def _get_choice(self, list_name, prompt, options, argument, default=None):
+        if default is None in options:
+            default = options[0]  # just pick the first one
+        if argument in options:
+            return argument
+        if self.force:
+            return default
+        raw_prompt = '%s:' % (name.title())
+        print '%s: %s' % (list_name, ', '.join(options))
+        while argument not in options:
+            argument = raw_input('%s [%s]:' % (prompt, default))
+            if not argument:
+                return default
+        return argument
+
     def do_genconfig(self, args):
         ''"""Usage: genconfig <filename>
         Generate a new tracker config file (ini style) with default values
@@ -495,12 +511,13 @@

         # is there already a database?
         if tracker.exists():
-            ok = raw_input(_(
+            if not self.force:
+                ok = raw_input(_(
 """WARNING: The database is already initialised!
 If you re-initialise it, you will lose all the data!
 Erase it? Y/N: """))
-            if ok.strip().lower() != 'y':
-                return 0
+                if ok.strip().lower() != 'y':
+                    return 0

             backend = tracker.get_backend_name()

@@ -1447,7 +1464,10 @@

         # make sure we have a tracker_home
         while not self.tracker_home:
-            self.tracker_home = raw_input(_('Enter tracker home: ')).strip()
+            if not self.force:
+                self.tracker_home = raw_input(_('Enter tracker home: ')).strip()
+            else:
+                self.tracker_home = os.curdir

         # before we open the db, we may be doing an install or init
         if command == 'initialise':
@@ -1521,7 +1541,7 @@

     def main(self):
         try:
-            opts, args = getopt.getopt(sys.argv[1:], 'i:u:hcdsS:vV')
+            opts, args = getopt.getopt(sys.argv[1:], 'i:u:hcdfsS:vV')
         except getopt.GetoptError, e:
             self.usage(str(e))
             return 1
@@ -1566,6 +1586,8 @@
                 self.separator = ' '
             elif opt == '-d':
                 self.print_designator = 1
+            elif opt == '-f':
+                self.force = True

         # if no command - go interactive
         # wrap in a try/finally so we always close off the db
msg4580 Author: [hidden] (wking) Date: 2012-06-28 16:06
# HG changeset patch
# User W. Trevor King <wking@tremily.us>
# Date 1340390137 14400
# Node ID c43a1c8c5a403fbab2f03182e02d2f13c95a56f7
# Parent  139a50e588d0052ee04cc75dcb78e05e8c16dedc
Add testAdminDuplicateInitialisation to db_test_base.py

This shows how issue2550757 is currently failing.

diff -r 139a50e588d0 -r c43a1c8c5a40 test/db_test_base.py
--- a/test/db_test_base.py	Fri Jun 22 14:30:36 2012 -0400
+++ b/test/db_test_base.py	Fri Jun 22 14:35:37 2012 -0400
@@ -1854,6 +1854,33 @@
             roundup.admin.sys = sys
             shutil.rmtree('_test_export')

+    # test duplicate relative tracker home initialisation (issue2550757)
+    def testAdminDuplicateInitialisation(self):
+        import roundup.admin
+        output = []
+        def stderrwrite(s):
+            output.append(s)
+        roundup.admin.sys = MockNull ()
+        t = '_test_initialise'
+        try:
+            roundup.admin.sys.stderr.write = stderrwrite
+            tool = roundup.admin.AdminTool()
+            tool.force = True
+            args = (None, 'classic', 'anydbm',
+                    'MAIL_DOMAIN=%s' % config.MAIL_DOMAIN)
+            tool.do_install(t, args=args)
+            args = (None, 'mypasswd')
+            tool.do_initialise(t, args=args)
+            tool.do_initialise(t, args=args)
+            try:  # python >=2.7
+                self.assertNotIn(t, os.listdir(t))
+            except AttributeError:
+                self.assertFalse('db' in os.listdir(t))
+        finally:
+            roundup.admin.sys = sys
+            if os.path.exists(t):
+                shutil.rmtree(t)
+
     def testAddProperty(self):
         self.db.issue.create(title="spam", status='1')
         self.db.commit()
msg4581 Author: [hidden] (wking) Date: 2012-06-28 16:07
# HG changeset patch
# User W. Trevor King <wking@tremily.us>
# Date 1340389913 14400
# Node ID fb6e92f5f59d50ef2c1cc378f5b68948ccc46d03
# Parent  c43a1c8c5a403fbab2f03182e02d2f13c95a56f7
Committed edited fix for issue2550757 by W. Trevor King.

diff -r c43a1c8c5a40 -r fb6e92f5f59d CHANGES.txt
--- a/CHANGES.txt	Fri Jun 22 14:35:37 2012 -0400
+++ b/CHANGES.txt	Fri Jun 22 14:31:53 2012 -0400
@@ -14,6 +14,8 @@

 Fixed:

+- issue2550757: Avoid duplicate relative tracker homes when reinitialising
+  (W. Trevor King)
 - issue2550574: Restore sample detectors removed in roundup 1.4.9
   (Thomas Arendsen Hein)
 - Prevent AttributeError when removing all roles of a user
diff -r c43a1c8c5a40 -r fb6e92f5f59d roundup/admin.py
--- a/roundup/admin.py	Fri Jun 22 14:35:37 2012 -0400
+++ b/roundup/admin.py	Fri Jun 22 14:31:53 2012 -0400
@@ -525,7 +525,7 @@
             tracker.nuke()

             # re-write the backend select file
-            init.write_select_db(tracker_home, backend, tracker.config.DATABASE)
+            init.write_select_db(os.curdir, backend, tracker.config.DATABASE)

         # GO
         tracker.init(password.Password(adminpw, config=tracker.config))
msg4582 Author: [hidden] (wking) Date: 2012-06-29 13:29
This patch set is also available as branch `issue2550757` of
  https://bitbucket.org/wtking/roundup
msg5503 Author: [hidden] (rouilj) Date: 2016-04-09 03:41
It looks like the reinitialize issue is not present in current code.
I can't even find enough context to try to apply the patch.

Using a demo instance, I edited main to put the database in
rel/db and from one directory above demo ran:

python ./roundup/scripts/roundup_admin.py -i ../roundup.hg/demo initialise

It seemed to complete ok.

I am planning on applying the doc changes later.
msg5557 Author: [hidden] (rouilj) Date: 2016-05-01 01:34
doc patch for mailgw and doc strings applied in commit d995ee7d49bf,
msg5887 Author: [hidden] (rouilj) Date: 2016-07-23 21:21
Your original patch for the backend file is not needed. With 1.6
the database backend is in the config.ini file for the tracker.

I did incorporate your patches to roundup admin to use _get_choice
(after removing a line that uses name.title() which doesn't exist).

I also removed access to the -f flag from the command line.
I have driven roundup_admin from shell here scripts and other
automation mechanisms, so I don't think -f is required at the
cli.

The test you supplied works fine.

See commit: 6ae426092d7d

-- rouilj
History
Date User Action Args
2016-07-23 21:21:48rouiljsetstatus: new -> fixed
resolution: fixed
messages: + msg5887
2016-05-01 01:39:27rouiljsetassignee: rouilj
type: behavior
components: + Documentation
2016-05-01 01:34:27rouiljsetmessages: + msg5557
2016-04-09 03:41:47rouiljsetnosy: + rouilj
messages: + msg5503
components: + Mail interface, Command-line interface
2012-06-29 13:29:35wkingsetmessages: + msg4582
title: (PATCH 3 of 3) Committed edited fix for issue2550757 by W. Trevor King -> Avoid duplicate relative tracker homes when reinitialising.
2012-06-28 16:07:17wkingsetmessages: + msg4581
title: (PATCH 2 of 3) Add testAdminDuplicateInitialisation to db_test_base.py -> (PATCH 3 of 3) Committed edited fix for issue2550757 by W. Trevor King
2012-06-28 16:06:23wkingsetmessages: + msg4580
title: (PATCH 1 of 3) Add -f (force) option to roundup-admin -> (PATCH 2 of 3) Add testAdminDuplicateInitialisation to db_test_base.py
2012-06-28 16:05:27wkingsetmessages: + msg4579
title: (PATCH 0 of 3) Avoid duplicate relative tracker homes -> (PATCH 1 of 3) Add -f (force) option to roundup-admin
2012-06-28 16:04:04wkingsetmessages: + msg4578
2012-06-28 15:59:44wkingsetmessages: + msg4577
title: Document mailgw help and confirmation handling -> (PATCH 0 of 3) Avoid duplicate relative tracker homes
2012-05-15 19:48:28wkingsetfiles: + document-mailgw-help-and-confirmation-handling.patch
messages: + msg4559
title: Avoid duplicate relative tracker homes when reinitialising. -> Document mailgw help and confirmation handling
2012-05-15 18:11:59wkingcreate