Roundup Tracker - Issues

Issue 2550833

classification
Title: Enhancing the functionality of the CSV export function 'export_csv_names'
Type: rfe Severity: normal
Components: Web interface, User Interface Versions: 1.6, 1.5
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: rouilj Nosy List: anrounham14, ber, rouilj, tekberg
Priority: normal Keywords: patch

Created on 2014-03-09 18:28 by anrounham14, last changed 2019-02-16 20:40 by rouilj.

Files
File name Uploaded Description Edit Remove
ExportCSVNamesAction.py anrounham14, 2014-03-09 18:28
ExportCSVNamesAction-andreas-new.py tekberg, 2014-03-13 20:17 Changes I made to Andreas' version.
actions.py.hgdiff tekberg, 2019-02-06 18:21 ran 'diff roundup/cgi/actions.py' and saved to this file.
do_not_break_output_on_inaccessible_objects.patch rouilj, 2019-02-07 02:15
actions.py-patch.hgdiff tekberg, 2019-02-07 16:23 Patch for devel version of cgi/actions.py. Generarated with hg diff.
unnamed tekberg, 2019-02-07 23:20
Outlook-deqa5wiz.jpg tekberg, 2019-02-07 23:20
unnamed tekberg, 2019-02-08 15:16
Outlook-x2r23z3u.jpg tekberg, 2019-02-08 15:16
Messages
msg5003 Author: [hidden] (anrounham14) Date: 2014-03-09 18:28
The current 'export_csv_names' is very basic. Inspired by the extension
'ExportCSVNames' I have update this functionality a little bit further.
So complex data types are rendered in a more human readable format, i.e.
Mulitlink, Link and Date are rendered in a better string representation.

For example for users the realname and not the id is rendered. The date
string is either only a date (if hour and minute are both zero) or date
plus hour and minutes, a multilink becomes a string of values separated
by ';'. The code is not really beautiful but should serve as a
specification on what should be added to make the CSV export more useful
to my understanding. The function as an extension is added to this issue.
msg5021 Author: [hidden] (ber) Date: 2014-03-13 07:55
Hi Andreas,
thanks for the contribution!

Did you see that Tom also did an improved version of this extension in 
issue2550792 ?

What about we get a unified one and you or Tom tests the other 
changes. I am happy to commit the improvements then. 

Best Regards,
Bernhard
msg5028 Author: [hidden] (tekberg) Date: 2014-03-13 16:20
I like Andreas's version better as a base and I'm in the process of 
making changes. I'll post it here when have finished. Perhaps Andreas can 
comment on my changes.
msg5029 Author: [hidden] (anrounham14) Date: 2014-03-13 18:24
On Thursday 13 March 2014 16:20:39 Tom Ekberg wrote:
> Tom Ekberg added the comment:
> 
> I like Andreas's version better as a base and I'm in the process of
> making changes. I'll post it here when have finished. Perhaps Andreas 
can
> comment on my changes.

@Tom: It is nice that you like the additions I made. I will try to do my best to 
provide feedback. But don't expect too much on the Python part, I am no 
more than an advanced beginner.

@Bernhard: Yes, I had a look at the version issue2550792 and commented 
on it in the mailing list roundup-users@lists.sourceforge.net.

> 
> ________________________________________________
> Roundup tracker <issues@roundup-tracker.org>
> <http://issues.roundup-tracker.org/issue2550833>
> ________________________________________________
msg5030 Author: [hidden] (tekberg) Date: 2014-03-13 20:17
Here is another version with some slight changes.

Any comments are welcome.

This doesn't work for my custom object - it outputs ids for everything :-
( but it does work for the issue object. I have other higher priority 
projects to work on but I thought I'd at least put my updated version 
out.
msg5031 Author: [hidden] (ber) Date: 2014-03-14 08:08
Tom, Andreas!

On Thursday 13 March 2014 16:20:39 Tom Ekberg wrote:
> I like Andreas's version better as a base and I'm in the process of
> making changes. I'll post it here when have finished. 

Maybe we can close issue2550792 in favour of this one, then. 

Another idea: I see that sf now has a "clone your own copy" version.
Maybe you could try this and jointly develop the extension until is it 
ready. :)

> @Bernhard: Yes, I had a look at the version issue2550792 and commented
> on it in the mailing list roundup-users@lists.sourceforge.net.

Thanks for doing so. Sometimes I read the devel mailinglist before
the users list. 

Best,
Bernhard
msg5034 Author: [hidden] (tekberg) Date: 2014-03-14 13:56
OK. I'll bite. What is "clone your own copy"?
msg5035 Author: [hidden] (ber) Date: 2014-03-14 14:22
If you got to http://sourceforge.net/p/roundup/code/ci/default/tree/
ther is a button "fork". I'm a bit uneasy because other forges offer
the ability to "clone" your personaly copy and then do "push requests" to 
the master.
msg5795 Author: [hidden] (rouilj) Date: 2016-07-09 20:55
Tom or Andreas:

Have you guys done any more with this?

I think replacing the classic tracker's export CSV with this one
would be a suitable patch.

Have either of you looked at:

http://www.roundup-tracker.org/cgi-bin/moin.cgi/ExportCSVNames_Extended

that was uploaded June 2016. Maybe it handles Tom's issues with his
custom object?
msg5796 Author: [hidden] (rouilj) Date: 2016-07-09 21:01
Hmm, except for some logging removal the version on the wiki seems
the same as file1532 - andreas-new attached here.
msg5807 Author: [hidden] (anrounham14) Date: 2016-07-10 10:09
Hello John,

I didn't get around to look into the roundup repository and how to add something to it. Neither did I do any further updates. I only added the CSV export script to the wiki since I didn't know how to propose it as a patch. It might not be also the most beautiful Python code since I am doing Python programming only once in a while.

Hope this explains matters better,
Andreas

On Saturday, July 09, 2016 08:55:17 PM John Rouillard wrote:
> 
> John Rouillard added the comment:
> 
> Tom or Andreas:
> 
> Have you guys done any more with this?
> 
> I think replacing the classic tracker's export CSV with this one
> would be a suitable patch.
> 
> Have either of you looked at:
> 
> http://www.roundup-tracker.org/cgi-bin/moin.cgi/ExportCSVNames_Extended
> 
> that was uploaded June 2016. Maybe it handles Tom's issues with his
> custom object?
> 
> ----------
> nosy: +rouilj
> 
> ________________________________________________
> Roundup tracker <issues@roundup-tracker.org>
> <http://issues.roundup-tracker.org/issue2550833>
> ________________________________________________
msg5835 Author: [hidden] (tekberg) Date: 2016-07-11 14:07
I haven't done any more this this.

>> Maybe it handles Tom's issues with his custom object?

I don't remember what custom object I was having trouble with, so 
obviously it isn't a big deal now.
msg6324 Author: [hidden] (tekberg) Date: 2019-02-06 18:21
Incorporated this into my cgi/actions.py replacing the CSV export
function that outputs ids for objects with one that outputs names. The
changes are in actions.py.hgdiff.
msg6328 Author: [hidden] (rouilj) Date: 2019-02-07 02:15
There is another bug in ExportCSVNamesAction.

If there is an issue in your list that you don't have access to, the
current code will result in every csv element being reported
as [hidden].

Tom this may be the bug you ran into with custom objects. As soon as you
hit any
object that the user can't view everything is set to [hidden]. If the
first custom
item is not accessibly, you get a nice matrix of [hidden] even for
entries the user can
access.

This little bit of code:

                  if not self.hasPermission(self.permissionType,
itemid=itemid,
                        classname=request.classname, property=col):
                    represent[col] = repr_no_right(request.classname, col)

results in all the represent[col] set to repr_no_right on the first
inaccessible object/issue.

I have attached a file with two patches that applies to my copy of
file1532 that
fixes this by assigning the current representation to a new variable and
not changing represent[col].

The second patch in the file hides any object that the user doesn't have
access to.
Otherwise you get a row of [hidden] output for every field. It is meant
to mimic the
access check in the html so the row is not displayed in any way.
msg6329 Author: [hidden] (tekberg) Date: 2019-02-07 16:23
Applied John's changes to the devel version of cgi/actions.py and
uploaded the patch. Thanks John. We don't see that problem at my site.
msg6331 Author: [hidden] (rouilj) Date: 2019-02-07 22:12
Tom Ekberg on 2019-02-07 11:23 said:
> Thanks John. We don't see that problem at my site.

The easiest way to see this is to use a non-admin user
to do a csv export of the user class. Include the roles
and address properties. The roles and address are not
usually visible to non-admin users.

Without the patch you should see a column of [hidden] for
role and email. After the patch, you should see the role and email
for the user that is doing the csv export and [hidden] everywhere else.
msg6332 Author: [hidden] (tekberg) Date: 2019-02-07 23:20
John,

I see how to display a list of users by clicking on 'Class List' and selecting the user object. The only page that has an export_csv action is index.index.html. I suppose I could add it to the user.index.html page and experiment with that.

[cid:a24a59b5-522a-471e-b24a-1e44aca320d5]

________________________________
From: issues=roundup-tracker.org@roundup.psfhosted.org <issues=roundup-tracker.org@roundup.psfhosted.org> on behalf of John Rouillard <issues@roundup-tracker.org>
Sent: Thursday, February 7, 2019 2:13 PM
To: Tom Ekberg
Subject: [issue2550833] Enhancing the functionality of the CSV export function 'export_csv_names'

John Rouillard added the comment:

Tom Ekberg on 2019-02-07 11:23 said:
> Thanks John. We don't see that problem at my site.

The easiest way to see this is to use a non-admin user
to do a csv export of the user class. Include the roles
and address properties. The roles and address are not
usually visible to non-admin users.

Without the patch you should see a column of [hidden] for
role and email. After the patch, you should see the role and email
for the user that is doing the csv export and [hidden] everywhere else.

_________________________________________________
Roundup tracker <issues@roundup-tracker.org>
<https://issues.roundup-tracker.org/issue2550833>
_________________________________________________
msg6333 Author: [hidden] (rouilj) Date: 2019-02-08 00:21
Hi Tom:

In message <DM6PR08MB477820F4B1E8B8B6530A7EE4CA680@DM6PR08MB4778.namprd08.prod.
outlook.com>,
Tom Ekberg writes:
>
>Tom Ekberg added the comment:
>I see how to display a list of users by clicking on 'Class List' and
>selecting the user object. The only page that has an export_csv
>action is index.index.html. I suppose I could add it to the
>user.index.html page and experiment with that.

Just edit the url. E.G. my base directory is:

  https://rouilj.dynamic-dns.net/demo

when displaying csv for issues, I get:

  /issue?@action=export_csv&@columns=id,activity,...[filter items removed]

so to dump users, I append:

  /user?@action=export_csv&@columns=username,address,roles,realname,phone,

and get:

  "username","address","roles","realname","phone"
  "admin","[hidden]","[hidden]","The Admin","603-555-4423"
  "anonymous","[hidden]","[hidden]","A. Nonymous","555-555-5555"
  "demo","demo@example.com","User","De Mo","231-555-5544"
  "agent","[hidden]","[hidden]","James Bond","041-555-2345"
  "provisional","[hidden]","[hidden]","Eye Cansee","321-555-5445

without the patch, the "demo" line is:

  "demo","[hidden]","[hidden]","De Mo","231-555-5544"

since I was unable to see the address and roles for the admin user
which bleeds through to the demo user.

One thing I have been thinking about is adding support for a

   @showid=username,other_field

url parameter. The intent is to allow the userid (or other property)
to be displayed as the userid can be reused to create rest and other
url's.

Setting @showid=all would disable all ID -> name mappings similar to
the current export_csv code. Not sure when I will try implementing it,
or how useful it is given rest support but...
msg6334 Author: [hidden] (tekberg) Date: 2019-02-08 15:16
John,

Works great with your code that I applied to actions.py. All show '[hidden]' for the email and role except my email and role which are visible.

[cid:a1b9f468-232c-48c2-a83d-4e5bbe07cc22]

________________________________
From: issues=roundup-tracker.org@roundup.psfhosted.org <issues=roundup-tracker.org@roundup.psfhosted.org> on behalf of John Rouillard <issues@roundup-tracker.org>
Sent: Thursday, February 7, 2019 4:21 PM
To: Tom Ekberg
Subject: [issue2550833] Enhancing the functionality of the CSV export function 'export_csv_names'

John Rouillard added the comment:

Hi Tom:

In message <DM6PR08MB477820F4B1E8B8B6530A7EE4CA680@DM6PR08MB4778.namprd08.prod.
outlook.com>,
Tom Ekberg writes:
>
>Tom Ekberg added the comment:
>I see how to display a list of users by clicking on 'Class List' and
>selecting the user object. The only page that has an export_csv
>action is index.index.html. I suppose I could add it to the
>user.index.html page and experiment with that.

Just edit the url. E.G. my base directory is:

  https://rouilj.dynamic-dns.net/demo

when displaying csv for issues, I get:

  /issue?@action=export_csv&@columns=id,activity,...[filter items removed]

so to dump users, I append:

  /user?@action=export_csv&@columns=username,address,roles,realname,phone,

and get:

  "username","address","roles","realname","phone"
  "admin","[hidden]","[hidden]","The Admin","603-555-4423"
  "anonymous","[hidden]","[hidden]","A. Nonymous","555-555-5555"
  "demo","demo@example.com","User","De Mo","231-555-5544"
  "agent","[hidden]","[hidden]","James Bond","041-555-2345"
  "provisional","[hidden]","[hidden]","Eye Cansee","321-555-5445

without the patch, the "demo" line is:

  "demo","[hidden]","[hidden]","De Mo","231-555-5544"

since I was unable to see the address and roles for the admin user
which bleeds through to the demo user.

One thing I have been thinking about is adding support for a

   @showid=username,other_field

url parameter. The intent is to allow the userid (or other property)
to be displayed as the userid can be reused to create rest and other
url's.

Setting @showid=all would disable all ID -> name mappings similar to
the current export_csv code. Not sure when I will try implementing it,
or how useful it is given rest support but...

_________________________________________________
Roundup tracker <issues@roundup-tracker.org>
<https://issues.roundup-tracker.org/issue2550833>
_________________________________________________
msg6347 Author: [hidden] (rouilj) Date: 2019-02-16 20:40
Fixed in be99aa02c616. I left the original id based csv export code in
place for people that choose to use it. So:

        ('export_csv',  actions.ExportCSVAction),
        ('export_csv_id',  actions.ExportCSVWithIdAction),

are the actions with the two different outputs.
History
Date User Action Args
2019-02-16 20:40:58rouiljsetstatus: new -> closed
versions: + 1.6
messages: + msg6347
priority: normal
assignee: rouilj
resolution: fixed
2019-02-08 15:16:44tekbergsetfiles: + unnamed, Outlook-x2r23z3u.jpg
messages: + msg6334
2019-02-08 00:21:48rouiljsetmessages: + msg6333
2019-02-07 23:20:13tekbergsetfiles: + unnamed, Outlook-deqa5wiz.jpg
messages: + msg6332
2019-02-07 22:12:56rouiljsetmessages: + msg6331
2019-02-07 16:23:57tekbergsetfiles: + actions.py-patch.hgdiff
messages: + msg6329
2019-02-07 02:20:57rouiljsetfiles: - unnamed
2019-02-07 02:18:43rouiljsetfiles: - unnamed
2019-02-07 02:15:14rouiljsetfiles: + do_not_break_output_on_inaccessible_objects.patch
messages: + msg6328
2019-02-06 18:21:48tekbergsetfiles: + actions.py.hgdiff
messages: + msg6324
2016-07-11 14:07:31tekbergsetmessages: + msg5835
2016-07-10 10:09:03anrounham14setfiles: + unnamed
messages: + msg5807
2016-07-09 21:01:28rouiljsettype: behavior -> rfe
2016-07-09 21:01:14rouiljsetmessages: + msg5796
2016-07-09 20:55:17rouiljsetnosy: + rouilj
messages: + msg5795
2014-03-14 14:22:31bersetmessages: + msg5035
2014-03-14 14:01:49berlinkissue2550792 superseder
2014-03-14 13:56:37tekbergsetmessages: + msg5034
2014-03-14 08:08:03bersetmessages: + msg5031
2014-03-13 20:17:42tekbergsetfiles: + ExportCSVNamesAction-andreas-new.py
messages: + msg5030
2014-03-13 18:24:27anrounham14setfiles: + unnamed
messages: + msg5029
2014-03-13 16:20:39tekbergsetmessages: + msg5028
2014-03-13 07:55:48bersetkeywords: + patch
nosy: + ber, tekberg
messages: + msg5021
2014-03-09 18:28:32anrounham14create