(not sure what Type rfe is, sorry).

We have a tracker with about 3,000 "custcomp" (customer company) entries and about 35,000 
issues overall. The issue.item.html for creating or modifying an issue loads in about 1 second, 
but the issue.search.html takes 9 seconds. Most of that time is in rendering the custcomp 
pulldown for searching.

I'd like to know how to either make this faster or replace the search retrieval with the item 
retrieval.

In issue.item.html, the following is the line of interest:

<td tal:content="structure context/custcomp/menu">custcomp</td>

And the corresponding section of issue.search.html is:

<tr tal:define="name string:custcomp;                                                                                   
                db_klass string:custcomp;                                                                               
                db_content string:name;">
  <th>Company:</th>
  <td metal:use-macro="search_select"></td>
  <td metal:use-macro="column_input"></td>
  <td metal:use-macro="sort_input"></td>
  <td metal:use-macro="group_input"></td>
</tr>

If I simply replace search_select line with the context/custcomp/menu line, the select list 
renders very fast, but of course then the search doesn't work.

In message <1358965339.4.0.483830445644.issue2550790@psf.upfronthosting.co.za> 
<1358965339.4.0.483830445644.issue2550790@psf.upfronthosting.co.za>,
David Hancock writes:
>New submission from David Hancock:
>
>(not sure what Type rfe is, sorry).

Request For Enhancement (i.e. new feature request).

> We have a tracker with about 3,000 "custcomp" (customer company)
> entries and about 35,000 issues overall. The issue.item.html
> for creating or modifying an issue loads in about 1 second, 
> but the issue.search.html takes 9 seconds. Most of that time is
> in rendering the custcomp pulldown for searching.
>
>I'd like to know how to either make this faster or replace the search
>retrieval with the item retrieval.
>
>In issue.item.html, the following is the line of interest:
>
><td tal:content="structure context/custcomp/menu">custcomp</td>
>
>And the corresponding section of issue.search.html is:
>
><tr tal:define="name string:custcomp;                                       
>                                            
>                db_klass string:custcomp;                                   
>                                            
>                db_content string:name;">
>  <th>Company:</th>
>  <td metal:use-macro="search_select"></td>
>  <td metal:use-macro="column_input"></td>
>  <td metal:use-macro="sort_input"></td>
>  <td metal:use-macro="group_input"></td>
></tr>
>
>If I simply replace search_select line with the context/custcomp/menu line,
>the select list renders very fast, but of course then the search doesn't
> work.

One thing to look at is to see what the html generated with the
original template and the modified template look like.

That may give an indication of what component is missing that stops
the two from being tied together. Then you can use actual html rather
than the macros to replicate the original components.

I think this may also be another instance of issue2550702.

David,
thanks for creating the issue. What would be the easiest
way for someone like me to actually see your issue in a running
tracker that I can examine? Just start one with postgresql db
and then fill in 35K issues and 3K accounts?

John,
yes, issue2550702 could be related, but this one may be more specific.

Sorry to be late responding. Unfortunately, our tracker is behind our corporate firewall, so to test 
would require populating a few thousand customers and a similar number of issues. I don't even 
think the number of issues is important, because the problem manifests itself on the search 
page (with good response on the item page), and not on any of the indexes that list issues.

I haven't been successful trying to fix differences between the two menu-generation methods. 
but I can't fiddle too much as this tracker is in use 24x7.

David,
as I've said, if you are making this easier for a developer
to see on a vanilla tracker that is close enough to your problem, 
we can easier come up with a workaround.
So if you start with a vanilla tracker 1.4.21 or hg tip you say
and then show us a short way to show the problem. That is the next step.

Best,
Bernhard

Might the mechanism discussed in:

   http://www.roundup-tracker.org/cgi-bin/moin.cgi/UserMenu

be a workaround? Basically it uses a detector to generate
the html statically on user changes and defines a tal macro
that can just be substituted when the page is rendered.

So it takes the generation of the list by querying the database
out of the flow of displaying the page.

Not the best solution I agree but it is a basic caching
mechanism.

David another way to handle this is to pre-cache the user list
as described in:

  http://www.roundup-tracker.org/cgi-bin/moin.cgi/UserMenu

Would that solve your problem?

I appreciate all the help and ideas. I think the approach in UserMenu on
the Wiki will solve the problem. I will give it a try soon.

It's still interesting that the retrieval of essentially the same list is
so fast on the issue.item.html page and so slow on the issue.search.html
page.

Cheers! 
-- 
David Hancock | dhancock@arinc.com

There is some discussion on how to improve this situation occurring on
the roundup-devel list.

See:

  https://sourceforge.net/p/roundup/mailman/message/31511901/

titled: warm_cache call initial results and failures

for a prototype of a mechanism to replace many single
item loads with a batch item load that provides a speedup.

Also see:

  https://sourceforge.net/p/roundup/mailman/message/31511633/

titled: Re: load spikes due to user lookup

for tuning to make sure that the rdbms cache is correctly tuned for
your workload along with background discussion on the thread
that lead to the warm_cache implementation.