On Tue, Mar 23, 2021 at 09:25:28PM +0000, John Rouillard wrote:
> >This is implemented with filter_iter a long time ago.
> >
> >But it isn't used in the html framework. 
> 
> What has to be done to get this implemented? I see that tests in
> test/db_test_base.py do include calls to filter_iter.
> 
> Do we need a test for the templating system
> (e.g. LinkHTMLProperty::menu())? The data in the database and schema
> probably need to be set up to test the conditions:

Yes, that would be cool, I think the HTML layer is currently largely
untested.

> Question should the html be tested as a string or by comparing the two
> parse trees. I think comparing by tree is more robust, but I don't
> know of any tools in python to do that. Simple html parser, difflib etc.
> don't quite fill the niche.

Beautiful soup (bs4) is a nice tool for parsing html and coding
regression tests. But this would be an external dependency. Note that we
already optionally use it for parsing incoming HTML emails.
On the other hand the created string is not changing too much, so a
string compare would probably be enough.

> While I was looking at the code, we support xhtml and html4. At this
> point I think xhtml is kind of dead, but we probably should maintain
> it.  However should we add html5 to the output format? In most cases
> html4 and html5 would be the same but .... If we do this should we
> change the templates to html5 format (doctype declaration, lang=
> attribute etc.).

Yes that would be nice. I think the generated html type can currently be
configured. I'd go for html5 for the standard templates of the classic
tracker.

> Also I noticed that there is a checklist() spec commented out at the
> end of the menu() definition. I see a reference to checklist in the
> customizing.doc and in doc/html_extra/spec.html. Do you know if there
> was ever an implementation for this (series of checkboxes to select
> options (e.g. for search) or radiobuttons (with a none/unset value)
> for setting the value for a Link)? Would have been nice to have as I
> had to implement these at the templating level.

No I haven't used this ever.
I'm mainly using classhelp, this pops up a separate window where you can
select things that end up in the input field when pressing a button.

> >This should be clarified: If the code in the loop accesses Multilinks
> >properties of the node, those would be fetched separately. Which
> >certainly does *not* happen in the original use-case of optimizing
> >LinkHTMLProperty.menu and will not happen in many other use cases (like,
> >e.g., fetching rows for display in index templates in the web interface
> >in most cases I've seen).
> 
> To make sure I understand. If issue.index.html included the superseder
> property, that would trigger this addition set of lookups (one per id
> in the superseder list) correct?

Code:
for id in db.someclass.filter_iter(...):
    node = db.someclass.getnode(id)
    x = node.some_multilink <- This triggers the SQL lookup of the multilink

And, yes, concerning your question: As soon as something accesses a
multilink inside the loop it would be materialized with an additional
query. I don't find this too worrying:
- If a class has several multilinks only the used ones are materialized,
  e.g., 'files' and 'messages' would not be queried for in your example
- The multilink is a separate table in SQL anyway. Better to do this
  lazily than materialize the data up-front with a large join.

> Also if we use filter_iter we can't sort by multilink in the sql. Can
> we detect if the user is requesting a multilink sort and fall back to
> filter()? I think sorting by multilink might be a requirement for
> index views when grouping by a multilink. (I agree with you that
> sorting by multilink is confusing.)

The internal data structure that represents the tree for the generated
sql is aware of the fact that there is sorting by multilinks. So in
theory this is possible to detect. On the other hand: I do not think
that a list sorted by multilinks is comprehensible to humans. So I guess
we should drop this feature and just ignore it during searches. Maybe
issue a warning. In addition it is probably not easy to understand why
some queries take a lot more time than others.

> Do you know if the multilink 1,9,10 has the same value as the
> multilink 9,10,1? I think it must for sorting at the db level to make
> any sense but ....

Yes, they have the same value. First the individual values are sorted
(by the order property of the multilink) then the dataset is sorted by
the resulting lists (again by order prop of course).

> Have a great week.
Same to you!

Ralf
-- 
Dr. Ralf Schlatterbeck                  Tel:   +43/2243/26465-16
Open Source Consulting                  www:   www.runtux.com
Reichergasse 131, A-3411 Weidling       email: office@runtux.com