Just a note here:
- I'm considering enumerations of numbers (1,2,3) to be enumerations of
  degenerate ranges (with a single point). This makes the curly-brace
  syntax obsolete. We can specify it as
  [1] or [2] or [3]
  I agree that this is not a very nice syntax.
- To make the syntax easier we might want to find a way to combine
  intervals into sets *without* requiring brackets/parentheses. 
  - Space won't work as date expressions already can contain spaces
  - '+' won't work as it is also contained in date expressions (and
    might be used as a sign in numbers)
  - 'or' is fine, it doesn't collide with anything
- So maybe we *require* an 'or', then braces can be left out in the
  general case? Or find another unused symbol instead of ',' or space?

On Mon, Jul 13, 2009 at 05:57:46PM +0000, Tobias wrote:
> - the comma currently works as an 'or' operator, which I admit
>   I wasn't aware of (I didn't use it, since whitespace works as well)
>   and should continue to work.

This I don't understand, maybe it is a misunderstanding. As far as I
know, date ranges currently accept only a single range, not multiple
ranges. Neither separated by comma nor by space.

Or do you mean in integer fields ?

I currently don't have a schema containing an integer to test with.
I don't think you can search multiple integers currently.

> Then something where you are IMO wrong:
> 
> - The whitespace syntax is *not* new; it is supported e.g. when seeking
>   issues by id.  Thus, it *must* continue to work.

Ah, yes, searching ids works with whitespace.
Note that this is different from integers. IDs are *strings* in roundup.
That's why you can combine them with space!

It doesn't work when searching by name, e.g., if I'm searching for two
creators with their usernames, I can only use a comma, no space (and no
optional whitespace after the comma). No matter if I'm searching
numerically or by username.
It looks like form-processing is doing some additional magic here, a
space becomes a '+' in the request URL in my test.

> - *Requiring* the comma (or 'or') as an enumeration operator thus
>   would *break* compatibility (yes, it is supported; but it is
>   currently not *required*).

I haven't thought about using a space for separation of the ranges. Now
that I think of it, it may be a good for numbers. But what does the
form-processing create from a space? And as you point out later, date
expressions like '. -1d' can contain spaces. That rules out the space as
a separator.

> I disagree about the decimal comma cause. You and me, we are IT experts
> who are used to decimal points; but many people who are supposed to
> simply *use* Roundup are not.  One major strength of Roundup is its huge
> customizability which makes it suitable for non-geek usage (for people
> who won't use Bugzilla because it is -- or looks -- too complicated).
> Roundup should treat them well.

Fine with me. But the comma should *always* be accepted and should not
change with the browser language (otherwise changing browser language
settings will invalidate stored queries).

> Of course it would be possible to recognise floats in both flavours,
> with decimal points or decimal comma, regardless of the locale setting. 

See above, otherwise stored queries will be a problem.

> It just occurred to me: we *have* a difference already between search
> syntax for date and integers, which is derived from the value syntax;
> and we know it doesn't make very much sense to support enumerations for
> floats, because of the nature of the data.  Thus we could simply drop
> enumeration support for floats (only open intervals and 'or'), and we
> could support decimal commas right away.

See my examples, I've always seen enumeration of values a special case
of single-point intervals, i.e., 1,2,3 is [1],[2],[3]
So you can have enumeration of numbers when seeing them as single-point
intervals. So even if it doesn't make sense to enumerate floats (in most
cases there are exceptions if you know what you are doing) i wouldn't
want to restrict the syntax to integers.

No, many queries will use integer numbers.

See above for enumeration syntax. Don't make enumeration of numbers a
special case. Use enumeration of intervals only.

> I strictly disagree about whether curly brackets would make the syntax
> harder to understand; and I disagree about the "problem" of having both
> ranges and enumerations in the same expression.

So far I've not understood what curly braces are for.

> I consider it a quite common need to specify both a range and a
> enumeration.  It is easy, it is implemented, and it should be possible.

Ah, curly braces for enumerations?

> What is difficult about  "[3; 7] or {2 3 5}"?  The most difficult bit is
> the difference between open and closed interval edges.  The curly
> brackets are a possibility to explicitly state you want an enumeration.
> This is very "pythonic": explicit is better than implicit ;-)

In the example above 3 and 5 are already in the interval, so lets pick a
diffent example, curly braces
[3; 7] or {9 10 11} would be equivalent to
[3;7] or [9] or [10] or [11]
?
See above, maybe we can invent something shorter for the 'or' that works
for numbers and for dates so that we can leave out the brackets. It
can't be a space (as dates may contain spaces) nor a comma (as numbers
may contain commas).

> There is a problem about the range "(3; 5)" and an enumeration "(3, 5)":
> The range doesn't contain the edges (thus, for integers it would yield
> 4), while the enumeration *does* contain the edges.  This is
> inconsistent.  The better solution would be to enforce the usage of
> curly brackets (instead of others; see below) for enums, which make
> clear that an enumeration is a completely different beast.

I'm not seeing enumerations at all. We only have enumerations of
intervals. I had proposed the comma for this operation but that requires
brackets in all cases. Maybe we allow only the 'or' or come up with
something else not contained in numbers and dates.

So lets see a number as a degenerated interval with only one point. Once
you take that step you don't need curly braces and everything becomes
easier :-)

You don't ever want to match the empty set?
And for all other cases I'd consider it a degenerated interval:
(1;)
and a set is 
1 + 2

Maybe we want to drop the comma. Then we can leave off the brackets in
most cases.

Doesn't work for dates. And has never worked for integers. So please
lets drop whitespace for enumerations.

Fine

We don't have sets, just degenerate intervals [a], [b]
I don't see why we need a special syntax.
If we get rid of comma and space we don't need the brackets. Then we can
write:
a or b

Yes, but see above, we probably should leave out the
brackets/parenthesis in the general case. 

not needed.

Fine.

fine

fine.

We don't need set braces I think.

the latter case would mean
[a;b] or [c] if we see a simple number as a degenerate interval.

I don't think we need expressions in the generic sense. No multiple
parentheses. Just a set of intervals.

> - when an error occurs, a nice message (including the position)
>   explains it.
> 
> Now the compatibility topic.
> 
> We need a new syntax anyway; for numeric fields, the new syntax will be
> the *only* possibility to specify ranges.  To stay compabible, it *must*
> support whitespace enumerations for numbers.

Why would you need a new syntax for specifying ranges numbers????
There is no whitespace enumeration for numbers. Forget about IDs: They
are strings.

I don't think whitespace currently works for anything other than
strings.

Compatibility: When we treat a single number as a degenerate interval
we're backward compatible with numbers.

OK. Maybe we can find something different that doesn't break existing
syntax. Then we could leave out the braces in the general case
(especially for degenerate intervals or number-enumerations).

can you give a test-case/example of an expression with whitespace that
currently works? With integers?

OK.
So lets drop whitespace for numbers too. It has never worked.
So there are no stored queries with that syntax.

> For both cases (and for floats as well, of course),
> there are two possibilities they have in common:
> - the 'or' operator
> - inside curly brackets, the semicolon divides values, but changes
>   nothing about the nature of the enumeration-implying curly bracket.

Once you're using the semicolon for ranges I don't think we have a
compatibility problem. If a single number is treated as a degenerate
interval.

> I can't see a problem with my proposal of a configuration switch
> (msg3765).  The current functionality for the old date search syntax is
> there, and so are hopefully the unittests.  There will be unittests for
> the new syntax as well.  What else do you need?!

One syntax. Which passes the old unittests. Otherwise users will have to
change *all* their stored queries.
Roundup is deployed for large installations.
- Think about the python.org tracker
- Think about a customer of mine with >200 users and *lots* of stored
  queries.

But I think we can achieve that when using semicolon for ranges.

it's '. -1d' for yesterday at the same time as 'now'.

> 0:00, this is fine; nobody wants to change this.  The 2nd edge seems to
> be seldom used anyway; however, we should be as consistent between data
> types as possible, and thus a specification of "yesterday;today" should
> find *all* of yesterday and *all* of today (some date values can lie in
> the future).

There is no "yesterday" in roundup. And '.' is the current date/time
including seconds.
If you're referring to example date lets use a concrete example:
 2009-06-01;2009-06-02
includes all times after 2009-06-01 midnight (including midnight) up to
and *not* including 2009-06-02 midnight.
If we change that to an *inclusive range* we have only a problem with
midnight. I *think* we can live with that but lets ask Richard to
comment on this once we've agreed on a spec.

The alternative would be to make *all* ranges without explicit
parentheses/brackets compatible to the current date semantics, so that 

1;2
is the same as
[1;2)

I think so, too. But lets ask Richard. The proposal above (ranges
include the start but exclude the end of the interval if no explicit
parentheses/brackets are given) is also possible.

> Let's take the opportunity to do it right.  I'll implement it -- maybe
> with a little help with the dates --, including the unit tests, and
> everyone will like it; I promise ;-)

Fine. But lets start with a specification not an implementation.

I also want to do it right ...
Thats why I'm discussing it with you -- to get to a point where we can
present a proposal ...

Ralf
-- 
Dr. Ralf Schlatterbeck                  Tel:   +43/2243/26465-16
Open Source Consulting                  Fax:   +43/2243/26465-23
Reichergasse 131                        www:   http://www.runtux.com
A-3411 Weidling                         email: office@runtux.com
osAlliance member                       email: rsc@osalliance.com