Tested in the current devel version, but I assume it affects all
previous versions, too:

>>> from roundup import date
>>> date.Date("2014-10-31 +1m")
<Date 2014-12-01.00:00:00.000>
This should yield 2014-11-30

>>> date.Date("2014-10-31 -1m")
<Date 2014-10-01.00:00:00.000>
This should yield 2014-09-30

This affects at least the popup calender and searching for date ranges.

The culprit is the following code from roundup.date.Date addInterval():

    while month < 1 or month > 12 or day < 1 or day > get_mdays(year,month):
        # now to day under/over
        if day < 1:
            # When going backwards, decrement month, then increment days
            month -= 1
            day += get_mdays(year,month)
        elif day > get_mdays(year,month):
            # When going forwards, decrement days, then increment month
            day -= get_mdays(year,month)
            month += 1

        # possibly fix up the month so we're within range
        while month < 1 or month > 12:
            if month < 1: year -= 1; month += 12 ; day += 31
            if month > 12: year += 1; month -= 12

Reproducible by running "./run_tests.py test_dates.py" after applying the
following patch:

diff -r b76710818d31 test/test_dates.py
--- a/test/test_dates.py	Mon Oct 20 14:10:32 2014 -0400
+++ b/test/test_dates.py	Fri Oct 31 16:35:41 2014 +0100
@@ -122,6 +122,19 @@
         date = Date('2000-01-01 + 2m')
         ae(str(date), '2000-03-01.00:00:00')
 
+        date = Date('2000-01-29 + 1m')
+        ae(str(date), '2000-02-29.00:00:00')
+        date = Date('2001-01-29 + 1m')
+        ae(str(date), '2000-02-28.00:00:00')
+        date = Date('2001-01-30 + 1m')
+        ae(str(date), '2000-02-28.00:00:00')
+        date = Date('2000-10-31 + 1m')
+        ae(str(date), '2000-11-30.00:00:00')
+        date = Date('2000-10-31 - 1m')
+        ae(str(date), '2000-09-30.00:00:00')
+        date = Date('2000-03-31 - 1m')
+        ae(str(date), '2000-02-29.00:00:00')
+
         date = Date('2000-01-01') + Interval('60d')
         ae(str(date), '2000-03-01.00:00:00')
         date = Date('2001-01-01') + Interval('60d')

A similar problem occurs when adding/substracting years when the
current day does not exist in the target year, i.e. only for
February 29th:

This works as expected:
>>> date.Date('2020-02-29') + date.Interval('+4y')
<Date 2024-02-29.00:00:00.000>

Here you can see the issue:
>>> date.Date('2020-02-29') + date.Interval('+1y')
<Date 2021-03-01.00:00:00.000>
>>> date.Date('2020-02-29') + date.Interval('-1y')
<Date 2019-03-01.00:00:00.000>

This patch is from 2019, not sure if it still applies or works correctly:


fix-popup-cal-month.patch:

Calendar Popup: Fix prev/next month for shorter months

Jump to correct month if the prev/next month has fewer days (e.g. 28/29/30)
than the currently selected day (e.g. 29/30/31)

Based on a fix by Ludwig Reiter <ludwig.reiter@intevation.de>

Second patch from 2019:

fix-popup-cal-year.patch

Calendar Popup: Fix prev/next year for shorter February

Patch by Ludwig Reiter <ludwig.reiter@intevation.de>

Thanks, Thomas, for reminding us of the existence of this old issue.

Concerning the patches, as outlined in issue2551208: I don't think this should be fixed in the calendar popup code but instead in roundup.date (after adding tests that specify what results should look like).

The bug has the potential of turning up in other places where the Data + Interval logic doesn't do the right thing.

> but instead in roundup.date

I agree that this would be better.

> specify what results should look like

This is the hard part :)

It looks like the current code acts the same as gnu date:

1 month adds the number of days in month of starting date
take that date as the new starting date and add another current month's
worth of days.

so 2022-01-29 + 1 month adds 31 days -> 2022-03-01,
   2022-01-29 + 2 months adds 31 + 28 days (non-leap year) -> 2022-03-29

   2020-02-10 + 1 month (leap year) + 29 days

now when going backwards it subtracts the number of days in the previous month:

   2022-05-29 - 1 month subtracts 30 days (April) -> 2022-04-29

Similarly:

   2022-01-10 - 1 month subtracts 31 days (dec) -> 2021-12-10

So I claim roundup is doing a right thing here at least.

Thomas I think your desired test cases are handled by changing to:

  adding the number of days in the next month
or
  subtracting the days in the current month?

Is that less confusing/more desirable or more correct according to some authority?

* John Rouillard <issues@roundup-tracker.org> [20220617 01:37]:
> Thomas I think your desired test cases are handled by changing to:
> 
>   adding the number of days in the next month

No, this doesn't work in months with more days than the next, e.g.:
2022-05-01 + 1 month (30 days) = 2022-05-31

-> still the same month instead of the next

> or
>   subtracting the days in the current month?

This would work.

So either changing the UI part instead of the date calculations is
the right thing here or we have to accept that
date + 1 month - 1 month != date
and just properly document it.

And don't forget that +/- 1 year in February needs to be adjusted,
too:
  2024-02-29 + 1 year = 2025-02-28
  2025-02-28 - 1 year = 2024-02-28 (or -29?)

On Thu, Jun 16, 2022 at 11:17:54PM +0000, John Rouillard wrote:
> 
> John Rouillard added the comment:
> 
> It looks like the current code acts the same as gnu date:

John, the mechanism is mainly used for the prev/next links in the
calendar widget. So if this sometimes does not navigate to the
previous/next month and sometimes jumps two months this is a bug. No
matter what Gnu date says :-)

I think we should fix it so that it works for the documented bug cases.

Ralf
-- 
Dr. Ralf Schlatterbeck                  Tel:   +43/2243/26465-16
Open Source Consulting                  www:   www.runtux.com
Reichergasse 131, A-3411 Weidling       email: office@runtux.com

Silly question. Can we do away with the calendar widget entirely by setting
"type=date" on the input and use the browser native calendar?

If I can get this to work, it would fix:

  issue2551124 - localize calendar
  issue2550527 - calendar styles buggy (left margin)
  issue2550935 - Calendar popup does not show current value of date field.
                 Shows original value.

and partly addresses:

  issue2551260 - Replace classhelp/calendar frame system

This fix is literally 12 characters.

Anybody got any ideas/thoughts here?

Using type=date would be an improvement to the current situation,
even though we would lose copy&paste for this field, unless adding
something like
https://stackoverflow.com/questions/49981660/enable-copy-paste-on-html5-date-field

+1 from me

I attached the file datecopy.js and include it using:

      <script tal:attributes="nonce request/client/client_nonce"
              tal:content="structure python:utils.readfile('datecopy.js')">
      </script>

in the template.

datecopy.js is derived from the stackoverflow you mention except I attach the
doubleclick handler to the body element. Then use event delegation to receive
the doubleclick and check to see if the event target is an input of type
'date' or 'datetime-local'. If that is true, swap the type, select the contents,
attach a focusout handler to reset the type to it's original type.

I am not handling any touch events, so table users will not be able to copy,
but I don't think the calendar popup worked well on touch displays so this
should be a net win for them too.

I also had some failed experiments trying to capture copy/paste on the date
element. I was able to capture them at the body level, but it seems like the
target was (in most cases) the body element itself not the focused date element.

For reference and easy review here is datecopy.js:

=====
/* capture doubleclicks and if it's a date element, select the date for
   copying. Derived from
   https://stackoverflow.com/questions/49981660/enable-copy-paste-on-html5-date-field
*/

document.querySelector("body").addEventListener('dblclick', (evt) => {
    if (evt.target.tagName != "INPUT") {
        return
    }

    if (! ["date", "datetime-local"].includes(
        evt.target.attributes.type.value.toLowerCase())) {
        return
    }

    /* we have a date type input */
    target = evt.target
    original_type = target.attributes.type.value;
    target.type = "text";

    // After changing input type with JS .select() won't
    // work as usual
    // Needs timeout fn() to make it work
    setTimeout(() => {
        target.select();
    })

    // register the focusout event to reset the input back
    // to a date input field. Once it triggers the handler
    // is deleted to be added on the next doubleclick.
    // This also should end the closure of original_type.
    target.addEventListener('focusout', () => {
        target.type = original_type;
    }, {once: true});
})
=====

The fix for issue2551390 - Replace text input/calendar popup with native date input;
committed on changeset:   8285:2bf0c4e7795e. Also a newer version of datecopy.js was
committed. See msg8289 for details.

So only the underlying interval calculations in date.py need to be addressed.

Since our date.py has been here since 2001, would we be able to use python's
date routines and intervals instead? That way we just default to the standard
libraries?

* John Rouillard <issues@roundup-tracker.org> [20250119 04:43]:
> So only the underlying interval calculations in date.py need to be addressed.
> 
> Since our date.py has been here since 2001, would we be able to use python's
> date routines and intervals instead? That way we just default to the standard
> libraries?

Sounds good to me

First of all: Thanks John for implementing the html5 date, looks much nicer than the previous thing.

One wish: In roundup we always had the international date format YYYY-MM-DD and it now (at least in my test instance) has mm / dd / yyyy which is a format not understood in german-speaking countries (the traditional format there is dd.mm.yyyy) so can we have the international format back as the default? I guess this *is* configurable?

Concerning re-implementing date using python's datetime: I think this is a good idea but hard to get right so that no code changes are necessary, for example I'm using date.Date('.') a lot for the current date. And I'm not sure we have test cases for all these special effects of date.Date :-)

Hi Ralf:

> One wish: In roundup we always had the international date format YYYY-MM-DD and
> it now (at least in my test instance) has mm / dd / yyyy  ... [traditional format
> ... is dd.mm.yyyy) so can we have the international format back as the default?
> I guess this *is* configurable?

Hmm, maybe not or at least not mixable (e.g. English locale with dd.mm.yyyy date
format).

The browser is supposed to handle the date format automatically. The date
(or datetime-local) input takes a YYYY-mm-ddTH:MM[:ss] format value and
displays it according to the browser's locale (or system date setting (Edge)).

If I change my chrome browser to use German (chrome://setting/language) and make
it the default display language for chrome and restart the browser, I see
31.12.2024 00:00 for a 2024-12-31T00:00 date value. I didn't need to refresh
the page, but if you don't see that maybe a page refresh would help.

What browser are you using and what language is its interface in?

AFAIK, there is no way to configure this outside of setting the locale.
It has to be handled at the tracker level by reverting to the text input
setting type="text" explicitly on a date/datetime display as shown at:

  https://sourceforge.net/p/roundup/code/ci/default/tree/doc/upgrading.txt#l292

> Concerning re-implementing date using python's datetime: [emulating edge cases in
> current date and lack of test coverage]

That's my concern too. I asked the question to see if anybody knew of any
blockers I don't know about.

-- rouilj

* John Rouillard <issues@roundup-tracker.org> [20250123 16:27]:
> John Rouillard added the comment:
> > One wish: In roundup we always had the international date format YYYY-MM-DD and
> > it now (at least in my test instance) has mm / dd / yyyy  ... [traditional format
> > ... is dd.mm.yyyy) so can we have the international format back as the default?
> > I guess this *is* configurable?
> 
> Hmm, maybe not or at least not mixable (e.g. English locale with dd.mm.yyyy date
> format).
> 
> The browser is supposed to handle the date format automatically. The date
> (or datetime-local) input takes a YYYY-mm-ddTH:MM[:ss] format value and
> displays it according to the browser's locale (or system date setting (Edge)).

For me it shows as dd / mm / yyyy in Firefox and Chromium, but upon
further investigation I found that it reacts to my
LANG="en_IE.UTF-8" environment variable (which also sets LC_TIME)

If I set it to de_DE.UTF-8, then Firefox incorrectly shows mm / dd / yyyy,
but Chromium shows tt.mm.jjjj

I have chromium-l10n installed, but not firefox-esr-l10n-de, maybe
that causes the different display.

With en_US.UTF-8 both "correctly" show mm / dd / yyyy.

Searching for "custom date input format html5" shows some solutions
to configure the date format to yyyy-mm-dd, e.g.
https://stackoverflow.com/questions/76009630/javascript-to-to-change-input-type-date-format-and-value

But I'm not sure if this is appropriate for Roundup.

Hmm, I would like to have a solution where I can force the display to the international format even if the browser language is English. I think not using national formats for the date is a big step forward :-)
My use-case is an international company where *all* employees have an English locale. They are used to the international format.
As it currently is I cannot force this globally. Instead -- as far as I understand it now -- I would need to wade through several dozens of templates to force it everywhere? With lots of chances to miss one or two?

So maybe we can make this configurable as a global default (I'm fine when the global default is to use the browser setting) similar to the "use_browser_language" setting in section "[web]" where we can force a language instead of using the language the browser requests (for my use-case this is set to "no" which forces the "language" configured in section [tracker])

What do you think?

Thanks
Ralf

* Ralf Schlatterbeck <issues@roundup-tracker.org> [20250124 10:54]:
> What do you think?

I'm very much in favo(u)r of the yyyy-mm-dd format, but fear the
extra maintenance overhead of additional javascript.

Hi folks:

Thomas> If I set it to de_DE.UTF-8, then Firefox incorrectly shows mm / dd / yyyy

I set the language to Deutch from the firefox general settings menu and it displayed
correctly dd.mm.yyyy without a restart.

Thomas> javascript solution to showing date in normal format

That link overlays the date control with a reformatted span AFAICT. So
it would not allow editing the fields of the date. I came across something
similar in another thread that has a jsfiddle implementing it.

Also the extra maintenance burden of js is an issue.

Ralf> So maybe we can make this configurable as a global default

This can be done. It would have to display a text input for date and use the existing
broken popup calendar. I'm not sure how invasive it would be. I think the input code
has access to the config (self._client.db.config??). Maybe adding:

    if config['TRACKER_EDIT_INTERNATIONAL_DATES']:
        kwargs.setdefault("type", "text")
        if popcal is not False:
             popcal = True

    # before existing
    kwargs.setdefault("type", "date")

will work?

AFAIK there is no way to get yyyy-mm-dd formatting from the
date/datetime-local control without changing to a locale and
language that uses that format.

I'm also ok with reverting the whole date patch. I think the number/integer can still
stay though.

If this sounds good, can one of you implement it and add the config setting to the
update docs.

I'll reopen issue2551390 and reference these 4 messages. Also it sounds like we still
need to fix the calendar for Ralf's use case. Agreed?

One other nit as I was writing up the docs on that patch was the manual addition of the
javascript. Keeping the javascript out of the python code is a good thing, but it would
be good if we can either:

  inject a reference to the file (via <script>)
  insert the contents of the file (via readfile)

into the generated template exactly once. Not once for every date type field.
Adding it for every field is easy by putting the js into the string returned by 
field()/field_date().
 
I'm glad we are finding this pre-release. So thanks for your help.

Using the native date widget of the browser is now a configurable option, see config variable use_browser_date_input in section [web]. With native dates the problem is fixed but it is impossible to configure international date format server-side.

So for non-native date (when the config item above is set to 'no' which is the default) the problem is not fixed.

Note that I meanwhile think that we should really fix this in the old date widget and *not* in the date.Date and date.Interval implementations. The reason is that an interval of '1m' (one month) will always be 30 days (it is the same in pythons own implementation as was pointed out during discussions). So adding/subtracting this from the current date will often be wrong depending on the current date. Instead the prev/next links in date navigation should compute the correct date from the current date in the widget and not use '1m'.

Another option would be to integrate a 3rd-party date widget instead of fixing the ancient one we have in our code.

So I'm leaving this open for now.