On issue2550858, I proposed replacing a text input and calendar popup for
editing date/time fields with a native input of type date or datetime-local.

This doesn't handle every use case, but I think it handles the majority of
them and addresses all existing issues with the calendar popup.

As part of this change, the field method for DateHTMLProperty class 
changes popcal=True to popcal=False by default. It adds a new:

  field_time()

method for editing date and hours/minutes. Editing seconds only works
if the seconds value is not 0. If you need to add seconds, use a text
input (python:context.datefield.field(type="text", popcal=True)) to
allow editing for the full date format and enable popcal. (See below
for a double clicking action that helps this case.)

The value returned by a datetime-local is set to an RFC3339 format date_time.
The hyperdb Date class accepts input in the form YYYY-mm-DD.HH:MM:SS, but rfc3339
uses 'T' in place of '.'. So hyperdb Date will parse rfc3339 as well.

The field() and field_time() methods force the proper format when date or
datetime-local type is used. Trying to specify a format will result in an
exception.

I also created a javascript file on Thomas's suggestion we support copy/paste.
Including this file (see msg8268) allows copying of date/datetime-local by
double clicking on the input. This changes it to a text input that allows
copying or pasting. Note that this can also allow raw editing of the date/time
for editing/adding seconds. One thing to note is that the value shown is in
RFC3339 format not roundup full date format.

Using the native date/time should be an improvement for tablet users as well.
Note that double-click may not work on touch devices.

This change hides issues with backend the date interval calculations from the
front end, but we still need to handle the backend issues on another ticket.

Anybody want to try the patches?

See also: issue2551260, issue2550935, issue2550527, issue2551124

Hmm, I wonder if I should make field generate a datetime-local input
and have a new field_date use a date input when the user just wants
to edit a date without time.

I use field(format="%Y-%m-%d") in my trackers, so it acts like a date input.
Do any of you use the field method for input of full date and time?

* John Rouillard <issues@roundup-tracker.org> [20250115 03:06]:
> I use field(format="%Y-%m-%d") in my trackers, so it acts like a date input.
> Do any of you use the field method for input of full date and time?

Just the date should be fine here and improve the situation for
almost everyone. I've noticed our users use the time in very few
situations to indicate a hard deadline for finishing something
(e.g. this must be submitted before 14:00), but losing this isn't a
big deal.

Of course the indicators of "how much time until the deadline"
should be changed to days instead of hours as well and it shouldn't
be affected by daylight saving time (which currently sometimes
changes a time of midnight to an hour before midnight on the
previous day). I haven't checked if the patches already take care of
this.

I have been working through the changes and docs.

The biggest hangup is handling format strings for the dates.
For example:  in German, "%d-%m-%Y" is common, for the US: "%m-%d-%Y"
is common.

After this change, when using the native date input, the format must be
"%Y-%m-%d" so that the native date input can parse/display it. Currently
the code throws an exception if format is set and does not exactly match
the required format.

Similar rules apply for datetime-local inputs.

I am considering just ignoring format if the input type is a
native date/datetime-local.

Alternatvely, ignore the date if it looks compatible. For example: "%m,%d|%Y"
is compatible with a date input. I can do this with )use set to get rid of ordering
issues):

  set(re.findall('%([A-z])', format)) == set("Y", "m", "d")

to consider the format compatible with 'date' and

   set(r.findall(...)) in [set(['Y', 'm', 'd', "H", "M"]),
                           set(['Y', 'm', 'd', "H", "M", "S"])]

to consider it compatible with datetime-local. A year/month format like: "%m-%Y" would
not be compatible and would throw an exception. (Yes weird formats like:
"%Y: %m %m-%d will be incorrectly accepted but ... they deserve it 8-).)

I don't like spending cpu cycles doing the compatibility test, but...

Thoughts?

I didn't get any feedback on the users list. So I decided to keep the status
quo if no changes are done by the admin making supporting this an optional
step during an upgrade.

Here is how it works:

 1) a simple field() or context/dateprop/field returns a month/day/year 
    selector. The new field_time() method includes hours and minutes. If
    the date-time value has non-zero seconds, seconds will also be shown.

 2) if format is supplied to the methods and isn't exactly: %Y-%m-%d 
    for field() or %Y-%m-%dT%H:%M:%S for field_time(), a text input using
    the supplied format will be produced and the calendar popup link 
    will be supplied -- fall back to 2.4 and earlier.
 
    A WARNING log message will be produced. Note that the default logging
    level is ERROR, so  it will not be seen without a change to the default.
    This log is not an ERROR level issue.
 
    The admin can set type="text" in the method call to suppress the warning.

 3) A 'datecopy.js' module (and minimized 'datecopy.min.js') is added to all
    templates. It includes an event handler that activates when any native date
    input receives a double-click. On double-click it changes the input to
    text input mode and selects the date. copy: ^C, paste: ^V, undo: ^Z and
    editing can be done in text mode.

    Hitting Enter/Return or moving to a new field by mouse or TAB will set the
    value and return to a native date input. Hitting Escape will reset the value
    to the original value and return to a date input.

    When the input is in text mode the class 'mode_textdate' is added to allow
    CSS to target it.


Currently there is no keyboard method to enter text mode. Only one handler is
added to the document and event delegation is used to activate it. This reduces
the memory footprint of the page and also allows it to work if a new date field
is added by javascript after the page loads. Double clicks are rare enough that
having the event handler activated every time isn't an issue. But activating it
on every keystroke to the page to detect a hotkey to move into text edit mode
seems excessive.

Support for tables/touch devices is more mixed. The native date input works
in chrome. Single tap activates the native date picker. It also supports
double-tap for double click and text input mode is activated. But in
firefox single press or an attempt to double press (click??) brings up
the native date editor. So there is no way to get into text input mode
that I can find in ff. I can't test with safari. Even so, the native date
picker should be an improvement.

changeset:   8285:2bf0c4e7795e

Reopening in response to msg8322, msg8323, msg8324, and msg8325.

Synopsis: Need a way to allow all date inputs to use YYYY-mm-dd[T .]HH:MM:ss in
one swoop rather than editing every date item. Ralf suggested a setting similar
to "WEB_USE_BROWSER_LANGUAGE". Thomas suggested some javascript to overlay the
edit display with a static date display. You couldn't edit the individual date
fields fields, but in theory (if you sized/placed it correctly) could use the
native calendar popup. He also had concerns about maintaining and testing the JS.

When this is implemented, I am not sure if it should enable native date editing
or disable it (using current 2.4 setup) by default. Naming of the config option is
also open.

I am also open to unclosing the date picker issues (which may have to happen anyway)
and reverting this patch.

* John Rouillard <issues@roundup-tracker.org> [20250124 14:25]:
> I am also open to unclosing the date picker issues (which may have to happen anyway)
> and reverting this patch.

The native date picker is so much nicer than the old JS one that I'd
be willing to endure any usability issues with the date format :)

On Fri, Jan 24, 2025 at 02:08:39PM +0000, Thomas Arendsen Hein wrote:
> 
> Thomas Arendsen Hein added the comment:
> 
> * John Rouillard <issues@roundup-tracker.org> [20250124 14:25]:
> > I am also open to unclosing the date picker issues (which may have to happen anyway)
> > and reverting this patch.
> 
> The native date picker is so much nicer than the old JS one that I'd
> be willing to endure any usability issues with the date format :)

:-)
Yes, I think so, too

Maybe we can find a solution where the new time picker is used but
custom javascript is only inserted into the template when the time
format is being forced. This at least reduces the problem of maintaining
custom javascript for most people. Since I would be one of the users at
least a change would be detected :-)

I try to make some time for looking into this more closely...

Thanks
Ralf
-- 
Dr. Ralf Schlatterbeck                  Tel:   +43/2243/26465-16
Open Source Consulting                  www:   www.runtux.com
Reichergasse 131, A-3411 Weidling       email: office@runtux.com

Looks like gmail isn't sending this via smtp, only TLS smtp.

Anyway my response:

Hi Ralf:

On Mon, Jan 27, 2025 at 3:57 AM Ralf Schlatterbeck
<issues@roundup-tracker.org> wrote:
> On Fri, Jan 24, 2025 at 02:08:39PM +0000, Thomas Arendsen Hein wrote:
> > * John Rouillard <issues@roundup-tracker.org> [20250124 14:25]:
> > > I am also open to unclosing the date picker issues (which may have to happen anyway)
> > > and reverting this patch.
> >
> > The native date picker is so much nicer than the old JS one that I'd
> > be willing to endure any usability issues with the date format :)
> Yes, I think so, too
>
> Maybe we can find a solution where the new time picker is used but
> custom javascript is only inserted into the template when the time
> format is being forced.

That would be custom javascript to overlay a custom formatted date display
correct?

> This at least reduces the problem of maintaining custom javascript for
> most people.

The existing doubleclick handler that turns the date input into a text
input (with YYYY-mm-ddTHH:MM[:SS] doesn't have much JS. Also it is
needed to add seconds. If the time has 0 seconds, the date input won't
display them for editing. All in all it is a very simple bit of JS
that shouldn't need maintenance.

I assume you want to try creating a JS module to display a tracker
localized date format, that's much more complex to get right. Browser
date inputs look different, have different sizes
(which might not be controlled by the CSS width property).

> Since I would be one of the users at least a change would be detected :-)

True. We don't have a good frontend testing framework. The grad
students who produced the new classhelper also developed a front
end test with selenium. However it developed timing issues in some
test cases and wouldn't run to completion reliably. Of course it
failed during the final class demo despite working fine in all
the prior demos 8-(.

> I try to make some time for looking into this more closely...

If you want to discuss:

  > javascript to overlay the edit display with a static date display

a new issue linked here would be good. The existing doubleclick
handler might be useful to look at as a model.

(added on resubmit) The doubleclick handler applies one handler to
the page saving memory. So adding a new doubleclick handler for your
date overlay span should work fine. However to create the overlay,
you would have to run when the DOM is complete and replace/overlay
all the date/datetime-local elements on the page with the formatted
display.

Any newly added elements (e.g. due to javascript) IIUC would
have to be processed as part of a mutation observer. I have no clue
how that works.

I've looked more closely into this:
It seems to be a frequently asked question how to change the displayed date format server-side, e.g.
https://stackoverflow.com/questions/76009630/javascript-to-to-change-input-type-date-format-and-value
https://stackoverflow.com/questions/7372038/is-there-any-way-to-change-input-type-date-format

It seems the consensus is that it cannot be changed server-side (except, of course creating your own widget).
Note that the format that comes back from the date widget seems to be the international format (at least by default) but what the user sees depends *only* on the locale settings in the browser(s).

In Linux we can set the LC_TIME environment variable to force the time format this page:
https://ccollins.wordpress.com/2009/01/06/how-to-change-date-formats-on-ubuntu/
shows how one can define a custom format.

Unfortunately the environment setting seems to be ignored at least by firefox in some versions.

On Windows there is a configuration in the OS where one can configure the time format but it is also by default ignored by some browsers (firefox).

This leads me to believe that the W3C has b0rked the date format and that it is too early to use it in production. Note that I have not seen many web applications using the html native date format. I would have noticed the broken format (I have an us-english locale by default).

So please lets at least make this configurable to use the old widget. It is old-fashioned but at least it does what it should (except for the month-increment bug)

On that note: I have some templates that try to show the full format (including time with seconds) but when I upgrade I'm only seeing a date. The field() method in the html templates by default shows the full date + time, this has changed to only use a date widget.

I noticed this in a simple logging application I wrote for keeping my QSOs as a ham-radio operator (a qso is a two-way radio communication with certain mandatory info transmitted). This shows the full date+time (with seconds) with the old date picker but only the date with the new one. This means by default we break some applications with the new date picker.

The definitions in my qso app look like this:
...Class...
    ... qso_start = Date(), qso_end = Date() 
(so there are no special options)
and in the templates:
<td tal:content="structure python:context.qso_start.field()"/>
(also without any special options)

So my vote would be to make the use of the date picker configurable and make the old format the default -- the new date picker is much nicer but non-functional for many applications. And it does not seem to be widely used due to these problems.

Update: We have a 'format' parameter in the html 'field' method for dates. I do not think we can support this with the new html5 date picker?

Ralf said:

> So my vote would be to make the use of the date picker configurable
> and make the old format the default

I am fine with that. Can you please make the changes.
A setting under the [web] key sounds reasonable to disable
it tracker wide. Probably leave it disabled by default.

If you can please add a setting for the native number type as well. I think
it works better than the date types, but for the same reason a user may want
to turn it off.

Please also update upgrading.txt. I think most of the text there can
stay. Add info about it being disabled by default at the top and how to
enable it and replace any words about it being enabled by default.

Also change user_guide.txt. Move the part about using a native date element
to a subsection with a new header.

Ralf the issue you noted with the native time input is why the doubleclick
hander was added. It allows the user to fall back to the text input with
international date. I agree however that the usability issues when using
date heavy workflows makes it reasonable to default to off.

Also Ralf, setting format to anything except international format should
result in the normal text input.

Ok, the native date format is now configurable, see config variable use_browser_date_input in section [web]. The default is 'no'.

There are still some issues with the native date / datetime-local <input> fields in browsers: It is not possible to specify international date format (at least not easily and not server-side). Moreover some browsers leave out seconds when the time given has the seconds part =0 (yukk). This is problematic for use-cases where full seconds are needed even in the case where the seconds part is zero. An application where this matters is my QSO logging program (a QSO is a two-way radio communication between hamradio operators): Some newer digital modes have QSOs ending in 0, 15, 30, or 45 seconds. This means 1/4 of all logged QSO times would be displayed without seconds.

These are the reasons why the configuration default is corrently 'no' for the config variable described above.

I'm closing this issue as fixed.