Roundup Tracker - Issues

Issue 2551350

classification
Python changes for 3.12 with roundup 2.3.0
Type: crash Severity: normal
Components: Mail interface, User Interface Versions: 2.3.0
process
Status: open
:
: rouilj : kragacles, rouilj
Priority: high : patch

Created on 2024-05-14 16:08 by kragacles, last changed 2024-05-19 23:18 by rouilj.

Files
File name Uploaded Description Edit Remove
mailer.py.patch kragacles, 2024-05-14 16:08
cgitb.py.patch kragacles, 2024-05-14 16:08
Messages
msg8042 Author: [hidden] (kragacles) Date: 2024-05-14 16:08
Just updated roundup to 2.3.0 and python 3.12. Ran into a couple of crashes; one when running the mailer and the other when submitting an issue through the web UI.

The mailer appears to be a change in the starttls() method of smptlib.SMTP where the key and cert file args are no longer accepted. The attached patch 'mailer.py.patch' resolves this by creating a separate SSL context and passing it to starttls.

The submission crash looks like a change to pydoc.HMTL.heading() where the number of arguments changed. The attached cgitb.py.patch file does get it working, but as additional arguments were just dropped it is unlikely to be a proper fix. Still might be helpful.
msg8044 Author: [hidden] (rouilj) Date: 2024-05-15 01:29
Hi Andrew:

Thank you for the patches.

Can you try the following patch for cgitb.py.

This should work for 3.10 and prior which use the version with foreground
and background colors. Version 3.11 and newer use the two argument version.

====
diff -r 791b61ed11c9 roundup/cgi/cgitb.py
--- a/roundup/cgi/cgitb.py      Tue May 14 20:50:25 2024 -0400
+++ b/roundup/cgi/cgitb.py      Tue May 14 21:08:12 2024 -0400
@@ -127,10 +127,18 @@
     if type(etype) is type:
         etype = etype.__name__
     pyver = 'Python ' + sys.version.split()[0] + '<br>' + sys.executable
-    head = pydoc.html.heading(
-        _('<font size=+1><strong>%(exc_type)s</strong>: %(exc_value)s</font>')
-        % {'exc_type': etype, 'exc_value': evalue},
-        '#ffffff', '#777777', pyver)
+
+    if sys.version_info[0:2] >= (3,11):
+        head = pydoc.html.heading(
+            _('<font size=+1><strong>%(exc_type)s</strong>: '
+              '%(exc_value)s</font>')
+            % {'exc_type': etype, 'exc_value': evalue}, pyver)
+    else:
+        head = pydoc.html.heading(
+            _('<font size=+1><strong>%(exc_type)s</strong>: '
+              '%(exc_value)s</font>')
+            % {'exc_type': etype, 'exc_value': evalue},
+            '#ffffff', '#777777', pyver)

     head = head + (_('<p>A problem occurred while running a Python script. '
                      'Here is the sequence of function calls leading up to '
====

I'll followup on your mailer patch in a bit.
msg8045 Author: [hidden] (rouilj) Date: 2024-05-15 01:30
Oops forgot the changeset. changeset:   7965:6763813d9d34
corresponds to the patch I pasted in.
msg8046 Author: [hidden] (rouilj) Date: 2024-05-15 01:44
Hi Andrew:

Your patch to mailer.py includes a call to load_cert_chain(). Should that be
sslctx.load_cert_chain()?

If so can you try this patch:

====
diff -r 6763813d9d34 roundup/mailer.py
--- a/roundup/mailer.py Tue May 14 21:27:28 2024 -0400
+++ b/roundup/mailer.py Tue May 14 21:42:08 2024 -0400
@@ -6,6 +6,7 @@
 import os
 import smtplib
 import socket
+import ssl
 import sys
 import time
 import traceback
@@ -312,8 +313,15 @@
         # start the TLS if requested
         if config["MAIL_TLS"]:
             self.ehlo()
-            self.starttls(config["MAIL_TLS_KEYFILE"],
-                          config["MAIL_TLS_CERTFILE"])
+            if sys.version_info[0:2] >= (3, 12):
+                sslctx = ssl.SSLContext()
+                if config["MAIL_TLS_CERTFILE"]:
+                    sslctx.load_cert_chain(config["MAIL_TLS_CERTFILE"],
+                                           keyfile=config["MAIL_TLS_KEYFILE"])
+                    self.starttls(context=sslctx)
+            else:
+                self.starttls(config["MAIL_TLS_KEYFILE"],
+                              config["MAIL_TLS_CERTFILE"])

         # ok, now do we also need to log in?
         mailuser = config["MAIL_USERNAME"]
====

if I have the origin for load_cert_chain wrong, can you supply an updated
patch with the definition.

Thanks.

-- rouilj
msg8048 Author: [hidden] (rouilj) Date: 2024-05-15 04:13
Hi Andrew:

Can you use the patch from:

  https://github.com/roundup-
tracker/roundup/commit/2fb480eaed37b3247d61e183301b140be22423f1.patch

(corresponding to changeset:   7968:d7e79f8eb943).

I made a few changes to your patch in addition to making it conditional
on python version:

  initialize SSLContext with ssl.PROTOCOL_TLS_CLIENT
  if there is a cert file specified, enable sslctx.check_hostname
  if there is no cert file, call sslctx.load_default_certs()

Sadly there are no tests for this code in the test suite because of the need to
have an SSMTP server that we can configure as part of the tests. Issue2551351 has
been opened to record this deficiency.
msg8049 Author: [hidden] (kragacles) Date: 2024-05-15 13:44
John,

Thank you for the quick turnaround. I tested both patches and they work great! My roundup service is running smooth.

-Andrew
msg8050 Author: [hidden] (rouilj) Date: 2024-05-15 16:58
Hi Andrew:

> I tested both patches and they work great!

Just to confirm you used the github linked patch for mailer.py correct?

All these patches should be in 2.4.0 which will be going to beta in about 2-3 weeks
and then released on July 13.

Are you able to test the other TLS mail code path where the user doesn't supply
client or server certs? (Which makes me realize I wonder if we need to support
no server cert but does have a client cert...)

Thanks for using Roundup. I'm glad to hear it is running smoothly.
msg8051 Author: [hidden] (kragacles) Date: 2024-05-15 17:21
John:

> Just to confirm you used the github linked patch for mailer.py correct?

Yes, that is the one I used.

> Are you able to test the other TLS mail code path where the user doesn't supply client or server certs?

I did not try that, but I will give it a test this afternoon and see what happens.
msg8054 Author: [hidden] (rouilj) Date: 2024-05-19 23:18
Hi Andrew,

Did you have any luck testing the other code paths?

-- rouilj
History
Date User Action Args
2024-05-19 23:18:21rouiljsetmessages: + msg8054
2024-05-15 17:21:54kragaclessetmessages: + msg8051
2024-05-15 16:58:47rouiljsetmessages: + msg8050
2024-05-15 13:44:09kragaclessetmessages: + msg8049
2024-05-15 04:13:50rouiljsetstatus: new -> open
messages: + msg8048
2024-05-15 01:44:11rouiljsetmessages: + msg8046
2024-05-15 01:30:28rouiljsetmessages: + msg8045
2024-05-15 01:29:44rouiljsetpriority: high
assignee: rouilj
messages: + msg8044
nosy: + rouilj
2024-05-14 16:08:16kragaclescreate