We need to get this working in some form for 2.5.0 release.

See:

https://github.com/roundup-tracker/roundup/actions/runs/11767117283/job/32775443731

for the error.

Hmm, this appears to be broken on all CI builds. Even when gpgme is correctly installed.
I am stumped.

Debian no longer ships gpgme-config and 'pip download gpg' fails.

Do we need to move to another package for pgp/gpg support?

Maybe we can use:

  https://pypi.org/project/gpg-lite/?? It supports python 3.7+ but last update was 2022. 

  https://pypi.org/project/gpyg/ was updated in 2024 but support starts at 3.12.

I emailed the roundup-users list over the weekend to see about dropping pgp support.

Email:

Hello everybody:

I am looking at the future of support for PGP encrypted email.

Does anybody (still) use PGP encrypted email?

For those that don't know, you can sign/verify email sent to Roundup
by enrolling user's public keys. I think the intent was to validate
commands and changes sent on the subject line of an email.  It can
also encrypt outgoing PGP emails, but it has issues (issue2550943,
issue2550942).

As far as I can tell it was never well documented. After much struggle
a while ago I managed to get it to work, but I still wasn't able to
document a working process. Also sending PGP encrypted email was not
exactly straightforward.

Before Roundup 1.5, tracker auditors did not even know the source of
the transaction. tx_Source was added to support "email" and
"email-sig-openpgp". Before then, you had to use PGP for all email
from a user. PGP had to be used even if they weren't doing something
that required authentication/authorization. Otherwise you were
vulnerable to forged email.

With 1.5, your auditor can check the source of the transaction. If it
was not pgp signed, you can reject the transaction if certain
operations (e.g. status changes etc.) arrive via email.

Also articles such as:

  https://www.latacora.com/blog/2019/07/16/the-pgp-problem/

  https://soatok.blog/2024/11/15/what-to-use-instead-of-pgp/

make me reconsider PGP support. Some of the problems discussed above
don't apply to Roundup's use of PGP. But the overall impression is not
favorable.

Wgat bring this to the front is that I can't build a working PGP
toolchain in Python for our continuous integration platform. See:
https://issues.roundup-tracker.org/issue2551368. This means PGP code

won't be tested unless done locally by a developer.

My thoughts are that PGP was useful in earlier days (2000-2010) when
internet connectivity wasn't as easy, but is not so useful today.

My plan at this point is:

  * announce that PGP support will be included but not tested for
    Roundup 2.5.

  * if a tracker has pgp support enabled, 2.5 will print a warning
    when starting up.

  * remove the PGP code in release 2.6.

Does anybody currently use PGP for tracker emails? If so are you
willing to handle maintenance/documentation of this subsystem and
resolve issues?

More generally, does a method for:

  * verifying the sender and contents of email
  * encrypting sent emails

still have value? If so
https://soatok.blog/2024/11/15/what-to-use-instead-of-pgp/ lists a
couple of ideas (e.g. encrypted/signed attachment).

> I am looking at the future of support for PGP encrypted email.

I've answered there. I think there is a good future.

And I'll see if we can look into the GnuPG python module.
There is an officially supported one from the GnuPG team,
the question is just how to build it.

Hi Bern,

Any update on this? All my CI instances are by default 24.04 now.
So testing for 3.10, 3.12 and 3.13 are all missing gpg testing.

> Any update on this? 

Hi John, not yet, 
the end of the year business is very time consuming right now.
It is on my list, thought.

Hi Bern:

Have you been able to free up some time to look at this?

-- rouilj

Hi John,
not yet, but still on the list!

Hi John,

can you put the failing log file parts in here for reference?
(https://github.com/roundup-tracker/roundup/actions/runs/11767117283/job/32775443731
has meanwhile expired and you seem to have removed the code in questions for newer runs.)

> 'pip download gpg' fails

Unfortunately the pip package was never official.
The history of this is referred to in the first posts of 
   https://lists.gnupg.org/pipermail/gnupg-devel/2023-January/thread.html

Debian still ships gpgme, see
https://packages.debian.org/search?keywords=libgpgme
e.g. https://packages.debian.org/bookworm/libgpgme-dev

I'll see further how the current GnuPG python bindings
can be build on current Ubuntu and Debian systems and get back here.

Regards
Bernhard

Bern, here is the failing output:

====
Collecting gpg
  Using cached gpg-1.10.0.tar.gz (39 kB)
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'error'
  error: subprocess-exited-with-error
  
  × Getting requirements to build wheel did not run successfully.
  │ exit code: 1
  ╰─> [2 lines of output]
      <string>:163: SyntaxWarning: invalid escape sequence '\s'
      Could not find gpgme-config.  Please install the libgpgme development package.
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.
Ignoring error ubuntu-24.04: issue2551368

====

The code is still being built. The error is filtered and results in the
"Ignoring error ... issue2551368" message and suppresses the error status
that would terminate the build.

See:

https://github.com/roundup-
tracker/roundup/actions/runs/13607167708/job/38039992107#step:15:127

for the current log output.

Search any 3.12 log for:

  Collecting gpg

to see the output if the direct link doesn't work.

While debian does include gpgme, and I install gpgme-devel which include gpgme:

https://github.com/roundup-
tracker/roundup/blob/41b16fad435e6819380c21ce594adeff8c191612/.github/workflows/ci-
test.yml#L232-L244

gpgme/gpgme-devel doesn't include gpgme-config AFAICT.

Hi John,
thanks for the additional information.
My employee Paul will help resolving the situation. 
He has the roundup tests running with a gpgme build manually.

The pip gpg is unlikely to work and can probably not be fixed, but
Debian and Ubuntu do include python3-gpg, that seems to be the way to go
and should be tested next.

The reason for the failing installation is this line in setup.py:
```
line_break = re.compile(';|\\$|\\x0c|^\s*#|{')
```
`\s` is not a valid escape character and, therefore, not valid Python code. Before Python 3.12, this resulted in a deprecation warning; with Python 3.12, this is no longer allowed.

A workaround can be to build the bindings yourself using the upstream version. Here is an example of how to do this with the current CI: https://github.com/koplas/roundup/commit/6120422aa2686a23997592a58d41139fe223ec2f.

Here is the running workflow: https://github.com/koplas/roundup/actions/runs/13729858872/job/38404537982.

The upstream support for creating packages that can be built and uploaded to PyPi is currently broken (I guess this has not been tested, as there are no up-to-date gpgme bindings on PyPi).

> Unfortunately the pip package was never official.
It is official, citing the README of the upstream repository:
> The python bindings were renamed from PyME to “gpg” in 2016.

The official setup.py also uses the name gpg; if this is not official, the upstream developers should strive to take over the package name or at least replace it with a dummy package.

If only the CI needs to be fixed, it should be enough to install the already mentioned python3-gpg package. The Python bindings were recently moved to a separate repository: https://github.com/gpg/gpgme/commit/932caf37d36eca2caec59bf48bc505364a5765bb, and it may be better to contact the developers to restore the maintenance of the PyPi package.

## gpg in pypi is not official

> > Unfortunately the pip package was never official.
> It is official, citing the README of the upstream repository:
> > The python bindings were renamed from PyME to “gpg” in 2016.

"gpg" is the name of the python gpgme bindings, the official ones.

But the packaging on pypi was not official.
See https://lists.gnupg.org/pipermail/gnupg-devel/2022-November/035167.html


> The official setup.py also uses the name gpg; if this is not official, 
> the upstream developers should strive to take over the package name 
> or at least replace it with a dummy package.

As interims holder I have control of the gpg pypi entry, suggestions are welcome how to resolve
and improve the situation. But this is outside of this issue. Send suggestions to gnupg-devel 
(and read the several discussions about this in the past first).

For this case, what should we at roundup do, to roundup's use of GnuPG via gpgme functional and tested.
Is using the official packages for Debian and Ubuntu enough?
And then we need to check if the documentation recommends current packages, either via the distributions or via building it from the upstream releases. And we need to discourage the current pypi entry.

I just uploaded a prerelease version of the Python bindings to PyPi. It is now possible to install gpgme using: `pip install gpgme`. Here is the working workflow: https://github.com/koplas/roundup/actions/runs/13765181443/job/38489865763.

> As interims holder, I have control of the gpg pypi entry; suggestions are welcome on how to resolve
> and improve the situation.

I attached a source build of gpgme. It would be kind to upload this to the gpg PyPi entry. It is a beta build, so it should not disturb users running an older Python version and using this package. This would allow roundup to continue using the gpg package. If there is no interest in maintaining the PyPi entry, please mark it as archived and add a UNMAINTANED warning to the Readme. It would also be nice if the developers who maintain the Python bindings also get access to the PyPi gpg entry.

Thanks Paul and Bern for your work on this.

Bern you said: 'Debian and Ubuntu do include python3-gpg,'

I can't just install the gpg bindings using apt. CI does not use
the OS packaged python. Installing using apt only affects the OS
packaged python, not the one we are testing against. That's
why pip was used to build/install during CI. (CI would be much much
faster if I didn't have to pip install/compile the world.)

Paul thanks for explaining why '\s' is now failing with 3.12 or newer. Fixing
this is an option, but I think it is better to uses the same upstream
method used to build the vendor packages like python3-gpg.

I had analyzed the original workflow with the skipped PGP tests:

  https://github.com/koplas/roundup/actions/runs/13729858872/job/38404537982#step:19:24129

due to missing gpg.

I was not sure if this was because:

  * the install is to the wrong python directory (the one we need is under /opt)
  * the package name is different and a change to handle older and newer package
    names is needed

Also there were warnings about installing using setup.py, and I suggested using pip
to do the install to get it in the right directory etc. But I wasn't sure if using pip
to install the beta was broken (given your comments about upstream package building being broken).

I see you have fixed the above issues and added a tarball to this issue in your last update.

  https://github.com/koplas/roundup/actions/runs/13765181443/job/38489865763.

shows these tests are no longer skipped. It would be good to have the steps you took
to build that tarball in case we need to do this again in the future. I am even ok if
we can't just 'pip install gpg' and have to do a full build as long as we have
the directions for doing so.

Once you have this working, I have no issue with your checking any changes into the
roundup repo. Let Ralf S. know. He can push to our github CI instance, or help you
get set up so you can push to it yourself.

Also has anybody been able to try the PGP test using the python3-gpg package on debian/ubuntu
using the native python. All this CI is really trying to make sure that a native OS python
will work for various python versions.

Thank you for your efforts. You got much further than I did/would have.

-- rouilj

> I had analyzed the original workflow with the skipped PGP tests:

>   https://github.com/koplas/roundup/actions/runs/13729858872/job/38404537982#step:19:24129

> due to missing gpg.

I missed that. It only installs it into the OS python installation. This could be the reason why it is not found. The package name is identical, so this should not be the issue.

> It would be good to have the steps you took to build that tarball in case we need to do this again in the future.

The steps were quite hacky and can not be upstreamed:
```sh
git clone git://git.gnupg.org/gpgmepy
cd gpgmepy
./autogen.sh
mkdir build && cd build && ../configure --enable-maintainer-mode --prefix=/usr && make

# edit setup.py in build folder
# setup(
#     name="gpg",
#     cmdclass={"build": BuildExtFirstHack},
#     version="2.0.0-beta4",
#     # Change BEGIN --------------------
#     package_data={
#         "": [
#             "../gpgme.i",
#             "../helpers.c",
#             "../helpers.h",
#             "../private.h",
#             "../version.py",
#         ]
#     },
#     # Change END ---------------------
#    # Note: description appears as Summary in egg-info file.
#     description="Python bindings to the GPGME API of the GnuPG cryptography library.",

cp version.py ../version.py
python -m build
```

`build/dist` now contains the source tar that can be uploaded to PyPi. If you want to maintain your own PyPi entry, edit the name parameter in the setup function to one that is not already used.

> I am even ok if we can't just 'pip install gpg' and have to do a full build as long as we have the directions for doing so.
If we do this, we need to add documentation for building the bindings, which I do not like. This can happen to various dependencies, resulting in half the world needing to be built to set up roundup. This should be fixed upstream; until then this https://pypi.org/project/gpgme/ can be used.

> `build/dist` now contains the source tar that can be uploaded to PyPi. If
> you want to maintain your own PyPi entry, edit the name parameter in the
> setup function to one that is not already used.

Only create new pypi entries if this is a module which will be maintained
for a longer period in time. (Mybe pypi testing can be used for this, which 
needs to be confirmed.)

John,

for your information: 

> It would be kind to upload this to the gpg PyPi entry.

Paul and I will discuss how to proceed with the python module
with other GnuPG devs outside of this issue.
(The gnupg-devel@ mailinglist is the central point of the communication
for it, if you or somebody else is interested.)