Remote server tls authentication by asher-pem-arm · Pull Request #1885 · labgrid-project/labgrid

asher-pem-arm · 2026-06-03T17:02:49Z

This MR adds support for gRPC SSL/TLS server authentication to labgrid-coordinator, labgrid-exporter and labgrid-client.

Enabling a secure channel on all three components is done by adding the --secure argument. Paths to the certificate and key (labgrid-coordinator-only) can be specified with the --cert and --key arguments.

Testing has been added to verify that, given a secure-enabled labgrid-coordinator, both labgrid-client and labgrid-exporter can connect successfully and perform an operation.

Note on use of `labgrid-client` and `labgrid-exporter` without an explicit `--cert` specified

On Linux, this currently reads the Debian/Ubuntu CA bundle at:
/etc/ssl/certs/ca-certificates.crt

On macOS, this reads certificates from the system Keychain using:
security find-certificate -a -p

If the coordinator certificate is not trusted by the host, or the platform's system roots cannot be loaded, pass the coordinator certificate explicitly with --cert.

Closes: #1541

codecov · 2026-06-03T18:35:14Z

Codecov Report

❌ Patch coverage is 69.69697% with 30 lines in your changes missing coverage. Please review.
✅ Project coverage is 61.1%. Comparing base (7121ff3) to head (9f39d0f).
✅ All tests successful. No failed tests found.

Files with missing lines	Patch %	Lines
labgrid/remote/common.py	34.2%	25 Missing ⚠️
labgrid/remote/client.py	84.2%	3 Missing ⚠️
labgrid/remote/coordinator.py	94.7%	1 Missing ⚠️
labgrid/remote/exporter.py	85.7%	1 Missing ⚠️

Additional details and impacted files

@@          Coverage Diff           @@
##           master   #1885   +/-   ##
======================================
  Coverage    61.0%   61.1%           
======================================
  Files         182     182           
  Lines       14906   14996   +90     
======================================
+ Hits         9105    9164   +59     
- Misses       5801    5832   +31

Flag	Coverage Δ
3.10	`61.0% <69.6%> (+<0.1%)`	⬆️
3.11	`61.0% <69.6%> (+<0.1%)`	⬆️
3.12	`61.0% <69.6%> (+<0.1%)`	⬆️
3.13	`61.0% <69.6%> (+<0.1%)`	⬆️
3.14	`61.0% <69.6%> (+<0.1%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

Emantor

Otherwise looks good

Emantor · 2026-06-09T09:05:44Z

+
+.. code-block:: bash
+
+    $ labgrid-client --secure [--cert PATH] places


Feels very much like bikeshedding to write this, but should we rename --secure to --tls instead? This makes it clear that we are using TLS for gRPC secure channels instead of a generic --secure which does not carry much meaning.

gastmaier

Looks good! Some minor comments

gastmaier · 2026-06-09T11:35:59Z

+Refer to the ``labgrid-coordinator`` man page for details.
+
+When you are connecting with ``labgrid-client`` or ``labgrid-exporter`` to a
+``labgrid-coordinator``that has secure gRPC channels enabled you need to pass


Suggested change

``labgrid-coordinator``that has secure gRPC channels enabled you need to pass

``labgrid-coordinator`` that has secure gRPC channels enabled you need to pass

gastmaier · 2026-06-09T11:46:06Z

+--secure
+    enable TLS gRPC channel
+--cert
+    path to TLS certificate (in PEM format)
+--key
+    path to TLS key (in PEM format)


On the coordinator side, the tls at the reverse proxy is also an option, right?
At servers, I would rather have nginx handling the certificates.

If so, could you add an alternative, such as

Or use a reverse proxy to add TLS, for example with ``nginx``: .. code-block:: nginx server { listen 20407 ssl http2; server_name labgrid.example.com; ssl_certificate /etc/ssl/labgrid-coordinator.crt; ssl_certificate_key /etc/ssl/labgrid-coordinator.key; location / { grpc_pass grpc://127.0.0.1:20408; } }

https://blog.nginx.org/blog/nginx-1-13-10-grpc

gastmaier · 2026-06-09T12:37:34Z

+the ``--secure`` (and ``--cert`` if the certificate is not trusted by the host
+machine) option.
+Refer to the ``labgrid-client`` and ``labgrid-exporter`` man pages for details.
+


explain roots.pem and GRPC_DEFAULT_SSL_ROOTS_FILE_PATH precedence here, please

asher-pem-arm · 2026-06-10T09:34:00Z

@Emantor I have added 3 commits for the comments in the PR, if you've happy with these I'll squash all 3 into the first commit to keep the PR tidy

gastmaier

Sounds good!

Users have the option to pass explicitly or read gprc doc for the env alternative GRPC_DEFAULT_SSL_ROOTS_FILE_PATH.

Emantor · 2026-06-10T12:25:34Z

@Emantor I have added 3 commits for the comments in the PR, if you've happy with these I'll squash all 3 into the first commit to keep the PR tidy

Looks good, please squash.

jluebbe · 2026-07-08T11:56:06Z

+
+Instead of enabling TLS in ``labgrid-coordinator`` directly, a reverse proxy can
+terminate TLS and forward cleartext gRPC to the coordinator. For example, with
+``nginx``:


In which cases would a reverse proxy preferred over having the coordinator do TLS itself?

I think that depends on the deployment environment. I would expect the default option to be the direct coordinator TLS but some orgs might standardise TLS at the ingress layer, even then they might want to use the TLS enabled coordinator on top of that

jluebbe · 2026-07-08T12:27:33Z

+the ``--tls`` option. If ``--cert`` is not set, labgrid uses the host CA
+certificates to verify the coordinator certificate. Use ``--cert`` to provide
+a specific CA certificate instead.


For the client side, the usual approach is to use --cacert (like curl) to set the trusted CA certificates to check against. This leaves --cert free to allow future extension to client certificates.

asher-pem-arm · 2026-07-16T13:11:15Z

updated with a new commit for supporting external clients with the client stream, will squash the latest 2 when you're both happy with the content

Emantor

IMO this looks good, two things:

The first commit should have a proper commit message instead of the commit message for the work done in the second commit.
The second commit should have the commit message of the first.

Also please squash, the kw_only=True commit for compatibility makes sense to me.

jluebbe

With the changes requested by @Emantor and @Bastian-Krause fid, I'd be fine with this.

Allow the coordinator to serve gRPC over TLS, and allow clients, exporters, and RemotePlace connections to authenticate the coordinator. Add command-line and configuration options along with documentation and tests for secure channels. Signed-off-by: Asher Pemberton <asher.pemberton@arm.com> Reviewed-by: Asher Pemberton <asher.pemberton@arm.com> # gatekeeper Co-authored-by: Luke Beardsmore <luke.beardsmore2@arm.com>

The Linux root certificate handling is intentionally done in labgrid instead of relying on grpcio's Python default. In gRPC's ComputePemRootCerts() precedence, GRPC_DEFAULT_SSL_ROOTS_FILE_PATH is considered first. The Python grpcio package then installs an ssl_roots_override_cb from grpc/_cython/cygrpc.pyx which loads the bundled grpc/_cython/_credentials/roots.pem. Because that callback succeeds before the later LoadSystemRootCerts() path is used, a normal pip-installed grpcio package can ignore CA certificates installed in the host trust store. Therefore labgrid's TLS certificate precedence is: 1. explicit certificate argument 2. system trust store loaded by labgrid 3. grpcio default roots, including bundled roots.pem, as fallback Signed-off-by: Asher Pemberton <asher.pemberton@arm.com> Reviewed-by: Asher Pemberton <asher.pemberton@arm.com> # gatekeeper Co-authored-by: Luke Beardsmore <luke.beardsmore2@arm.com>

Emantor requested review from Bastian-Krause, Emantor and jluebbe and removed request for Emantor and jluebbe June 3, 2026 18:29

Emantor previously approved these changes Jun 9, 2026

View reviewed changes

gastmaier reviewed Jun 9, 2026

View reviewed changes

asher-pem-arm dismissed Emantor’s stale review via c6d1da1 June 10, 2026 09:32

gastmaier approved these changes Jun 10, 2026

View reviewed changes

Comment thread labgrid/remote/common.py

Emantor requested a review from jluebbe June 10, 2026 12:25

asher-pem-arm force-pushed the remote-server-tls-authentication branch from c6d1da1 to 33ae157 Compare June 30, 2026 12:44

Bastian-Krause added enhancement architectural change coordinator labels Jul 7, 2026

Bastian-Krause mentioned this pull request Jul 7, 2026

gRPC authentication proposal - using gRPC secure channels #1541

Open

7 tasks

jluebbe reviewed Jul 8, 2026

View reviewed changes

Emantor requested changes Jul 8, 2026

View reviewed changes

Comment thread labgrid/resource/remote.py

asher-pem-arm force-pushed the remote-server-tls-authentication branch 2 times, most recently from d814bd4 to c925307 Compare July 8, 2026 14:41

asher-pem-arm requested a review from Emantor July 16, 2026 13:10

Emantor requested changes Jul 17, 2026

View reviewed changes

Emantor requested a review from jluebbe July 17, 2026 08:07

asher-pem-arm force-pushed the remote-server-tls-authentication branch from b47b9f4 to 2d97b99 Compare July 17, 2026 08:22

asher-pem-arm requested a review from Emantor July 17, 2026 08:22

Emantor approved these changes Jul 17, 2026

View reviewed changes

Emantor requested changes Aug 7, 2026

View reviewed changes

Comment thread labgrid/remote/common.py Outdated

Comment thread labgrid/resource/remote.py Outdated

Bastian-Krause requested changes Aug 7, 2026

View reviewed changes

Comment thread labgrid/remote/client.py

jluebbe reviewed Aug 7, 2026

View reviewed changes

asher-pem-arm and others added 2 commits August 13, 2026 16:43

asher-pem-arm force-pushed the remote-server-tls-authentication branch from 2d97b99 to 9f39d0f Compare August 14, 2026 08:07

asher-pem-arm requested review from Bastian-Krause and Emantor August 14, 2026 08:10

Emantor approved these changes Aug 14, 2026

View reviewed changes


		.. code-block:: bash

		$ labgrid-client --secure [--cert PATH] places

	``labgrid-coordinator``that has secure gRPC channels enabled you need to pass
	``labgrid-coordinator`` that has secure gRPC channels enabled you need to pass

Uh oh!

Uh oh!

Conversation

asher-pem-arm commented Jun 3, 2026 • edited by Bastian-Krause Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Note on use of labgrid-client and labgrid-exporter without an explicit --cert specified

Uh oh!

codecov Bot commented Jun 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Emantor left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

gastmaier left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

gastmaier Jun 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

asher-pem-arm commented Jun 10, 2026

Uh oh!

gastmaier left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Emantor commented Jun 10, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

asher-pem-arm commented Jul 16, 2026

Uh oh!

Emantor left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

jluebbe left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

asher-pem-arm commented Jun 3, 2026 •

edited by Bastian-Krause

Loading

Note on use of `labgrid-client` and `labgrid-exporter` without an explicit `--cert` specified

codecov Bot commented Jun 3, 2026 •

edited

Loading

gastmaier Jun 9, 2026 •

edited

Loading