Skip to content

Reduce noise in GS logs caused by gateway disconnections - #7993

Open
vlasebian wants to merge 2 commits into
v3.36from
fix/reduce-noise-in-gs-logs
Open

Reduce noise in GS logs caused by gateway disconnections#7993
vlasebian wants to merge 2 commits into
v3.36from
fix/reduce-noise-in-gs-logs

Conversation

@vlasebian

@vlasebian vlasebian commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

Summary

TLDR; Reduce log noise caused by disconnections logs in tasks and add counter to track reasons for disconnects.

Whenever I try to look for an issue in the GS logs and filter by error level or warning level, I always get these two errors everywhere. It makes it really hard to debug issues and I'd like to change how they are handled so the GS logs are less noisy.

  1. websocket: close 1006 (abnormal closure): unexpected EOF - caused by the gateway's TCP connection dying without a proper websocket close.
  2. error:pkg/gatewayserver/io/semtechws:missed_too_many_pongs (gateway missed too many pongs) - happens when the gateway stops responding to the pings of TTS. If the gateway stops responding to pings, the connection gets shut down.

These errors are closely related, both refer to disconnections.

Changes

  • Don't return an error when context is done for upstream tasks (NS and packetbroker) - this will avoid logging the error as 'Task Failed' in the task handler. Same for startDisconnectOnChangeTask. The error will be logged on connection tear down.
  • Wrap the websocket errors, don't return them directly.
  • Add unit tests.
  • Add a counter to track the number of gateway disconnects per protocol and error cause.

Testing

Unit tests.

Regressions

The gs.gateway.disconnect event for these disconnections now carries structured error details instead of a plain string.

Notes for Reviewers

None.

Checklist

  • Scope: The referenced issue is addressed, there are no unrelated changes.
  • Compatibility: The changes are backwards compatible with existing API, storage, configuration and CLI, according to the compatibility commitments in README.md for the chosen target branch.
  • Documentation: Relevant documentation is added or updated.
  • Testing: The steps/process to test this feature are clearly explained including testing for regressions.
  • Infrastructure: If infrastructural changes (e.g., new RPC, configuration) are needed, a separate issue is created in the infrastructural repositories.
  • Changelog: Significant features, behavior changes, deprecations and fixes are added to CHANGELOG.md.
  • Commits: Commit messages follow guidelines in CONTRIBUTING.md, there are no fixup commits left.

dependabot Bot and others added 2 commits August 6, 2026 15:54
Bumps the all group with 1 update: [actions/labeler](https://github.com/actions/labeler).


Updates `actions/labeler` from 6 to 7
- [Release notes](https://github.com/actions/labeler/releases)
- [Commits](actions/labeler@v6...v7)

---
updated-dependencies:
- dependency-name: actions/labeler
  dependency-version: '7'
  dependency-type: direct:production
  update-type: version-update:semver-major
  dependency-group: all
...

Signed-off-by: dependabot[bot] <support@github.com>
@vlasebian
vlasebian requested review from a team as code owners August 6, 2026 12:58
@vlasebian
vlasebian requested a review from johanstokking August 6, 2026 12:58
@github-actions github-actions Bot added the c/gateway server This is related to the Gateway Server label Aug 6, 2026
@vlasebian vlasebian changed the title Fix/reduce noise in gs logs Reduce noise in GS logs caused by gateway disconnections Aug 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

c/gateway server This is related to the Gateway Server

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant