Panic in scheduleSweepBatchOutput when nbxplorer loses DB connection

[arkanaai]

Summary

arkd panics with a nil pointer dereference in scheduleSweepBatchOutput when nbxplorer temporarily loses access to its database (e.g. during DB maintenance/migration).

Panic trace

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x8 pc=0x9352de]

goroutine 3973311 [running]:
github.com/arkade-os/arkd/internal/core/application.(*service).scheduleSweepBatchOutput(0x182fad9fa788, 0x182fb016d600)
        /app/internal/core/application/service.go:3480 +0x17e
created by github.com/arkade-os/arkd/internal/core/application.(*service).registerEvent

Root cause

scheduleSweepBatchOutput is a background goroutine that interacts with nbxplorer. When nbxplorer loses its DB connection (e.g. during a maintenance window or rolling restart), it returns an error or nil response that scheduleSweepBatchOutput does not guard against, resulting in a nil pointer dereference and a full arkd crash.

Impact

• arkd process exits and must restart (VTXO restore watch required, ~14 min downtime observed on 2026-04-22)
• Any in-progress rounds are aborted

Expected behaviour

scheduleSweepBatchOutput (and background goroutines in general) should recover from transient nbxplorer errors gracefully — log the error, skip or retry the sweep, and continue running rather than crashing the entire process.

Fix direction

• Add nil checks / error guards before dereferencing return values from nbxplorer calls in service.go:3480
• Consider a recover() in the goroutine spawned by registerEvent as a safety net
• Optionally: retry logic with backoff for transient nbxplorer unavailability

Context

Discovered during DB maintenance on 2026-04-22. nbxplorer's DB migration caused a brief connection blip; arkd's auto-unlocker triggered a wallet lock/unlock round-trip which hit this code path.

[arkanaai]

Triage — arkanaai

Classification: Bug — nil pointer dereference / crash
Labels: bug, crash, nbxplorer

---

Root cause confirmed

scheduleSweepBatchOutput calls waitForConfirmation at service.go:3468, which returns (blockTimestamp, err). When nbxplorer loses its DB connection, waitForConfirmation returns an error — but the code only logs the warning and falls through. The blockTimestamp value is a zero/nil struct at this point, so the unconditional dereference at service.go:3480:

expirationTimestamp = blockTimestamp.Time + s.batchExpiry.Seconds()

panics with invalid memory address or nil pointer dereference.

The same issue affects the BlockHeight branch at line 3478 (blockTimestamp.Height).

---

Fix direction

Minimal (unblocking): Guard the dereference — fall back to time.Now().Unix() / current block height when err != nil. This is what PR #1034 proposes for the time branch. This allows the sweep to be scheduled (possibly slightly early, which is safe per BIP68 retry logic) rather than crashing.

Robust: Wrap the goroutine body with a recover() so any future panics in this path don't crash the entire arkd process. Related: consider retry-with-backoff for transient nbxplorer unavailability before giving up on the timestamp.

Related fix: See #1034 (fix: waitForConfirmation fallback to time.Now() in case of error) — a contributor has already opened a PR with the time-branch fix. That PR should be reviewed and merged promptly. The BlockHeight branch at line 3478 should receive the same treatment.

---

Duplicates

Issues #1032, #1033, and #1035 describe the same panic in the same goroutine. Treating this issue as canonical.

---

Flagged for human review — active crash that terminates in-progress rounds and requires VTXO restore watch on restart.

[Kukks]

Reproduced on v0.9.3 from a different upstream trigger

Same crash site (service.go:3480, identical hex PC 0x9352de), but the upstream that goes nil isn't nbxplorer this time — it's the ark-wallet sidecar's gRPC stream during graceful shutdown.

Trigger

docker restart ark (SIGTERM → 10s grace → SIGKILL) in the arkade-regtest setup with the v0.9 split topology (arkd + arkd-wallet as separate containers).

Log immediately preceding the panic

time="2026-05-28T19:48:30Z" level=error
  msg="failed to get wallet ready update stream"
  error="rpc error: code = Canceled desc = grpc: the client connection is closing"

time="2026-05-28T19:48:30Z" level=warning
  msg="failed to wait for confirmation of commitment tx f4da748d8b7d7e614741dcb272276fe020e2dc8bf3f9891fb045b0797e0a51af,
       schedule task time may be inaccurate"
  error="rpc error: code = Canceled desc = grpc: the client connection is closing"

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x8 pc=0x9352de]

goroutine 32866 [running]:
github.com/arkade-os/arkd/internal/core/application.(*service).scheduleSweepBatchOutput(0x3d0fa2bfe008, 0x3d0fa5a25700)
	/app/internal/core/application/service.go:3480 +0x17e
created by github.com/arkade-os/arkd/internal/core/application.(*service).registerEventHandlers.func1 in goroutine 41
	/app/internal/core/application/service.go:345 +0x64d

Why this matters for the fix

scheduleSweepBatchOutput's call site at service.go:3480 is reachable from at least two different upstreams that can return a Canceled / nil result, not just nbxplorer:

1. nbxplorer losing its DB connection (original report).
2. ark-wallet's gRPC stream closing while arkd is mid-shutdown — the warning "failed to wait for confirmation of commitment tx … schedule task time may be inaccurate" is waitForConfirmation logging the Canceled error, then scheduleSweepBatchOutput proceeds with the nil/zero return value and dereferences something at offset +0x8 (likely the block timestamp pointer).

A defensive nil-check + recover() in the goroutine spawned by registerEventHandlers.func1 (the created by line above) would protect both call paths in one go.

Repro context

• arkd image: ghcr.io/arkade-os/arkd:v0.9.3
• arkd-wallet image: ghcr.io/arkade-os/arkd-wallet:v0.9.3
• Discovered while trying to use docker restart ark as a test-harness reset between SDK E2E test cases (feat(recovery): unified wallet recovery + legacy deprecated-signer discovery dotnet-sdk#104). We abandoned that approach because of this panic — once landed, it would unlock cleaner test isolation for downstream consumers.