Skip to content

Troubleshooting ​

Start with three places: docker compose logs pulsed (JSON logs), curl -fsS 127.0.0.1:9464/readyz on the server, and the dashboard's System, Security and Alerts pages.

Tools and clients ​

SymptomCheckFix
Authorization Violation on connectThe client's status and expiry on the Clients pageExpired or revoked credentials: rotate the client and deploy the new .creds.
Publish or subscribe times out (5 s in the SDKs, 10 s for pulse pub)The client's grantsNATS reports permission violations asynchronously, so a denied JetStream call looks like a timeout. Add the grant; the tool reconnects with it, no new .creds needed.
Permissions Violation for … _INBOX.…The tool sets its own inbox prefixUse an SDK, or --inbox-prefix _INBOX_<client-id> with the nats CLI.
Permissions Violation for … CONSUMER.CREATE …Subscribe and durable grantsThe filter must be inside a subscribe grant; durable names must match a durable grant. Library helpers that create consumers differently are refused; use the SDKs.
invalid event subjectThe subjectAt least four lowercase tokens starting with evt: see the envelope.
x509: certificate signed by unknown authority on 4222System → CertificatesUntil Let's Encrypt succeeds, NATS serves the internal bootstrap certificate: pass the internal CA (--ca, WithCAFile, ca=). ACME errors are on the System page. The extra CA is added to the system roots in every SDK and the CLI, so it can stay once the Let's Encrypt certificate is served.
Connection to 4222 or 443 hangsSecurity page: is the address banned?Lift the ban. The admin tunnel comes from loopback and is never banned, so the dashboard always works.
A durable replays from an old positionDurable start options only apply when the consumer is createdDelete the consumer on the Streams page (or use a new durable name) to start over.
Live tail shows "sampled out"200 events/s per tail sessionExpected under load; the Explorer and replay have every stored event.

HTTP and webhooks ​

SymptomCheckFix
401 from /api/v1Token prefix, expiry, revocation and networks on the HTTP tokens pageUse the full pulse_<prefix>_<secret> token; check the source network list. Repeated 401s ban the address.
403 from /api/v1The grant for that subject, or a banAdd the grant, or lift the ban on the Security page.
42920 requests/s per addressBack off for Retry-After; batch publishes (up to 100 per request).
Inbound webhook gets 401The hook's verifier and secret; the sender's signature headerPaste the provider's secret again (Stripe's whsec_…), or rotate and update the sender.
Inbound webhook gets 404Hook name and enabled flagThe URL is /hooks/<name> exactly.
Requests never reach pulsedCloudflare WAF events; origin firewallOnly Cloudflare may reach 443, and only with Authenticated Origin Pulls. Direct requests to the server's IP are dropped by design.
Outbound deliveries stuck on one eventDelivery log for that hookDelivery is in order: fix the endpoint, or wait until the event is marked dead (about two hours), or pause the hook.
Outbound delivery error "destination address is not allowed"The URL resolves to an internal addressAllowlist it with PULSE_OUTBOUND_ALLOW.

Dashboard sign-in ​

SymptomCheckFix
The sign-in page says no administrator exists—Run docker compose exec pulsed pulsed admin bootstrap (dev: make dev-admin) and open the link within 30 minutes.
"invalid credentials"Email and password; the audit log (user.login)The same message covers unknown emails, wrong passwords and disabled users. An administrator can issue a password link on the Users page.
"too many attempts; try again later"Sign-in throttlingWait (at most 15 minutes) or have an administrator clear the throttle on the user's page.
Locked out: lost TOTP and recovery codes, SSO down while password sign-in is off—Break-glass on the server: pulsed admin recover --email you@example.com (details).
SSO returns to the sign-in page with an errorThe provider's redirect URI, the issuer, a Test login on the provider pageRegister exactly <PULSE_ADMIN_URL>/auth/oidc/callback; the issuer must match the discovery document exactly. "Not provisioned" means the identity is not linked and the provider is not JIT.
The dashboard keeps asking to confirm it is youPULSE_REAUTH_WINDOW (5 min)Expected for secrets, deletes, restores and user or SSO changes; confirm and the action continues.
Signed out after a whilePULSE_SESSION_IDLE (30 min), PULSE_SESSION_MAX (12 h)Expected. Sessions are also tied to the listener: a tunnel session does not work on the public dashboard.
pulse clients … asks for a token, or 401The token, the listenerCreate a personal access token on the Account page and pass --admin-token-file or PULSE_ADMIN_TOKEN. Tokens work only through the tunnel (http://127.0.0.1:8080), not on the public dashboard.
pulse clients create gets 403The token's roleThe token needs the operator or administrator role, and you must still hold it.

Server ​

SymptomCheckFix
pulsed stays unhealthydocker compose logs pulsed; curl 127.0.0.1:9464/readyzUsually NATS is not reachable or pushing the APP account failed: check nats health and PULSE_NATS_URL.
pulsed refuses to start in productionThe error names the settingThe production profile refuses unsafe combinations: plain HTTP on the public listener, no AOP CA, a non-loopback admin listener, a reauth window over 15 minutes, an http:// dashboard URL, the removed PULSE_AUTH_MODE (refused in every profile), the test DNS provider. Fix the configuration; do not switch to the dev profile.
Dashboard says the UI is not builtA local build without the UIUse the release image, or make build (which runs make web first).
Dashboard answers 421 Misdirected RequestThe Host headerThe admin listener accepts only 127.0.0.1, localhost, ::1 and PULSE_ADMIN_HOSTS. Open it as http://127.0.0.1:8080 through the tunnel.
Alerts fire but no messages arriveAlerts → Notifiers (sent, failed, suppressed, last error)Fix the bot token, chat ID or webhook file; use Test. Rate-limited messages count as suppressed.
Backups failBackups page error; pulse backup lsS3 endpoint, credentials and bucket. The error names the failing step.
Disk filling upOverview forecast; disk_forecast and stream_usage alertsAdd disk, or accept that EVENTS discards its oldest events at max_bytes. pulsed never shrinks EVENTS: a lower PULSE_EVENTS_MAX_BYTES is ignored with a warning.
Docker networking breaks after restarting nftablesThe nftables.service overrideDebian's unit flushes the whole ruleset on stop. Install the drop-in that clears ExecStop, then restart Docker once. See Firewall and bans.
Bans have no effect on NATSbanagent logs; nft list set inet pulse_edge pulse_ban_v4banagent needs host networking with NET_ADMIN and runs as root inside its container (added capabilities do not reach the image's non-root user).

Still stuck? Open an issue on GitHub with the pulsed version output and the relevant log lines, after removing tokens, .creds content and addresses you do not want public. Report security problems privately (see the security model).

Released under the Apache-2.0 license.