etminan
User guide

Everything an operator needs, start to finish.

Installing all three binaries, configuring a monitored host and a verifier device, setting up mTLS and enrollment, every CLI command the agent and verifier ship with an explanation and a runnable example, day-to-day review, extending the plugin points, and what to do when something looks wrong. For the trust model and design rationale behind any of this, see Architecture & design.

01 — Who runs what

A single host is never both.

The verifier's entire trust value comes from being administratively separate from the hosts it checks — never share a machine between the two roles.

Monitored host

etminan-agent

A relay, not a trust anchor. Needs a TPM 2.0 device (physical or vTPM) and the kernel's IMA subsystem enabled.

  • Sweeps watched paths, takes a signed TPM quote, relays it with the IMA log delta.
  • Makes no pass/fail decision and sends no alerts — that's the verifier's job.
Separate device

etminan-verifier

No TPM needed. Polls every enrolled agent on a schedule and is the only place an alarm ever fires from.

  • Offline signature check + PCR replay against the approved baseline.
  • Zero TPM library dependency in the shipped binary — cross-compiles clean to a static musl target.
Third machine

etminan-witness

Checks the verifier's own signed evidence where the verifier is not: a signed integrity summary, or an exported audit bundle. No daemon, no database, no key, no TPM. It is a separate package for that reason — installing the verifier somewhere just to read one line of JSON would defeat the point.

02 — Installing

From a fresh host to three running binaries.

From the .deb package

$ sudo dpkg -i etminan_<version>_amd64.deb

The postinst step creates the etminan-agent (group tss) and etminan-verifier system users, /var/lib/etminan-{agent,verifier}, and /etc/etminan-{agent,verifier}/ — but enables nothing. Pick the role(s) this host plays:

# Monitored host (needs a TPM):
$ sudo systemctl enable --now etminan-agent-ima-policy.service etminan-agent.service
# Verifier device — the hourly attestation timer AND the RBAC daemon:
$ sudo systemctl enable --now etminan-verifier.timer etminan-verifierd.service

etminan-agent-ima-policy.service is a one-shot unit that writes the IMA measurement policy once, early at boot, before the agent daemon starts — zero manual reboot/policy steps needed. It's idempotent: if a policy is already active this boot, that's treated as success, not an error. etminan-verifier.timer runs etminan-verifier run hourly (OnBootSec=10min, OnUnitActiveSec=1h) — one pass over every enrolled host, firing the alarm path on anything wrong. etminan-verifierd.service is the long-running RBAC daemon that every operator action goes through: it authenticates each operator by Unix UID over a local socket, holds the single signing key, and signs authorised actions on their behalf (see Configuring the verifier to bootstrap the first admin, and the op identity reference).

From the prebuilt tarball

# verify, then unpack the signed release tarball
$ gpg --verify etminan-latest.tar.gz.asc etminan-latest.tar.gz
$ sha256sum -c etminan-latest.tar.gz.sha256
$ tar xzf etminan-latest.tar.gz

For a distro without .deb support, unpack the signed release tarball by hand (import the Etminan Team key from the Downloads section first). Copy the etminan-agent/etminan-verifier binaries onto PATH and the unit files under deploy/ into place, then create the service users and directories the .deb's postinst would have created for you. The .deb remains the recommended path on Debian/Ubuntu — it wires all of that up automatically.

Shell completion (bash/zsh/fish)

Tab-completes subcommand and flag names for both binaries (op's and op exclude's / op identity's nested actions included) — not flag values (host ids, file paths, etc.), which are operator-specific and not enumerable. Hand-written, not generated: neither binary uses clap (or any CLI-parsing crate) — both hand-roll std::env::args() + match — so there's no clap::Command for clap_complete to derive completions from. These scripts are a hand-maintained twin of the usage string in verifier/src/main.rs/agent/src/main.rs and the man pages — nothing automatically keeps them in sync if a flag changes, the same discipline this project already applies to this page and its Markdown twin.

Installed from the .deb package: nothing to do for zsh or fish — both load /usr/share/zsh/vendor-completions/ and /usr/share/fish/vendor_completions.d/ automatically. Bash needs the bash-completion package (a Suggests:, not a hard dependency — most Debian/Ubuntu systems already have it) to dynamically source /usr/share/bash-completion/completions/:

# if not already present
$ sudo apt install bash-completion

Open a new shell (or source /usr/share/bash-completion/completions/etminan-verifier in the current one) and etminan-verifier <Tab> / etminan-agent <Tab> should list subcommands.

Running from the tarball, not the .deb: the same files ship at deploy/completions/{bash,zsh,fish}/ — source or copy them by hand:

# bash, this session only:
$ source deploy/completions/bash/etminan-verifier
# zsh, this session only (repeat for etminan-agent):
$ fpath+=(deploy/completions/zsh) && autoload -U compinit && compinit
# fish, this session only:
$ source deploy/completions/fish/etminan-verifier.fish
03 — Configuring an agent host

Watched paths are config, never a hardcoded default.

Copy deploy/agent.env.example to /etc/etminan-agent/agent.env and edit it. An agent started with no watched-path setting watches nothing and prints a loud startup warning — deliberate, so a host never silently inherits another project's paths.

VariableDefaultPurpose
ETMINAN_AGENT_LISTEN0.0.0.0:7620Listen address for quote requests from the verifier.
ETMINAN_TLS_DIR/var/lib/etminan-agent/tlsWhere this agent's own mTLS keypair lives (generated once via keygen-tls).
ETMINAN_VERIFIER_CERT_FINGERPRINTunsetPins which verifier this agent accepts connections from. If unset, the agent refuses to start unless ETMINAN_ALLOW_PERMISSIVE_TLS=1 is also set (see below).
ETMINAN_ALLOW_PERMISSIVE_TLSunsetOpt-in for the one-time enrollment bootstrap: with the fingerprint unset, set this to 1 to accept a client certificate from any caller (only Quote is served, the log cursor never advances, SetProfile/PackageLookup are refused). Remove it once the fingerprint is pinned.
ETMINAN_WATCHED_PATHSunsetDirect, comma-separated path list. Highest precedence of the three ways to set watched paths.
ETMINAN_PROFILEunsetName of a profile in profiles.conf to watch. Second precedence, above a verifier-assigned profile.
ETMINAN_PROFILES_PATH/etc/etminan-agent/profiles.confWhere named profiles are looked up.
ETMINAN_ASSIGNED_PROFILE_PATH/var/lib/etminan-agent/assigned_profileWhere a profile name pushed by op assign-profile is persisted. Only relocate if /var/lib/etminan-agent isn't writable in your deployment.
ETMINAN_IMA_LOG_PATH/sys/kernel/security/ima/ascii_runtime_measurements_sha256IMA measurement log to read. Must be the SHA256-bank file, not the generic legacy one.
ETMINAN_IMA_POLICY_PATH/sys/kernel/security/ima/policyWhere write-ima-policy writes the IMA policy.

Watched-path precedence

Three ways to set watched paths, highest precedence first (per agent/src/profiles.rs's resolve()):

ETMINAN_WATCHED_PATHS

A direct, comma-separated list inlined in this host's own agent.env. Overrides everything else — use it for a one-off host that doesn't fit any shared profile.

Verifier-pushed assigned profile

A profile name centrally assigned via etminan-verifier op assign-profile (see the command reference below). Persisted locally in /var/lib/etminan-agent/assigned_profile.

ETMINAN_PROFILE

Looks up a named section in profiles.conf. Lets many hosts share one reusable path set instead of each inlining its own full list. Falls back to watching nothing (loud warning) if none of the three is set.

Writing a profiles.conf, worked example

profiles.conf is a hand-rolled [name] + one-path-per-line format — no TOML/YAML. Build one up for two shared fleets, fds-host and myapp-host:

/etc/etminan-agent/profiles.conf
# comments and blank lines are ignored

[fds-host]
/usr/local/bin/fds-proxy
/usr/local/bin/fds-admin
/etc/systemd/system
/etc/fds

[myapp-host]
/usr/local/bin/myapp
/etc/myapp

Set ETMINAN_PROFILE=fds-host in an fds fleet host's agent.env, ETMINAN_PROFILE=myapp-host on a myapp host, restart etminan-agent on each — done. Both fleets share this one file; each host only ever pulls its own section.

Two parsing behaviors are worth knowing before you rely on this file, since neither one produces an error:

A repeated [name] section is additive, not "last one wins"

[a]\n/one\n[a]\n/two produces a -> [/one, /two] — both path lists are kept, not the second replacing the first. If you meant to replace a profile's contents, delete the old section entirely rather than appending a second one with the same name.

A path line before the first [name] header is silently dropped

There is no "default" section — a path listed above every [name] header in the file belongs to no profile and is never swept, with no warning printed. Always put a [name] header first.

An empty section ([empty-profile] with no paths under it) is valid — it just watches nothing under that name. To confirm what an agent will actually resolve before restarting it for real, the same parser backs etminan-verifier op assign-profile's validation: pushing an unknown name gets refused with the exact list of names the agent found in its file.

How the centralized profile registry works

Rather than editing every host's agent.env by hand, the verifier can push a profile selection to a specific enrolled host — the profile's actual path list stays local to the agent, in its own profiles.conf. Only the choice of which named profile to use travels over the network:

The operator runs op assign-profile

$ etminan-verifier op assign-profile web-01 --profile myapp-host

A signed operator action, same accountability bar as op approve. The daemon authenticates your Unix UID and signs on your behalf; no key file is involved.

The verifier opens a dedicated connection

Same pinned mTLS as everything else, but this is its own request/response round trip — a SetProfile message carrying just the profile name, never routed through the quote-taking path and never touching the TPM.

The agent validates it locally

Looks up myapp-host in its own profiles.conf before accepting anything. If the name doesn't exist there, it refuses and replies with the list of profile names it does know about — the verifier can push a selection, never new path content, so an unrecognized name is always the agent's own configuration, not a mistake the verifier can inject.

The agent persists and acknowledges

On success, writes the selection to /var/lib/etminan-agent/assigned_profile and replies with SetProfileAck { applied: true, message: "profile 'myapp-host' assigned" }. The verifier only records the assignment in its own state.json — signed, audit-logged as profile_assigned — if ack.applied == true; it never claims success that didn't actually happen on the agent side.

It takes effect on the very next cycle

Watched paths are re-resolved fresh on every quote request, not once at agent startup — so a pushed assignment applies to the next quote taken, no agent restart required.

Every watched path — however it was set — is also picked up for event-triggered write detection: the agent watches for writes via unprivileged fanotify and sweeps a changed file immediately instead of waiting for the next hourly cycle — automatic, no extra config, needs Linux 5.13+ (older kernels fall back to per-cycle-sweep-only, still fully correct, just up to an hour slower).

What actually happens during a cycle

Three mechanics worth knowing exactly, since none of them are configurable — they're the fixed behavior every watched path goes through:

StepWhat it does
Sweep Before every quote, the agent recursively walks each watched path and reads every file it finds — this is what triggers the kernel's own IMA hook to measure and log it; the agent itself never computes or reports a content hash. There is no exclusion mechanism (no glob, no size/extension filter) — the only way to keep a file out of a cycle is to not list its containing path. A single file's userspace read is capped at 64 MiB (the kernel's own IMA measurement of that file is unaffected by this cap either way).
write-ima-policy Writes exactly two rules to the IMA policy node: measure func=BPRM_CHECK mask=MAY_EXEC and measure func=FILE_CHECK mask=MAY_READ fsuuid=<root-fs-uuid>. Useful to know verbatim if you ever need to sanity-check /sys/kernel/security/ima/policy against what this command is supposed to have written, or reason about interaction with a pre-existing custom IMA policy on the host.
Package lookup A separate, on-demand round trip from the verifier (never mixed with quote-taking) — capped at 200 paths per request (more than that is refused outright with an error, not truncated), each dpkg/rpm subprocess call bounded by a 5-second timeout. A host with neither package manager installed, or a path no installed package owns, comes back as "no package match" with no error — this is the normal, expected case for anything not installed via a system package manager.
04 — Setting up mTLS

Every connection is mutual TLS, pinned by fingerprint.

No plaintext fallback, no CA chain to manage. Neither binary does anything useful without a keypair — both refuse to start until one exists.

Generate the agent's keypair

On the agent host, once:

$ etminan-agent keygen-tls

Writes /var/lib/etminan-agent/tls/{cert,key}.pem (or $ETMINAN_TLS_DIR if set) and prints the certificate's SHA-256 fingerprint.

Generate the verifier's keypair

On the verifier device:

$ etminan-verifier keygen-tls

Copy the printed fingerprint into ETMINAN_VERIFIER_CERT_FINGERPRINT in every agent host's agent.env this verifier will talk to. Without it, the agent refuses to start unless you also set ETMINAN_ALLOW_PERMISSIVE_TLS=1 — the deliberate one-time enrollment bootstrap (accepts any client cert; only Quote is served, the log cursor never advances, SetProfile/PackageLookup are refused). Once the fingerprint is pinned, remove the flag.

Pin the agent's certificate

etminan-verifier op enroll captures the agent's certificate fingerprint on its first connection (trust-on-first-use) and stores it in state.json alongside the AK fingerprint. Confirm both out-of-band before trusting the enrollment, same discipline as an SSH host key on first connect. If the agent's keypair is ever regenerated, re-pin with etminan-verifier op rotate-tls rather than a full re-enrollment — AK trust and the approved baseline are untouched.

Rotating the verifier's own certificate instead is manual: keygen-tls, then update ETMINAN_VERIFIER_CERT_FINGERPRINT in every agent's agent.env and restart each agent, in that order — restarting before the env update lands locks that agent out.

05 — Configuring the verifier

State, baseline, and the RBAC daemon.

Copy deploy/verifier.env.example to /etc/etminan-verifier/verifier.env, then bring up the etminan-verifierd daemon and bootstrap your first admin — the daemon holds the single signing key and is what makes every approve/reject/exclude tamper-evident.

VariableDefaultPurpose
ETMINAN_VERIFIER_STATE/var/lib/etminan-verifier/state.jsonPer-host enrollment state: address, AK fingerprint, cumulative PCR value.
ETMINAN_BASELINE_DB/var/lib/etminan-verifier/baseline.dbSQLite store: approved / pending / rejected / excluded measurements, audit log.
ETMINAN_TLS_DIR/var/lib/etminan-verifier/tlsWhere this verifier's own mTLS identity lives.
ETMINAN_PENDING_REVIEW_SLA_HOURS24How long a pending item can sit unreviewed before it's escalated to a warning-severity finding.
ETMINAN_NOTIFY_EMAILunsetDestination for alarm emails. Unset disables the email leg only — the structured log and audit_log still happen.
ETMINAN_NOTIFY_FROMunsetFrom-address for alarm emails.
ETMINAN_NOTIFY_EMAIL_SEVERITIESallComma-separated severities that trigger an email; unset means every severity.
ETMINAN_NOTIFY_CHANNELSunsetComma-separated plugin channel names to enable (e.g. pagerduty,slack). Unset = feature fully off.
ETMINAN_NOTIFY_PLUGINS_CONF/etc/etminan-verifier/notify-plugins.confAllowlist file for notify plugins.
ETMINAN_NOTIFY_<CHANNEL>_SEVERITIESallPer-channel severity filter, e.g. ETMINAN_NOTIFY_PAGERDUTY_SEVERITIES=critical.
ETMINAN_CHANGE_SOURCESunsetComma-separated change-source plugin names to enable. Unset = feature fully off, no plugin ever executed.
ETMINAN_CHANGE_SOURCE_PLUGINS_CONF/etc/etminan-verifier/change-source-plugins.confAllowlist file for change-source plugins.

Plugin processes themselves read their own variables from this same environment (never the verifier binary) — e.g. ETMINAN_RT_BASE_URL/ETMINAN_RT_API_TOKEN for the Request Tracker reference plugin, or ETMINAN_PAGERDUTY_ROUTING_KEY/ETMINAN_SLACK_WEBHOOK_URL/ETMINAN_WEBHOOK_URL for the shipped notify plugins. See the Plugin API doc and each reference plugin's own comments for the full list.

Two of the paths above are the only things on this host that cannot be re-derived: state.json and baseline.db. Lose them and every enrolled host must be enrolled again, and every approval ever made is gone. Backup & disaster recovery covers what to copy, how often, and how to take the SQLite copy safely while the daemon is running.

# One time, as root at verifier bring-up: install the first admin identity.
# The daemon holds the signing key; operators never hold one.
# etminan-verifier op bootstrap --uid 1001 --label "admin-1"
# Thereafter, add operators by their Unix UID (admin only):
$ etminan-verifier op identity add --uid 1007 --role operator \
--scope region-a --label "carol (region-A operator)"
06 — Plugin execution sandboxing

A maliciously authored plugin still can't read your keys.

Every change-source/notify plugin (Request Tracker, PagerDuty, Slack, webhook, or anything custom) now runs as a separate, unprivileged etminan-verifier-plugin user instead of sharing etminan-verifier's own — so an allowlisted plugin that turns out to have been maliciously authored still can't read baseline.db or the daemon's signing key, just because it was allowed to execute at all. This is defense in depth on top of (not a replacement for) the existing integrity checks (allowlist, root-owned, pinned SHA-256) — see the Plugin API doc's security model section for the full picture, including what this does and doesn't protect against.

Installed from the .deb package: nothing to do. postinst already created the etminan-verifier-plugin user, and etminan-verifier.service already carries the AmbientCapabilities=CAP_SETUID CAP_SETGID grant that lets sandboxing actually happen. Confirm it's active:

$ getent passwd etminan-verifier-plugin
$ systemctl show etminan-verifier.service -p AmbientCapabilities

The second command should include CAP_SETUID and CAP_SETGID in its output. If either command comes back empty, see "Upgrading from an older install" or "Running from the tarball" below.

Protect the daemon signing key and the baseline database — this is the one thing sandboxing doesn't do for you automatically, because it's your files, not this project's:

$ ls -l /var/lib/etminan-verifier/daemon-signing.key /var/lib/etminan-verifier/baseline.db

Neither should be readable by etminan-verifier-plugin — in practice this almost always already holds with no changes needed (the daemon signing key is 0600 and owned by the etminan-verifier daemon user, not group-readable by anyone; baseline.db is created 0600, owned by etminan-verifier, and etminan-verifier-plugin isn't in that user's group by design — see debian/postinst's own comment on why it's deliberately a separate group). If you've manually loosened either file's permissions (e.g. chmod 644 while debugging something), tighten it back up — sandboxing only closes this gap if the file permissions actually keep the plugin user out.

Upgrading from an older install: apt upgrade/dpkg -i re-runs postinst, which creates the user and reloads systemd (systemctl daemon-reload) automatically — nothing else needed, since etminan-verifier.service is Type=oneshot (no persistent process to restart); the very next scheduled run (or manual op check) already picks up the new capability grant.

Running from the tarball, not the .deb (see "From the prebuilt tarball" above): postinst's steps aren't run for you, so do them by hand once:

$ sudo adduser --system --group --no-create-home \
  --shell /usr/sbin/nologin etminan-verifier-plugin

and copy deploy/etminan-verifier.service (which already carries the AmbientCapabilities=/CapabilityBoundingSet=/NoNewPrivileges= lines) into place rather than hand-rolling your own unit file.

Opting out: set ETMINAN_PLUGIN_USER= (explicitly empty) in verifier.env — plugins then run as etminan-verifier's own user, matching every install before this feature existed. There's rarely a reason to do this on a real deployment; it exists mainly for local development, where the dedicated user usually isn't provisioned anyway.

Running etminan-verifier interactively (sudo -u etminan-verifier etminan-verifier op check ..., during initial setup/testing, or any manual invocation outside the systemd unit) never holds CAP_SETUID/CAP_SETGID — only the unit's AmbientCapabilities= grants that. Sandboxing degrades automatically in that case: a warning is printed (could not drop plugin '<name>' to ETMINAN_PLUGIN_USER ...) and the plugin still runs, just as etminan-verifier's own user, exactly like before this feature existed. This is expected for interactive use, not a misconfiguration to chase down — only the scheduled run via etminan-verifier.timer needs the capability to actually be sandboxed.

07 — Enrolling a host

One trust-on-first-use moment, then everything is pinned.

$ etminan-verifier op enroll web-01 --addr 10.0.0.5:7620 \
--reason "onboarding web-01"

Takes a first quote, validates it self-consistently (refuses to enroll on a malformed/inconsistent quote rather than seed a bad baseline), and prints the AK fingerprint. Before pinning anything, enroll also runs an EK credential-activation challenge (TPM2_MakeCredential → ActivateCredential) and refuses to enroll unless the host's TPM recovers the sealed secret — proving the AK resides in a real TPM, not a software key of the right shape. Confirm the printed fingerprint out-of-band — e.g. read it off the host's console — before trusting it; the residency proof establishes it's a genuine TPM, this human step confirms it's the right host's TPM. --reason is required: the enrollment decision itself is signed and recorded in the audit log. No key file — the daemon authenticates your Unix UID and signs the record on your behalf.

Optional: require genuine manufacturer hardware with --ek-roots <dir>. Credential activation proves the AK and EK live in the same TPM, but not that that TPM is real silicon rather than a software/emulated one under a compromised hypervisor. When you pass a directory of trusted TPM-manufacturer CA certificates, enroll additionally requires the host to present an EK certificate (read from TPM NV) whose key matches the activated EK and which chains, by signature, to a self-signed manufacturer root in that directory — closing that gap for discrete TPMs:

$ etminan-verifier op enroll web-01 --addr 10.0.0.5:7620 \
--reason "onboarding web-01" \
--ek-roots /etc/etminan-verifier/ek-roots/

Put the manufacturer's root AND intermediate CA certificates (PEM or DER, one per file or bundled) in that directory — download them from your TPM vendor. With the flag, a host whose TPM ships no EK certificate (typical for vTPMs and swtpm) is refused, so it's a deliberate hardware-only mode; the verified certificate's fingerprint is recorded in the host's state and folded into the signed enrollment record. Without the flag, enrollment relies on residency alone (the default). Scope, stated plainly: RSA manufacturer CAs only (an ECDSA issuer is rejected with a clear error), and there is no CRL/OCSP revocation checking yet.

Every measurement from that first quote is recorded as pending, never auto-trusted — review and approve it to seed the baseline:

$ etminan-verifier op review web-01
$ etminan-verifier op approve web-01

What to expect right after enrollment: every entry in that first op review shows (new file, no prior baseline) — there is no baseline yet, so nothing can look like a modification. That's expected, not a warning sign; it's establishing the starting point, not detecting tampering. This is the one moment a single bulk op approve web-01 covering everything is normal — from the next hourly run onward, approvals should be small and specific to what actually changed.

08 — Change-source correlation

Was this change actually planned? Ask the ticket system.

When a pending measurement shows up in op review, package ownership answers "does this look like an install." Change-source correlation answers the harder question: was there a change ticket, maintenance window, or approved deploy that explains it? Purely informational — it never gates or auto-approves anything. A pending change is still only ever accepted via an explicit, signed op approve.

How it gathers information

For each pending measurement, the verifier runs every plugin named in ETMINAN_CHANGE_SOURCES as <plugin> --host <host_id> --anchor <RFC3339 timestamp> — the anchor is that measurement's first-seen time. The plugin itself queries the external system (e.g. Request Tracker's REST2 API) for anything matching that host within a time window around the anchor, and prints a JSON array of matches to stdout ([] for "checked, nothing found"). This runs entirely on the verifier, not the agent — a ticket system is a service the verifier can reach directly, so there's no agent round trip involved. Every configured and allowlisted source runs, and every match any of them finds is kept — never just the first.

Setting up the shipped Request Tracker plugin

Off by default — ETMINAN_CHANGE_SOURCES unset means no plugin is ever executed. To turn it on:

Install the plugin, root-owned

$ sudo install -D -m 0755 -o root -g root \
deploy/change-sources/request-tracker.sh \
/etc/etminan-verifier/change-sources/request-tracker.sh

Allowlist it with its pinned hash

$ sha256sum /etc/etminan-verifier/change-sources/request-tracker.sh

Add the name, path, and that hash as one line in /etc/etminan-verifier/change-source-plugins.conf:

request-tracker /etc/etminan-verifier/change-sources/request-tracker.sh <sha256sum output>

Configure RT access and enable it

In /etc/etminan-verifier/verifier.env — these are read by the plugin, not the verifier binary:

ETMINAN_CHANGE_SOURCES=request-tracker
ETMINAN_RT_BASE_URL=https://rt.example.com
ETMINAN_RT_API_TOKEN=1-24-abcdef0123456789...
# optional — omit to fall back to a Subject-only text search
ETMINAN_RT_HOST_FIELD=Host
# default shown; widen for maintenance-weekend changes
ETMINAN_RT_WINDOW_HOURS=24

ETMINAN_RT_BASE_URL must be https:// — the plugin sends the API token in a plain Authorization header on every request and refuses to run at all against an http:// base URL rather than send that token in cleartext.

Verify it checks out

$ etminan-verifier plugins verify

Confirm the query before trusting it

RT's TicketSQL date-literal format and your instance's actual custom field name are the two things this integration cannot know in advance — confirm them once:

$ ETMINAN_RT_DEBUG_QUERY=1 etminan-verifier op check web-01 --addr 10.0.0.5:7620
etminan-verifier: RT TicketSQL query for host 'web-01': (CF.{Host} = 'web-01' OR Subject LIKE 'web-01')
AND (LastUpdated > '2026-07-20 14:00:00' AND LastUpdated < '2026-07-22 14:00:00')

Paste that TicketSQL into RT's own web search UI (Search → Tickets → Advanced) and confirm it returns what you expect. If RT interprets the timestamps in a different timezone, or your custom field name doesn't match, this is where you'll see it — before it silently returns zero matches during a real review.

What it looks like in op review

Once configured, a second line appears under each pending entry's existing package-ownership verdict:

[42] host=web-01 path=/usr/local/bin/fds-proxy
Likely a legitimate install — matches package fds-proxy v2.3.1. Confirm this deploy was expected before approving.
Matches change RT#4821 (resolved) via request-tracker: 'Patch Tuesday rollout — web fleet' — confirm this change was expected.
detail: sha256=... (was ...) owner=uid:0/gid:0 mode=0755 size=... mtime=... first_seen=...

If nothing matched, or correlation hasn't run yet for this host, you'll see one of:

No approved change ticket found near this host for this time window across any configured system — verify this wasn't unplanned.
Change-source correlation not yet checked for this host — re-run `check`/`run` for this host.

If more than one configured source matches, every one is listed:

Matches 2 possible confirmations across configured systems: RT#4821 (resolved) via request-tracker; CHG0093 (approved) via servicenow

None of this changes what op approve/op reject do — it's context for the operator making that call, not an input to it. Adding a second ticket system is the same plugin recipe as any other category: see the Plugin API and the full change-source worked example.

09 — Notification channels

How an alarm actually reaches somebody.

etminan-verifier run always prints every finding, and emails them too if ETMINAN_NOTIFY_EMAIL is set — that leg is built-in, not a plugin, and predates this feature. PagerDuty, Slack, and a generic webhook are additional channels, each an external plugin process under the same Etminan Plugin API as change-source correlation — no in-process HTTP client exists in the verifier at all.

When and how a channel is executed

At the end of a op check/run cycle, if there's at least one finding, the verifier runs every plugin named in ETMINAN_NOTIFY_CHANNELS. Each finding is first filtered against that channel's own ETMINAN_NOTIFY_<CHANNEL>_SEVERITIES (unset means every severity reaches that channel), then the surviving findings are piped as one JSON array on the plugin's stdin — never argv, since finding text is free-form and shouldn't be shell-escaped into a command line:

what a notify plugin receives on stdin
[
  {"host_id": "web-01", "text": "quote signature failed to verify", "severity": "critical",
   "kind": "signature-invalid", "formatted": "[signature-invalid] web-01: quote signature failed to verify"},
  {"host_id": "web-02", "text": "3 pending item(s) aged past the 24h review SLA", "severity": "warning",
   "kind": "sla-exceeded", "formatted": "[sla-exceeded] web-02: 3 pending item(s) aged past the 24h review SLA"}
]

A channel's exit code is the only signal read back: 0 means it fired successfully, non-zero means "didn't fire this cycle" — logged, never retried, never allowed to block the rest of the cycle or the other configured channels. kind is a stable, machine-matchable category (e.g. signature-invalid, sla-exceeded); formatted is this channel's own pre-rendered display text, reshapeable per channel via ETMINAN_NOTIFY_<CHANNEL>_FORMAT with no plugin script edit. Full field-by-field reference and every kind value: Notification channels.

What each shipped reference plugin actually does with that input

PluginBehaviorNeeds
pagerduty.sh One Events API v2 event per finding, with dedup_key = "attest-<host_id>" — repeat findings for the same host correlate into one incident instead of paging fresh every cycle. ETMINAN_PAGERDUTY_ROUTING_KEY
slack.sh One digest message for the whole batch — a noisy cycle with many findings is still a single Slack message, not a flood. ETMINAN_SLACK_WEBHOOK_URL
webhook.sh Passes the findings array through unmodified as the POST body — the generic escape hatch, and the template to copy for a fully custom integration. ETMINAN_WEBHOOK_URL, optional ETMINAN_WEBHOOK_AUTH_HEADER

Setting up a channel

Install the plugin(s), root-owned

$ sudo install -D -m 0755 -o root -g root \
deploy/notify-plugins/pagerduty.sh \
/etc/etminan-verifier/notify-plugins/pagerduty.sh

Only install the channels you're actually enabling.

Allowlist each one with its pinned hash

$ sha256sum /etc/etminan-verifier/notify-plugins/pagerduty.sh

Add each as one line in /etc/etminan-verifier/notify-plugins.conf:

pagerduty /etc/etminan-verifier/notify-plugins/pagerduty.sh <sha256sum output>
slack /etc/etminan-verifier/notify-plugins/slack.sh <sha256sum output>

Enable and configure each channel

In /etc/etminan-verifier/verifier.env:

ETMINAN_NOTIFY_CHANNELS=pagerduty,slack
ETMINAN_PAGERDUTY_ROUTING_KEY=...
ETMINAN_SLACK_WEBHOOK_URL=https://hooks.slack.com/services/...
# optional: page only on critical findings, let slack see everything
ETMINAN_NOTIFY_PAGERDUTY_SEVERITIES=critical

ETMINAN_NOTIFY_CHANNELS is the master switch: unset or empty, this feature never executes any plugin at all. Every other setting only matters once a channel's name is both listed here and allowlisted in the previous step.

Verify before trusting it

$ etminan-verifier plugins verify

Do this right after setup — don't wait for a real finding to discover a typo'd hash or a missing routing key. This same check also runs automatically at the start of every run cycle; a failure there becomes an ordinary warning-severity finding, so a broken notify channel can never fail silently.

Adding a channel that isn't shipped (Microsoft Teams, SMS, an internal on-call system) is the same plugin recipe as any other category — write a script that reads the findings JSON from stdin and exits non-zero on failure: see the Plugin API and the full notification-channels worked example.

10 — Command reference

Every subcommand either binary ships with.

This is the complete surface of both CLIs — nothing else exists. Running either binary with no arguments (verifier) or an unrecognized subcommand prints this same list. Every subcommand below — and every nested op/plugins action — also accepts -h/--help for a focused, in-terminal usage summary (e.g. etminan-verifier op approve --help); this page remains the fuller reference, with worked examples and every ETMINAN_* environment variable.

etminan-agent

etminan-agent

No subcommand: runs the long-running daemon. Loads its mTLS keypair, checks watched paths resolve (warns loudly, doesn't refuse to start, if they don't), starts the fanotify write-watcher, and listens for quote/package-lookup requests from its verifier.

$ etminan-agent
etminan-agent: watched paths at startup: ["/usr/local/bin/fds-proxy", ...]
etminan-agent: listening on 0.0.0.0:7620
etminan-agent keygen-tls

Generates this host's mTLS keypair once, under $ETMINAN_TLS_DIR (default /var/lib/etminan-agent/tls), and prints its SHA-256 fingerprint for pinning on the verifier side.

$ etminan-agent keygen-tls
wrote /var/lib/etminan-agent/tls/{cert,key}.pem
fingerprint: 3a1f...c2
etminan-agent write-ima-policy

Writes the IMA measurement policy to $ETMINAN_IMA_POLICY_PATH (default /sys/kernel/security/ima/policy). Meant to run once, early at boot, via etminan-agent-ima-policy.service — not something to run interactively on a host that's already measuring. Idempotent: an already-active policy this boot is treated as success, not an error.

$ sudo etminan-agent write-ima-policy

etminan-verifier — lifecycle

etminan-verifier keygen-tls

Same as the agent's version — generates the verifier's own mTLS keypair and prints its fingerprint. Run once per verifier device.

$ etminan-verifier keygen-tls
etminan-verifier keygen --out <path>

Generates a standalone Ed25519 key for the root break-glass path only. Day-to-day operators hold no signing key: they act through op, and the etminan-verifierd daemon holds the single signing key and signs on their behalf (see op identity below). This --key flow is retained only for a root operator recovering a wedged daemon, and every use is flagged in the audit log — keep any such key safe and never commit it to a repo.

$ etminan-verifier keygen --out /etc/etminan-verifier/breakglass.key
etminan-verifier keygen-catalog --out <path>

One-time, offline: generates the dedicated, single-purpose Ed25519 keypair a real plugin catalog release is signed with — deliberately not the same key as keygen's break-glass key above, and not the GPG key that signs release tarballs. Run wherever catalog releases are actually built, never on a verifier host, then replace CATALOG_VERIFYING_KEY_HEX in verifier/src/plugin_catalog.rs with the printed public key and rebuild. See Plugins below for the install commands this key signs for.

$ etminan-verifier keygen-catalog --out catalog-signing.key
etminan-verifier op enroll <host> --addr <ip:port> --reason <text> [--force] [--ek-roots <dir>]

Enrolls a new monitored host: takes and validates a first quote, prints its AK fingerprint for out-of-band confirmation, and records every subsequent measurement as pending review. --ek-roots <dir> (optional) additionally requires a manufacturer EK-certificate chain to a trusted root in <dir>, proving genuine hardware rather than only same-TPM residency (a host with no EK cert is then refused). Carries no key — the daemon authenticates your Unix UID and signs on your behalf. See Enrolling a host above for the full walkthrough.

$ etminan-verifier op enroll web-01 --addr 10.0.0.5:7620 --reason "onboarding web-01"
etminan-verifier op check <id> --addr <ip:port>

Checks one already-enrolled host interactively, outside the hourly schedule — useful right after a change to confirm the result immediately rather than waiting for the next run. A plain read-only check — it changes no trust state and goes through no authorization.

$ etminan-verifier op check web-01 --addr 10.0.0.5:7620
etminan-verifier run

The main scheduled command — normally invoked by etminan-verifier.timer, hourly. Checks every enrolled host, diffs against the approved baseline, correlates package ownership and any configured change-source/notify plugins, escalates stale pending items past the SLA, and fires the alarm path (print, email, notify plugins, audit log) on any finding. Also runs plugins verify automatically every cycle, and verifies the audit_log hash chain every cycle — a break, gap, or rollback of the audit trail raises a critical audit-chain-invalid finding, so tampering is caught on the next run, not only when checked by hand.

$ etminan-verifier run
etminan-verifier op assign-profile <host> --profile <name>

Pushes a watched-path profile selection to a host, rather than editing that host's agent.env by hand — the profile's actual contents still live in the agent's own profiles.conf. A signed operator action, same accountability bar as op approve.

$ etminan-verifier op assign-profile web-01 --profile myapp-host
etminan-verifier op rotate-tls <host> --reason <text>

Re-pins a host's TLS certificate fingerprint after its agent's mTLS keypair was regenerated — without a full re-enrollment. Requires the AK fingerprint to still match; PCR replay continues from the stored value. Signed and audit-logged like every other trust-changing command.

$ etminan-verifier op rotate-tls web-01 --reason "agent TLS keypair regenerated after host reimage"
etminan-verifier op rotate-ak <host> --reason <text> [--ek-roots <dir>]

Re-binds a host's attestation key to a new one within the same TPM, without a full re-enrollment — the AK analogue of op rotate-tls. Use after a TPM owner-clear or firmware change re-derived the AK. The host's Endorsement Key is the continuity anchor: a fresh EK credential-activation ceremony proves the new AK lives in the same TPM as the EK already on record, and the EK fingerprint must still match. A different EK means a different TPM — a deliberate re-enroll, refused here. Baseline, TLS pin, and PCR replay state are preserved; signed and audit-logged.

$ etminan-verifier op rotate-ak web-01 --reason "TPM cleared during motherboard swap"

etminan-verifier op identity — the RBAC identity registry

Decides which operators may perform trust-changing actions. Operators are identified by their Unix UID (kernel SO_PEERCRED, unforgeable for non-root), not a key file. The daemon maps each UID to exactly one roleadmin (manages the registry, always unscoped), operator (day-to-day approve / reject / exclude / enroll / assign-profile / rotate, confined to a host-group scope), or viewer (read-only, signs nothing) — and the matrix is default-deny: an unmapped or revoked UID is refused every trust-changing action. All identity management is admin-only.

etminan-verifier op bootstrap --uid <n> --label <name>

One-time genesis admin — run as root at verifier bring-up. Binds UID <n> as the first admin, so it can add everyone else.

$ sudo etminan-verifier op bootstrap --uid 1000 --label "alice"
etminan-verifier op identity add --uid <n> --role admin|operator|viewer --scope <glob> --label <name>

An admin maps a Unix UID to a role, and — for an operator — a host-group scope (admins are always unscoped). Add a second admin first, so one can revoke the other.

$ etminan-verifier op identity add --uid 1002 --role operator --scope "web-*" --label "ci-approver"
etminan-verifier op identity revoke --uid <n>

Revoke a UID. Actions it took before the revocation stay valid; from then on it is refused every trust-changing action (default-deny). Refuses to revoke the last active admin (add a second admin first).

$ etminan-verifier op identity revoke --uid 1002
etminan-verifier op identity list

Show every mapped identity: its UID, role, scope, label, and status (active or revoked).

etminan-verifier — review, approval & exclusions

etminan-verifier op review [<id>]

Lists pending measurements. Each prints a one-sentence plain-language verdict first (e.g. "Likely a legitimate install — matches package nginx v1.24.0"), then change-ticket correlation if configured, then package-transaction/trust-store detail, with raw sha256/owner/size/mtime underneath for anyone who wants to double-check. Omit the host argument to review every host at once.

$ etminan-verifier op review web-01

Every pending entry is exactly one of two kinds, and the status field tells you which: (new file, no prior baseline) means this path has never been part of an approved baseline on this host — a genuinely new file. (was <old-sha256>) means this is a previously-approved path whose content hash no longer matches what was last approved — an existing, trusted file changed. There's no third bucket shown here: a hash that still matches what's already approved never becomes pending at all, it's silently counted as unchanged.

[42] host=web-03 path=/usr/local/bin/agentd
    ⚠ An existing trusted file changed and is NOT owned by any installed
    package — this pattern is a common sign of tampering. Investigate
    before approving.
    No approved change ticket found near this host for this time window
    across any configured system — verify this wasn't unplanned.
    detail: sha256=3dcbe799... (was a1b2c3...) owner=uid:0/gid:0 mode=0755
    size=48213 mtime=2026-07-21T14:02:11+00:00 first_seen=2026-07-21T14:05:00+00:00

(was a1b2c3...) is what marks this as a modification, not a new file — if agentd had never existed on this host before, that field would instead read (new file, no prior baseline). Everything else on the detail: line (owner, mode, size, mtime) is best-effort context for a human reviewer only, from a plain lstat() on the agent side — never part of the cryptographic trust decision, only path + sha256 matching the signed IMA measurement is.

etminan-verifier op approve <host>

Approves every pending item for one host in a single signed batch. Carries no key and no --reason: the daemon authenticates your Unix UID, checks your role and scope, and signs the batch on your behalf — attributed to you and folded into the tamper-evident audit chain.

$ etminan-verifier op approve web-01
etminan-verifier op approve <host> --matching <path> --hash <sha256>

Disambiguating form: when a single path has more than one pending hash on a host, approve exactly one of them by naming the path and its exact hash (the two flags must be given together). Scoped to the one named host.

$ etminan-verifier op approve web-01 --matching /usr/local/bin/fds-proxy --hash <sha256>
etminan-verifier op reject <host> --path <path> --hash <sha256>

Rejects a specific measurement — fires the alarm path immediately as a confirmed incident, not a soft warning. Same authorization requirement as approve: a rejection is a trust decision too, authenticated and signed by the daemon.

$ etminan-verifier op reject web-01 --path /usr/local/bin/fds-proxy --hash <sha256>
etminan-verifier op exclude create --path <path|prefix*> --reason <text> [--host <id>] [--confirm yes]

Suppressing recurring false alarms: approving the same noisy path over and over (a log file rewritten every cycle, a cache file, an auto-generated artifact whose hash legitimately changes) isn't what op approve is for — it just re-baselines one hash at a time, and the path comes right back as pending next cycle once it changes again. If you find yourself approving the same path repeatedly with no meaningful change in what it represents, that's the signal to create an exclusion instead — it stops the path from ever generating a pending item again, rather than re-approving it every cycle.

Creates a suppression rule for a path that's expected to churn and shouldn't generate pending review at all. An exact path needs no confirmation; a trailing * glob prints how many currently-known paths it would match and refuses to proceed without --confirm yes — a broad glob widens what's silently excluded going forward, so it can't be created by accident.

Without --host, the rule applies to every enrolled host — this was the only behavior available before the host-scoping fix landed, and stays the default so existing workflows keep working unchanged. Pass --host <id> to scope a rule to exactly one host instead: a rule created while reviewing one host's pending queue no longer silently also hides the same path if it later shows up as a genuinely different, unreviewed change on another host. op exclude list's output shows each rule's scope.

$ etminan-verifier op exclude create --path /var/log/myapp/current.log \
--reason "rotates hourly, content never security-relevant"

A trailing * is a plain string prefix match, nothing more — /etc* also matches an unrelated sibling like /etcxyz/anything, not just paths under /etc/. This is exactly why the preview step exists: it lists every currently-known path the pattern would actually match, so read that list before adding --confirm yes rather than assuming the pattern means what it looks like it means. If you want "everything under this directory," write the pattern with the trailing slash included: --path /etc/*.

etminan-verifier op exclude list

Lists every active exclusion rule with its id, pattern, reason, and who created it.

$ etminan-verifier op exclude list
etminan-verifier op exclude revoke --id <n> --reason <text>

Revokes an exclusion rule by id. Signed like creation — revoking re-exposes whatever the rule was hiding to review, exactly as consequential a decision as creating one.

$ etminan-verifier op exclude revoke --id 3 --reason "no longer needed"

An exclusion suppresses review, not measurement — the IMA policy still measures the path every cycle exactly as before; a match just goes to excluded_log instead of pending_review, so there's a permanent, auditable record of every measurement an exclusion rule ever hid, not a silent drop. op review never shows anything an active exclusion matched; exclude list is where you audit which paths are currently suppressed and why.

etminan-verifier op verify-signatures

Re-checks every approval batch's signature, every exclusion rule's create/revoke signature, and the full hash-chained audit-log in one pass. Entries written before this feature existed show up as "unsigned, predates this feature" rather than a failure. Run on demand, or schedule alongside run as a detective control.

It also confirms every past trust-changing action was signed by a signer that was authorized for its role at that action's time. Routine operator actions are signed by the etminan-verifierd daemon on the operator's behalf, so the check anchors on the daemon's key and the audit chain; any root break-glass --key action is additionally reconciled against the break-glass key registry — an action signed by a key at/after its revocation is a hard failure; one inside a revoked key's --compromised-since window is flagged for re-review; actions from before the registry existed are grandfathered.

$ etminan-verifier op verify-signatures

etminan-verifier — plugins

etminan-verifier plugins verify

Checks every plugin currently referenced by ETMINAN_CHANGE_SOURCES or ETMINAN_NOTIFY_CHANNELS: confirms each has an allowlist entry and passes the full security check (root-owned, not group/world-writable, content hash matches the pinned SHA-256) — without executing it. Exits non-zero and prints exactly what's wrong if anything fails. Also run automatically at the start of every run cycle.

$ etminan-verifier plugins verify
etminan-verifier plugins list

Fetches and verifies the certified plugin catalog (its Ed25519 signature checked against a compiled-in trust anchor before anything in it is trusted), then prints every entry with its install status: not installed, installed and up to date, or update available. Read-only — never installs or modifies anything. Note: the catalog itself isn't published anywhere yet, so this currently errors on the fetch — the client side is real and tested, the publishing pipeline isn't built yet.

$ etminan-verifier plugins list
etminan-verifier plugins install <name>

Downloads a certified catalog entry, re-verifies its content hash against the catalog's pinned value, writes it under ETMINAN_PLUGIN_INSTALL_DIR's <type>-plugins/ subdirectory (default /etc/etminan-verifier), and appends the matching allowlist entry — held to the identical bar plugin_exec.rs applies to a hand-installed plugin, including a final re-verification of the file actually on disk before this ever reports success. Must run as root (the file it writes has to be root-owned). Does not add <name> to ETMINAN_NOTIFY_CHANNELS/ETMINAN_CHANGE_SOURCES — it prints the exact line to add and a reminder to restart etminan-verifier.service. Refuses if <name> is already installed — use plugins update instead.

$ sudo etminan-verifier plugins install pagerduty
etminan-verifier plugins update <name>

Re-fetches and re-verifies the catalog, and if the pinned hash for <name> actually changed, downloads the new content, re-hashes it, and re-pins the allowlist entry. Only runs when invoked — never automatic or backgrounded. Requires <name> to already be installed.

$ sudo etminan-verifier plugins update pagerduty

etminan-verifier — notify

etminan-verifier notify-preview [--channel <name>]

Renders four representative fake findings through whatever _TEMPLATE/_FORMAT/_SEVERITIES is currently configured, and prints the result: the full email (subject + body) and, per channel in ETMINAN_NOTIFY_CHANNELS (or just --channel <name>), the raw JSON payload that would hit its plugin's stdin. Never executes a plugin, never calls sendmail — safe to run any time, including against a channel that isn't allowlisted yet. See Notification channels for the full explanation.

$ etminan-verifier notify-preview
$ etminan-verifier notify-preview --channel pagerduty
etminan-verifier notify-test --channel <name>

plugins verify confirms a channel is allowlisted, root-owned, and its hash matches — an integrity/config check, not a functional one; it never executes the plugin. notify-test closes that gap: it builds one synthetic finding and pipes it through the channel's real invocation path — the exact same verify-then-exec call a live run cycle uses — then reports the plugin's actual exit status back to you, so a wrong routing key or an expired webhook URL surfaces immediately instead of the first time a real alert needs to go out. See Notification channels for the full contract.

$ etminan-verifier notify-test --channel pagerduty
11 — Day-to-day operation

Read the verdict, act on it, sign what you approve.

etminan-verifier.timer calls run hourly. It confirms the presented AK still matches enrollment, the signature is valid, and the replayed IMA log's digest matches the signed PCR10 — any failure is a critical finding, fed straight to the alarm path and recorded in the audit log. Unchanged measurements are silent; new or changed ones go to pending_review. Any pending item older than the review SLA (24h by default) is escalated to a warning-severity finding — pending review is never a silent, indefinite state.

When package-ownership correlation finds a match, baseline review shows a third line beyond ownership: whether this exact version arrived via a real, timely, authenticated transaction — e.g. "✓ installed via an authenticated apt transaction at 2026-07-21 10:00:00." or, on an rpm host, "✓ signed with trusted key a1b2c3d4e5f6a7b8." A mismatch is a loud "⚠ ... — treat with suspicion." rather than a soft note. Note dpkg's line never claims a signature check — only whether the install went through APT's own authentication — the wording says so explicitly rather than implying rpm-level confidence it doesn't have.

Every command that changes trust-relevant state — op enroll, op approve/op reject/exclude create/revoke, op assign-profile, op rotate-tls — is signed by the etminan-verifierd daemon on the authenticated operator's behalf (the operator holds no key) and appended to a self-contained, hash-chained audit log: deleting, editing, or reordering any past entry breaks the chain for everything recorded after it. No external tool is involved. Verify the whole store on demand with op verify-signatures.

12 — Monitoring the verifier

Who watches the watcher.

The verifier is the single place an alarm can fire — so if it stops working, silence looks exactly like “all healthy”. A broken or compromised verifier can't be trusted to report its own failure, so an independent monitor, running on a separate system this box does not control, should watch everything the verifier's function depends on. It is deliberately product-agnostic — any external uptime / dead-man's-switch monitor works; what matters is which signals it watches:

Emit the heartbeat on a successful cycle and let an off-box service alert when a ping is overdue or a check fails:

# the endpoint is any external heartbeat / uptime monitor you run off this box:
$ etminan-verifier run && curl -fsS --retry 3 https://<your-heartbeat-endpoint>/<check-id>

Give each check a grace period slightly longer than the cycle interval (e.g. alert if no heartbeat within 75 minutes for the hourly timer) so one slow cycle doesn't page anyone, but two consecutive misses reliably does. The verifier-hardening guide covers the full rationale.

13 — Extending

Three extension points, none of them need a redesign.

A new watched-path profile

No agent code change needed. Edit /etc/etminan-agent/profiles.conf (copy from deploy/profiles.conf.example first) — a hand-rolled [name] + one-path-per-line format, # comments and blank lines ignored:

[myapp-host]
/usr/local/bin/myapp
/etc/myapp

Then set ETMINAN_PROFILE=myapp-host in that host's agent.env (or push it centrally with op assign-profile) and restart etminan-agent. Startup warns, but doesn't refuse to start, if any listed path doesn't exist.

Another package manager

Package-ownership lookups are pluggable via the PackageChecker trait in agent/src/package_lookup.rs. Dpkg and Rpm are the two existing implementations — each shells out to that manager's query command and parses the one line it needs, returning None on any failure rather than panicking. Add a new struct implementing the trait and one line in the checkers() list — nothing else in the verifier or protocol needs to change.

A ticket system or alert channel — the Etminan Plugin API

Change-ticket correlation and alarm/notification channels are both external, allowlisted, hash-pinned plugin processes — no Rust code, no verifier rebuild, ever. Write a script matching the category's contract, allowlist it by name/path/pinned SHA-256, and reference its name in an environment variable:

# sha256sum the plugin, then add it to the allowlist file:
$ sha256sum /etc/etminan-verifier/notify-plugins/my-plugin.sh
# notify-plugins.conf: <name> <path> <sha256>
# then enable it and verify:
$ etminan-verifier plugins verify

For a plugin already published in Etminan's certified catalog, this manual dance is optional: plugins list shows what's available and each entry's install status, and plugins install <name> downloads, re-verifies, and allowlists it for you — still root-only, still hash-pinned to the identical bar plugin_exec.rs checks at execution time. Writing your own script this way remains the only path for anything not (yet) in the catalog.

Request Tracker ships as the reference change-source plugin; PagerDuty, Slack, and a generic webhook ship as reference notify plugins. Full contract, worked examples, and the shared security model: Plugin API, change-source correlation, notification channels, Plugin Directory.

14 — Data & signature reference

For anyone building tooling on top of this, not just using the CLI.

Everything above is enough to operate the tool day to day through its own commands. This section is for a different reader: someone writing a backup/migration script, an independent compliance report, or a signature check that doesn't go through etminan-verifier itself.

state.json — per-host enrollment state

One JSON object per enrolled host, keyed by host id, at $ETMINAN_VERIFIER_STATE (default /var/lib/etminan-verifier/state.json). Written atomically (temp file + fsync + rename) on every update.

FieldMeaning
addrHost's ip:port, recorded at enroll time so run never needs --addr re-supplied.
ak_public_marshaled_hex / ak_fingerprint_sha256_hexThe enrolled AK public key and its SHA-256 fingerprint — every later quote is checked against this.
ek_public_marshaled_hex / ek_fingerprint_sha256_hexThe host's pinned Endorsement Key public and fingerprint from enroll's credential-activation ceremony (proof the AK resides in a real TPM). Empty for hosts enrolled before EK activation existed — re-enroll to bind an EK.
ek_cert_fingerprint_sha256_hexSHA-256 of the manufacturer EK certificate, set only when the host was enrolled with --ek-roots and its EK cert chained to a trusted root (hardware provenance verified). Empty for residency-only enrollments (the default, and the norm for vTPMs). Also folded into the signed enrollment payload.
tls_cert_fingerprint_sha256_hexThe agent's pinned TLS certificate fingerprint from the TOFU ceremony. Empty for hosts enrolled before mTLS existed — those need re-enrollment.
cumulative_pcr10_hex / last_log_offsetReplay state: the running PCR10 value and the IMA log offset already consumed.
assigned_profile / _reason / _signed_by / _signature_hexThe profile name last pushed by op assign-profile, and the operator accountability for that push.
enrolled_by / enrolled_reason / enrollment_signed_payload / enrollment_signature_hexWho enrolled this host, why, and the signed proof of that decision.
tls_rotated_by / _reason / _signed_payload / _signature_hexAccountability for the most recent op rotate-tls — only the latest; the full history is in audit_log.
ak_rotated_by / ak_rotated_reason / ak_rotation_signed_payload / ak_rotation_signature_hexAccountability for the most recent op rotate-ak — only the latest; the full history is in audit_log (action="ak_rotated"). Empty until op rotate-ak has run for this host.

baseline.db — SQLite schema

At $ETMINAN_BASELINE_DB (default /var/lib/etminan-verifier/baseline.db), WAL mode — back up the -wal/-shm sidecars alongside it.

TableKey columns
approved_measurements(host_id, path) primary key; sha256, baseline_batch_id, approved_by, approved_at.
pending_reviewhost_id, path, sha256, previous_sha256, file metadata (uid/gid/mode/mtime/size), package-correlation fields (package_name/_version/_manager/_transaction_time/_trust_verified/_trust_detail), first_seen_at, change_checked_at. Unique on (host_id, path, sha256).
rejectedhost_id, path, sha256, rejected_at. Unique on (host_id, path, sha256) — idempotent re-rejection.
exclusion_rulespath_pattern, reason, created_by/_at, revoked_at (null while active), host_id (null = global rule, applies to every enrolled host; otherwise scoped to one host, set via op exclude create --host).
audit_logseq, recorded_at, actor, action, resource, severity, detail, prev_hash, entry_hash — see the hash-chain reference below.
baseline_batchesbatch_id, host_id/match_pattern (whichever form of op approve was used), signed_payload, signature_hex, approved_by/_at, reason.
excluded_loghost_id, path, sha256, matched_pattern, observed_at — every measurement an exclusion rule ever suppressed, for audit purposes.
change_correlationshost_id, checked_at, source (the plugin name), change_id, summary, status, url.

Signed action payloads

Every signed action's payload is deterministic, canonical JSON — object keys sorted lexicographically at every nesting level, no whitespace between tokens, string fields escaped exactly as JSON requires (a literal newline inside a field becomes the two characters \n, never a raw newline byte). Built this way specifically so the exact bytes signed can be reconstructed and re-verified outside this tool (e.g. with a Python or openssl Ed25519 check), not just trusted because the tool says so — and so that a field containing attacker-influenceable content (a path sourced from the IMA log on a possibly-compromised host, which can legally contain a newline) can never be ambiguous about where one field ends and the next begins. Prior to 2026-07-22 these payloads were hand-rolled \0/\n-joined text; already-signed rows keep verifying against their own stored bytes regardless of which format produced them, so this was a forward-only change, not a migration.

ActionPayload format
op approve{"entries":[{"host_id":...,"path":...,"sha256":...},...],"reason":...} — entries sorted lexicographically first, so the same batch always signs identically regardless of processing order.
op exclude create/revoke{"action":"create"|"revoke","pattern":...,"reason":...} — the action word is baked in so a "create" signature can never be replayed to satisfy a "revoke" of the same rule.
op reject{"host_id":...,"path":...,"reason":...,"sha256":...}
op enroll{"ak_fingerprint":...,"ek_fingerprint":...,"host_id":...,"reason":...,"tls_fingerprint":...} — plus "ek_cert_fingerprint":... when enrolled with --ek-roots (added only when hardware provenance was verified, so a residency-only payload stays byte-identical to before the feature existed).
op rotate-tls{"host_id":...,"new_tls_fingerprint":...,"old_tls_fingerprint":...,"reason":...}
op rotate-ak{"host_id":...,"new_ak_fingerprint":...,"old_ak_fingerprint":...,"reason":...}
op assign-profile{"host_id":...,"profile_name":...,"reason":...}

Signatures are Ed25519 over the exact payload bytes above, hex-encoded (128 hex chars). Every one of these is signed by the etminan-verifierd daemon on the authenticated operator's behalf — the operator holds no key. The daemon's own signing key (/var/lib/etminan-verifier/daemon-signing.key) is a raw 32-byte seed, hex-encoded (64 hex chars) at file mode 0600 — no PEM, no DER, nothing but hex text.

audit_log — the hash chain, and how to recompute it independently

Each row's entry_hash is SHA256(canonical_json({seq, recorded_at, actor, action, resource, severity, detail_canonical: detail, prev_hash})), hex-encoded. "Canonical JSON" means object keys sorted lexicographically at every nesting level and no whitespace between tokens — deterministic regardless of what produced the row. The very first row's prev_hash is 68 zero characters (the genesis sentinel, not a real hash of anything). op verify-signatures already walks this chain for you; the formula above is what you'd reimplement to check it with an independent script instead of trusting this binary's own verification.

Alarm severities

Only two values exist anywhere in this project — there is no "info" or "error" tier for findings, by design:

SeverityWhat triggers it
criticalA genuine verification failure (invalid signature, AK/TLS fingerprint mismatch, PCR digest that doesn't match the replayed log, an unreachable host) or an operator op reject.
warningA pending item aged past the review SLA, or a configured plugin failing its plugins verify check.

Both values are valid in every severity-filter env var — ETMINAN_NOTIFY_EMAIL_SEVERITIES and every ETMINAN_NOTIFY_<CHANNEL>_SEVERITIES — as a comma-separated list.

15 — Troubleshooting

What a given symptom usually means.

Agent watches nothing at startup

Neither ETMINAN_WATCHED_PATHS nor ETMINAN_PROFILE is set, or the named profile doesn't exist in the profiles file — check the startup warning, which names the missing profile and lists known ones.

FINDING: ... PCR digest mismatch

Either a fabricated log (check the host's IMA violations counter) or a log/offset desync. Re-enroll to establish a fresh baseline.

FINDING: ... AK fingerprint mismatch

The host is presenting an AK that doesn't match enrollment — re-enroll after confirming out-of-band why the AK changed (TPM replaced, host reimaged, or something more concerning).

FINDING: ... unreachable, but the agent is actually up

Likely a TLS certificate fingerprint mismatch — check whether the agent's TLS keypair was regenerated (keygen-tls) without a matching op rotate-tls on this side. Confirm out-of-band that the new certificate is legitimate before running op rotate-tls, same discipline as enrollment.

Pending item stuck past the review SLA (24h by default)

run escalates it to a warning-severity finding automatically; review and approve/reject it rather than letting it accumulate. Set ETMINAN_PENDING_REVIEW_SLA_HOURS if 24h doesn't match your review cadence.

A configured plugin never seems to run

Run etminan-verifier plugins verify — it checks allowlist membership, ownership, permissions, and content hash for every currently-referenced plugin and prints exactly which check failed.