CYF-03 · Detection loom
Evaluates 14 labelled traces — not live detections.
Detection rules fail quietly in both directions: too loose and every benign hand-off is an alert, too tight and the real sequence never fires. This bench makes both costs visible by replaying the rule against 14 labelled traces — 6 attacks, 8 benign, with the false-positive bait left in on purpose.
The same rule is also evaluated under all three same-second ordering assumptions, because which source's clock a SOC trusts is an organisational choice, not a fact about the evidence.
Where this bench sits in a regime
Full registerRisk analysis and information system security policies.
Detection Loom lets a rule be authored, replayed against labelled traces, and its false positives counted before adoption.
The directive binds through national implementing law; transposition varies by Member State and is not tracked here.
A digital operational resilience testing programme including TLPT at least every three years.
The Detection Loom is a testing surface: rules are exercised against labelled benign/attack fixtures and their failures are counted.
TLPT at least every three years; the attestation itself is an authority act.
A regime row on a bench is a mapping, not a compliance result: the chip says how far the connection goes, and the register says where it stops.
Author
Correlate on
Same-second ordering
Correlation window
Sequences per group before alerting
The same rule is evaluated under all three ordering assumptions below. Raising the threshold silences noisy groups and hides real ones — both counts move. The window is the maximum invoke→export age that can pair; the package declares it as the correlation's timespan.
draft Sigma package — validate against your telemetry
title: Export-after-invoke on the same object
status: experimental
description: An invoke action followed within 300s by an export action on the same object, grouped by actor and object. Exported from the Security Laboratory's synthetic corpus; window and grouping are validated against the rule that was scored.
author: Security Laboratory (synthetic corpus)
correlation:
type: temporal_ordered
rules:
- security_lab_invoke
- security_lab_export
condition:
gte: 2
group-by:
- actor
- object
timespan: 300s
falsepositives:
- Benign hand-offs where the same actor invokes and exports the same object by design.
---
title: 'Lab: an invoke action was recorded'
name: security_lab_invoke
status: experimental
logsource:
product: poly_chat
service: audit
definition: 'Bound to the laboratory corpus: requires actor, action, object, and spans two event sources (app-audit, mcp-proxy) that a real deployment must map separately. Bind your own schema before import.'
detection:
invoke:
action: invoke
no_object:
object: null
condition: invoke and not no_object
---
title: 'Lab: an export action was recorded'
name: security_lab_export
status: experimental
logsource:
product: poly_chat
service: audit
definition: 'Bound to the laboratory corpus: requires actor, action, object, and spans two event sources (app-audit, mcp-proxy) that a real deployment must map separately. Bind your own schema before import.'
detection:
export:
action: export
no_object:
object: null
condition: export and not no_object
---
title: 'Lab: events that cannot join the export-after-invoke correlation'
name: security_lab_uncorrelatable_event
status: experimental
logsource:
product: poly_chat
service: audit
definition: Selects events whose object is null — the exact predicate the evaluator buckets as gaps.
detection:
missing_object:
object: null
condition: missing_object
Hunt question
Which actor invokes and exports the same object within 300s — and which of those hand-offs are benign by design?
1 event(s) cannot join any correlation — the package declares the null-object predicate; the query runs only once bound to a real log source.
True pos.
4
False pos.
2
False neg.
1
True neg.
6
Gaps
1
1 attack-labelled — no alert, outside the FN count — the rule did not evaluate them
Alerting sequences
3
Below threshold
0
Attack+attack
2
Benign+benign
1
Mixed
0
These count sequences, not traces; a sequence contributes two trace verdicts to the table above, which remains the register's unit.
Computed as run:demo:detection:d7fe14e9 — nothing is kept until you pin it.
This corpus, this evaluator, these four assumptions
Outside the counts — an event this rule could not evaluate
1 of 14 traces could not be correlated. The gap is counted here and nowhere else — it is not a false negative, and it does not enter the recall denominator.
Finding 04's sentence, rendered from this run — every clause checkable
On a synthetic corpus of 14 labelled traces (6 attack-labelled, 8 benign, digest sha256:a45e8e4399e6f03ba5312e0ee0d1bbbdd5f3bdf2e9b3fb1de6548fc4337615cb), evaluated by `detection-evaluate@2` under actor correlation, input ordering, a 300-second window and threshold 1, rule `rule-export-after-invoke` produced 4 true positives, 2 false positives, 1 false negative and 6 true negatives, with 1 attack-labelled event that could not be evaluated because its source emitted no `object` field. Recall over the 5 evaluable attack traces is 4/5 (Wilson 95% CI 37.6%–96.4%); precision within the corpus is 4/6 at trace level and 2/3 at sequence level. This is a statement about this corpus and this evaluator only. It is not a production false-positive rate, not an estimate of recall on real telemetry, and not a coverage claim.
The benign side also ran against a generated workload of 34 rows (security-lab-benign-workload@1), which produced 6 false positives.
Each with the unit it counts and an interval, not a percentage
Denominator excludes 1 attack-labelled gap event this rule could not evaluate — 4/5, never 4/6.
A flawless 14-trace bench cannot rule out a broken rule.
This is a statement about this corpus and this evaluator only. It is not a production false-positive rate, not an estimate of recall on real telemetry, and not a coverage claim.
The same evidence, three verdicts
| Assumption | TP | FP | FN | TN | Gap | Flips |
|---|---|---|---|---|---|---|
| Input orderapplied | 4 | 2 | 1 | 6 | 1 | — |
| App audit first | 2 | 2 | 3 | 6 | 1 | … |
| MCP proxy first | 4 | 2 | 1 | 6 | 1 | … |
T06 and T07 land in the same second from different sources. Which one is trusted decides whether the pair forms a sequence at all — the table is why the bench does not report a single verdict. Flips names the traces whose verdict would change against the applied assumption: two rows can agree on every total and still disagree about which event they misread.
14 labelled events
u-lea · read · doc-101app-audit
group actor:u-lea has 1 sequence(s), below threshold 1
u-lea · invoke · doc-101app-audit
completed invoke→export sequence on doc-101 (group actor:u-lea) — Legitimate export workflow: invoke then export.
u-lea · export · doc-101app-audit
completed invoke→export sequence on doc-101 (group actor:u-lea)
agent-7 · invoke · doc-202mcp-proxy
completed invoke→export sequence on doc-202 (group actor:agent-7)
agent-7 · export · doc-202mcp-proxy
completed invoke→export sequence on doc-202 (group actor:agent-7)
agent-7 · invoke · repo-9mcp-proxy
completed invoke→export sequence on repo-9 (group actor:agent-7) — Same-second batch with T06; source trust decides the sequence.
agent-7 · export · repo-9app-audit
attempted invoke→export sequence on repo-9 (group actor:agent-7)
u-sam · read · crm-55app-audit
no completed invoke→export sequence in group actor:u-sam
u-sam · export · crm-55app-audit
no completed invoke→export sequence in group actor:u-sam — Human hand-off export; shares object crm-55 with T12.
agent-7 · read · doc-101mcp-proxy
group actor:agent-7 has 2 sequence(s), below threshold 1
agent-7 · invoke · no object fieldmcp-proxy
object field absent — this source cannot be correlated — object field absent — source gap; cannot be correlated.
agent-7 · invoke · crm-55mcp-proxy
group actor:agent-7 has 2 sequence(s), below threshold 1 — Cross-identity hand-off: agent invokes, human exports.
u-lea · approve · repo-9app-audit
group actor:u-lea has 1 sequence(s), below threshold 1
agent-7 · export · doc-101mcp-proxy
group actor:agent-7 has 2 sequence(s), below threshold 1 — No preceding invoke on this object by this actor.
3 matched invoke→export sequence(s) under the applied rule.