Graph-aware LLM for Windows logons — compress first, then let the agent think.

Shusei Tomonaga (JPCERT/CC) tackles a problem every Windows blue team knows: attackers who abuse legitimate authentication leave almost no malware to signature, and the evidence hides inside millions of noisy logon events. His answer is a graph-first, LLM-second pipeline. Collapse raw Event Log lines into a small user↔host graph, rank suspicious nodes with PageRank plus a Hidden Markov Model, then let a guarded, closed-loop LLM agent investigate with allow-listed Cypher queries. On a public 12.8 GB dataset it scored precision 0.90 and recall 0.931, and on a real Ivanti-initiated intrusion it analysed 821,272 events in 195 seconds. This is a defensive talk, so I can go deep on the design.

Post 18 of my Black Hat Asia 2026 series. The core lesson: do not feed an LLM your logs. Feed it a structure computed from your logs, and box it in with guardrails.

1. The problem: logs are abundant, insight is rare

Windows Event Logs were built for auditing and troubleshooting, not intrusion detection. A mid-size Active Directory domain emits millions of authentication events a day. Modern intrusions are mostly living off the land: the attacker steals a credential or ticket and then logs in like anyone else. There is no malicious binary to hash. Every single event looks legitimate; only the pattern is wrong.

Signature and rule-based detection (Sigma and friends) covers known patterns well but is brittle against variation. Anomaly detection on raw features drowns in noise. Tomonaga's goal is narrower and more honest: find compromised accounts and high-risk logon patterns after the initial foothold, focusing on lateral movement and credential abuse, and hand analysts a short, evidenced list.

2. Why "just give the logs to an LLM" fails

The deck does the arithmetic, and it is sobering. A 600 MB Security log with about 1.23 million events costs roughly 469 tokens per event once serialised. That is about 577 million input tokens. At a typical frontier-model input price of around $1.75 per million tokens, one analysis run costs about $1,000 in input alone. With a 400k-token context window you would also need over a thousand chunked requests, and the model would never see the whole picture at once, which is exactly what lateral-movement detection needs.

3. Architecture: four stages

4. Which events matter

Windows has around 18 logon-related Event IDs. Tomonaga narrows to six with the strongest authentication signal: 4624 (successful logon), 4625 (failed logon), 4768 (Kerberos TGT request), 4769 (Kerberos service-ticket request), 4776 (NTLM credential validation) and 4672 (special privileges assigned). Others such as logoff and ticket renewal add volume but little detection value. The fields kept per event are the ones that describe who authenticated to what, how: account, host or IP, logon type, authentication package, service name, ticket encryption type and status.

5. The graph layer

Accounts and hosts become nodes; each logon event becomes an edge labelled with its Event ID and attributes. Then comes the key compression step: raw events are collapsed into unique (user, host) edges per time window with counts and timing statistics. In the deck's example, 50 MB of raw events becomes about 1 MB of graph. Token cost now scales with the number of nodes and edges, not the number of events.

Graphs also make the attack shape visible. Lateral movement is fan-out: one account suddenly touching many hosts. A single log line cannot show that; a node's degree does. The graph is stored in Neo4j (the implementation builds on JPCERT's own LogonTracer), and each subgraph can be serialised to JSON so the LLM gets structure plus attributes together: graphs reveal structure, LLMs understand language.

6. Ranking with PageRank and an HMM

PageRank for fan-out

PageRank was invented to rank web pages: important pages are linked to by many pages, especially by other important ones. Applied to a logon graph, an account that reaches many hosts, or a host reached by many privileged accounts, scores high. That turns "looks like lateral movement" into a single scalable number, used to limit the scope the LLM must analyse.

HMM for missing transitions

A Hidden Markov Model models hidden states that emit observable events. Here the hidden states are the logon stages: not logged on, logon attempt, logged on. Normal Kerberos behaviour follows a recognisable rhythm: a TGT request, then service tickets, then the logon. A pass-the-ticket sequence tends to skip the attempt stage because the attacker replays a ticket obtained elsewhere. The HMM flags those missing transitions, and the score is folded into the PageRank ranking. Together they produce a prioritised list of suspicious users and hosts using structure and sequence, not signatures.

7. The closed-loop guarded agent

The LLM is not asked "is this malicious?" once. It runs a bounded investigation loop: Plan → Generate → Execute → Analyze → Decide.

1. Plan: given the ranked list, known Windows attack patterns and risk weights, pick a hypothesis, such as privilege escalation or Kerberos weak-crypto exposure.
2. Generate: write a Cypher query from vetted, parameterised templates.
3. Execute: run it with hard row limits and time windows; only compact rows come back.
4. Analyze and Decide: label the finding, give a risk level, cite evidence, and either continue with a new focus or stop. The loop is capped at ten iterations.
5. Report: summary, risk level, threat types, and recommendations.

This is the same defensive pattern seen in the DFIR-agent prompt-injection talk: constrain what the model can do and demand a typed, checkable output. Note the difference in threat model, though. Here the LLM queries a database of pre-aggregated logs, but attacker-controlled strings (usernames, hostnames) still flow into its context, so the strict output schema and template allow-list matter.

8. Evaluation

Jaccard index is simply overlap divided by union: if run one flags accounts A, B, C and run two flags A, B, D, the score is 2/4 = 0.5. A mean of 0.806 means the agent mostly names the same accounts each time, a meaningful result given LLM non-determinism, though not perfect. Using Sigma as ground truth also means the benchmark measures agreement with rules, not with an independent adjudicator; the author is explicit that this is a baseline comparison.

9. The real incident

The case involved exploitation of an Ivanti Connect Secure appliance followed by predominantly living-off-the-land activity, consistent with a known espionage cluster. Using Domain Controller Security logs only, the agent worked through about ten steps: a privilege-escalation scan, a Kerberos weak-encryption check, a multi-host logon check, then a re-check and summary. It surfaced two findings: a Kerberoasting exposure (a service ticket requested with RC4 for an application server) and multiple privileged accounts making network logons to many hosts, a lateral-movement hotspot. Real accounts were anonymised in the slides. The value to responders was speed: a triaged, evidenced starting point in minutes rather than a day of manual pivoting.

10. From findings to Sigma rules

Detection should lead to prevention. A second agent turns confirmed evidence into Sigma rules so the lesson becomes durable detection. The system prompt (under 1,500 tokens) is a good example of prompt discipline: evidence-first (never invent fields or values), a strict field whitelist per Event ID, normalisation tables for ticket encryption types, a fixed logsource, and a final self-check that the YAML parses and uses only evidenced fields. Again the pattern is constrain the model with a schema, then verify.

11. Failure cases and evasion

• Low-and-slow movement may not stand out structurally in a graph.
• Shared admin accounts and jump hosts legitimately inflate centrality and cause false positives.
• Incomplete telemetry: if the right Event IDs are not audited or logs are cleared, confidence drops.
• Evasion: account hopping and blending into maintenance windows can defeat centrality signals.

12. What to copy in your own SOC

1. Enable the six events on domain controllers and key servers, including advanced Kerberos auditing, and forward them centrally.
2. Aggregate before you reason. Build (user, host, window) edges with counts. Whether you use Neo4j or a dataframe, shrink the data first.
3. Score structure cheaply with degree, PageRank or similar, and sequence checks for missing Kerberos steps.
4. Put an LLM last and give it templates, limits, an abstain path, and a schema.
5. Log the agent's own reasoning so you can audit and improve it.
6. Pair it with hygiene: disable RC4 where possible, tier admin accounts, and restrict where privileged accounts may log on. The graph will then show far fewer legitimate fan-outs.

Takeaways

• Raw-log LLM analysis is too expensive and too context-limited; compress to a graph first.
• Fan-out and skipped Kerberos stages are visible to PageRank and an HMM where single events are not.
• A closed-loop agent with template-only queries, iteration caps, evidence requirements and an abstain policy keeps the LLM honest.
• Reported results: precision 0.90, recall 0.931, Jaccard 0.806; a real 821k-event incident triaged in 195 s.
• Known limits: low-and-slow movement, shared admin accounts, telemetry gaps. Keep a human in the loop.

References & further reading

• Black Hat Asia 2026 — this talk's official page — abstract, speaker bio, and slides/video when posted.
• Official talk recording on YouTube — Black Hat channel.
• Black Hat Asia 2026 — full Briefings schedule — every talk in the conference.
• JPCERT/CC LogonTracer — the open-source tool the pipeline builds on.
• SigmaHQ — the rule baseline used for ground truth.
• JPCERT/CC blog — the author's team publications.
• Neo4j — graph store and Cypher query language.