Skip to content
Integrations
Skip navigation

Data Reference

The four Whisper custom tables with their full column contracts, and the five ingestion pipelines that fill them.

Published Last updated

On this page (8)

Data Reference Documentation

The solution writes everything to four custom tables in your Log Analytics workspace. Workbooks, analytics rules, and hunting queries read from them — and so can any KQL of your own.

What writes what

Only the five ingestion pipelines write to these tables. The ten playbooks call the Whisper API and post their result as a comment on the incident; none of them writes a row.

TableWritten byWhen it runs
WhisperThreatIntel_CLWhisper-EnrichmentPipelineOn incident creation, once you have wired the automation rule
WhisperInfraContext_CLWhisper-InfraChainPipelineOn incident creation, once you have wired the automation rule
WhisperHistory_CLWhisper-WhoisHistoryPipeline (domain rows) and Whisper-BgpHistoryPipeline (IP rows)Daily, and only for the entries on your watchlists
WhisperASNReputation_CLWhisper-AsnReputationPollerHourly, and only for your monitored ASNs

Both incident-triggered pipelines are a precondition, not an option — see Configuration.

Column contracts

Every column below comes from the shipped table schema and its writer's field mapping in solution 3.0.0. Guarantee is one of:

  • guaranteed — present on every row the writer produces.
  • conditional — present only when the graph had that data for the indicator.
  • declared, never emitted — the column exists in the table and nothing in the solution writes it.

A row appears at all only when the pipeline reaches its ingestion step; a failed API call is logged and the row is skipped, so a missing indicator is not a clean indicator.

Read coverage before band. Only known-clean — coverage: known-clean. In coverage, no malicious evidence. licenses the word "clean"; no-data — coverage: no-data. Not in coverage. This is not a verdict — nothing was looked at. means unknown, which is a different thing again; malicious-evidenced — coverage: malicious-evidenced. In coverage, with positive evidence of malice. and ambiguous — coverage: ambiguous. In coverage, and the evidence points both ways. mean there is evidence, whatever the band says. Full contract: Coverage — what we looked at.

WhisperThreatIntel_CL

Written by Whisper-EnrichmentPipeline from CALL whisper.explain().

ColumnTypeGuaranteeWhen absentWhat a rule must do
indicatorstringguaranteedneverJoin on this, not on the entity name — it is the raw entity value from the incident
indicatorTypestringguaranteedneverDo not filter on it. Status: known issue in 3.0.0 — the pipeline derives it as "contains a dot ⇒ domain", so every IPv4 address is written as domain. Filter on the shape of indicator instead
threatScorerealconditionalexplain() returned no row for the indicatorTreat null as not covered, never as zero
threatLevelstringconditionalas aboveSame. The level vocabulary is INFO, LOW, MEDIUM, HIGH, CRITICAL
isThreatboolconditionalas aboveUse == true, not != false — null is not false here
isC2, isMalware, isPhishing, isTor, isAnonymizer, isSpam, isBruteforce, isScannerboolconditionalas aboveOne flag being null does not mean the others are complete; test each flag you read
threatSourcesintconditionalas aboveNull and 0 both mean "no feed lists it", which is not the same as "clean"
feedNamesstringconditionalno feed lists the indicatorComma-joined, empty string when the list is empty. split() before matching
explanationstringconditionalexplain() produced noneDisplay only. Never parse it
factorsstringconditionalas aboveA JSON array stored as a string. parse_json() before indexing into it
lastSeendatetimeguaranteedneverThis is the write time, not the last time a feed saw the indicator — the pipeline sets it to utcNow(). Do not use it to age a listing
TimeGenerateddatetimeguaranteedneverThe column every ago() window should use

WhisperInfraContext_CL

Written by Whisper-InfraChainPipeline.

ColumnTypeGuaranteeWhen absentWhat a rule must do
indicatorstringguaranteedneverThe incident entity the chain was built from
indicatorTypestringguaranteedneverip, domain or unknown. This pipeline does parse the shape, so it is safe to filter here
ipAddresses, prefixes, asns, asnNames, cities, countriesstringconditionalthe traversal found nothing at that layerComma-joined lists, empty string when empty. split() and mv-expand; an empty string expands to one empty row, so filter isnotempty() after expanding
registrar, registrant, nameserversstringconditionalno WHOIS answer, or the indicator is an IPWritten as an empty string, never null — isnotempty() is the correct test, isnotnull() is not
domainAgeintguaranteedneverDo not use it. In 3.0.0 the value is always -1: nothing computes a registration age yet, and every rule and hunt that filters domainAge >= 0 returns zero rows. Treat the field as unavailable until a release notes otherwise
cohostedCountintguaranteedneverWritten as 0 when the co-hosting query returns no rows, so 0 means "not measured or genuinely zero" and cannot tell them apart
dnssecAlgorithmstringdeclared, never emittedalwaysNothing in the solution writes this column. Any panel or rule reading it reports 0 % coverage — including the SPF and DNSSEC Coverage panel of the External Attack Surface workbook
spfIncludesstringdeclared, never emittedalwaysAs above
bgpStatusstringguaranteedneverThe pipeline writes the constant "active" for every row. It carries no signal; do not branch on it
TimeGenerateddatetimeguaranteedneverThe column every ago() window should use

WhisperHistory_CL

One table, two shapes. Whisper-WhoisHistoryPipeline writes the WHOIS columns for domains; Whisper-BgpHistoryPipeline writes the BGP columns for IPs. Neither writes the other's columns, so every row has one half of this table empty.

ColumnTypeGuaranteeWhen absentWhat a rule must do
indicatorstringguaranteedneverThe watchlist entry the snapshot belongs to
indicatorTypestringguaranteedneverLiteral domain on WHOIS rows, literal ip on BGP rows. Filter on it first — it is what separates the two shapes
snapshotDatedatetimeguaranteedneverWhen the state was observed. Order by this, not by TimeGenerated
registrar, registrant, country, nameServersstringconditionalon every BGP row, and on a WHOIS row the registry redactedNote the capital S in nameServers — WhisperInfraContext_CL spells the same idea nameservers
createDate, updateDate, expiryDatedatetimeconditionalon every BGP row, and when WHOIS omits the dateAn absent date is written as an empty value and lands as null in a datetime column, so guard with isnotnull() before any datetime_diff()
bgpOrigin, bgpPrefixstringconditionalon every WHOIS row
bgpVisibilityrealconditionalon every WHOIS row
TimeGenerateddatetimeguaranteedneverIngestion time, not observation time

Both writers are scheduled and read a watchlist. An empty watchlist means an empty table, and an empty table is indistinguishable from an indicator with no history.

WhisperASNReputation_CL

Written by Whisper-AsnReputationPoller, hourly, for the ASNs named in monitoredAsns.

ColumnTypeGuaranteeWhen absentWhat a rule must do
asnstringguaranteedneverThe polled ASN. Only monitored ASNs are ever present — a join against this table silently drops every ASN you have not listed
asnNamestringconditionalthe graph has no name for the ASN
reputationScore, reputationLevelreal, stringconditionalexplain() returned no rowNull is not polled or not covered, not good
maxThreatScore, avgThreatScorerealconditionalas aboveAn inner join on these silently drops unpolled ASNs; use a left join if the absence matters
hasThreateningPrefixesboolconditionalas above
countrystringconditionalthe graph has no registration country
prefixCount, peerCountintconditionalas above
TimeGenerateddatetimeguaranteedneverThe column every ago() window should use

Watch for a table that stops receiving rows

A union over the four tables tells you what arrived. It cannot tell you about a table that has never received a row, because a wildcard union matches nothing for a table with no data — and a table that is silently missing reads exactly like an indicator that is genuinely clean.

Name the four tables and join them back with rightouter, so an absent table appears as a row rather than as nothing:

kusto
let Expected = datatable(TableName: string)
[
    "WhisperThreatIntel_CL", "WhisperInfraContext_CL",
    "WhisperHistory_CL", "WhisperASNReputation_CL"
];
union isfuzzy=true withsource = TableName
    WhisperThreatIntel_CL, WhisperInfraContext_CL,
    WhisperHistory_CL, WhisperASNReputation_CL
| summarize Rows = count(), Latest = max(TimeGenerated) by TableName
| join kind=rightouter (Expected) on TableName
| project Table = TableName1, Rows = coalesce(Rows, long(0)), Latest
| where Rows == 0 or Latest < ago(24h)

The rightouter is the whole point. An inner join drops a table that has never been written, which is the same mistake as reading no-data as clean, one layer down. The last line keeps only what is worth acting on — a table with no rows at all, and a table whose newest row is over a day old — so this query returning nothing is the answer you want.

Ingestion pipelines

Five Logic Apps feed the tables. Deployed name is what you will see in the Azure portal; repo file is the template it is deployed from, which is the name to quote in a support request.

Deployed nameRepo fileTriggerPurpose
Whisper-EnrichmentPipelineWhisperEnrichmentPipeline.jsonIncident (wire via automation rule)explain() enrichment of incident entities into WhisperThreatIntel_CL
Whisper-InfraChainPipelineWhisperInfraChainPipeline.jsonIncident (wire via automation rule)Infrastructure-chain context into WhisperInfraContext_CL
Whisper-WhoisHistoryPipelineWhisperWhoisHistoryPipeline.jsonDaily scheduleWHOIS snapshots for your domain watchlist into WhisperHistory_CL
Whisper-BgpHistoryPipelineWhisperBgpHistoryPipeline.jsonDaily scheduleBGP routing history for your IP watchlist into WhisperHistory_CL
Whisper-AsnReputationPollerWhisperAsnReputationPoller.jsonHourly scheduleReputation refresh for your monitored ASNs into WhisperASNReputation_CL

Four of the five take their deployed name from a template parameter, so a deployment that overrode it will show something else; Whisper-InfraChainPipeline is fixed in the template and is always that string.

The daily pipelines ship with empty watchlists and collect nothing until you set them — see Configuration.