Asset Observed State

Implementation status: Phase 3 read APIs and Asset Detail wiring are implemented. Phase 4A adds production DNSX enrichment for canonical Domain/Subdomain DNS observed state.

Ownership Boundary

Asset state is split into two ownership domains:

Domain	Storage	Owner	Purpose
Managed Context	assets.managed_context	admin, hacker, operator	Human-maintained context such as description, owner, team, environment, and tags.
Observed State	assets.metadata.asset_info	scanners and plugins	Canonical technical state observed by tools.

Scanner plugins must not write Managed Context. User edits must not be stored as scanner-observed asset_info.

Legacy metadata keys can remain temporarily for compatibility. New scanner-owned observed-state work should write canonical metadata.asset_info.

Canonical Namespace

Observed state is stored under:

{
  "asset_info": {}
}

inside assets.metadata.

The namespace is partial by design. A plugin may emit only the fields it actually observed. Omitted fields are not interpreted as removal.

Per-Asset Schema

Domain / Subdomain

{
  "dns": {
    "a": ["203.0.113.10"],
    "aaaa": ["2001:db8::10"],
    "cname": ["target.example.com"],
    "ns": ["ns1.example.com"],
    "mx": [{ "host": "mail.example.com", "priority": 10 }]
  },
  "whois": {
    "registrar": "Example Registrar",
    "created_at": "2020-01-01T00:00:00Z",
    "updated_at": "2026-01-01T00:00:00Z",
    "expires_at": "2027-01-01T00:00:00Z",
    "status": ["clientTransferProhibited"],
    "nameservers": ["ns1.example.com"]
  },
  "asn": {
    "number": 64500,
    "name": "EXAMPLE-NET"
  }
}

Production writers currently populate DNS A, AAAA, CNAME, NS, and MX where DNSX observes them. WHOIS/RDAP and Domain/Subdomain ASN fields are canonical but unsupported until a production producer is added.

Field	Canonical path	Producer	Merge semantics	Status
A records	dns.a	DNSX	replace_set for this record type only	PRODUCTION
AAAA records	dns.aaaa	DNSX	replace_set for this record type only	PRODUCTION
CNAME records	dns.cname	DNSX	replace_set for this record type only	PRODUCTION
NS records	dns.ns	DNSX	replace_set for this record type only	PRODUCTION
MX records	dns.mx	DNSX	replace_set for this record type only; elements are {host, priority}	PRODUCTION
Registrar	whois.registrar	none	reserved	UNSUPPORTED
WHOIS created time	whois.created_at	none	reserved	UNSUPPORTED
WHOIS updated time	whois.updated_at	none	reserved	UNSUPPORTED
WHOIS expiration time	whois.expires_at	none	reserved	UNSUPPORTED
WHOIS nameservers	whois.nameservers	none	reserved	UNSUPPORTED
WHOIS statuses	whois.status	none	reserved	UNSUPPORTED
Domain/Subdomain ASN number	asn.number	none	reserved	UNSUPPORTED
Domain/Subdomain ASN name	asn.name	none	reserved	UNSUPPORTED

DNS observed state records facts seen by DNS scanners. Asset Graph remains the topology source for relations such as domain/subdomain -> resolves_to -> ip; DNS records are not new Asset types and do not replace graph edges.

IP

{
  "network": {
    "asn": { "number": 64500, "name": "EXAMPLE-NET" },
    "ptr": ["host.example.com"]
  },
  "os": {
    "name": "Linux",
    "family": "Linux",
    "version": "5.x",
    "accuracy": 98
  }
}

Phase 4B production writers populate PTR through DNSX for IP targets and structured OS fields through Nmap when Nmap XML contains OS guesses/classes. ASN enrichment is canonical but unsupported until a production producer is added.

Field	Canonical path	Producer	Merge semantics	Status
PTR records	network.ptr	dnsx	replace_set when PTR output is observed	PRODUCTION
OS name	os.name	nmap	merge	PRODUCTION
OS family	os.family	nmap	merge when structured OS class is present	PARTIAL
OS version	os.version	nmap	merge when structured OS generation is present	PARTIAL
OS accuracy	os.accuracy	nmap	merge	PRODUCTION
ASN number	network.asn.number	none	reserved	UNSUPPORTED
ASN name	network.asn.name	none	reserved	UNSUPPORTED

Service

{
  "service": {
    "name": "http",
    "product": "nginx",
    "version": "1.24.0",
    "state": "open",
    "transport": "tcp"
  },
  "tls": {
    "observed_version": "TLSv1.3",
    "supported_versions": ["TLSv1.2", "TLSv1.3"]
  }
}

Phase 4B production writers populate Naabu open state/transport, Nmap service fingerprint fields, and TLSX observed negotiated TLS version when TLSX output includes a concrete IP and port. TLS supported-version enumeration is reserved until enumeration behavior is explicitly enabled.

Field	Canonical path	Producer	Merge semantics	Status
Service name	service.name	nmap	merge	PRODUCTION
Product	service.product	nmap	merge	PRODUCTION
Version	service.version	nmap	merge	PRODUCTION
State	service.state	naabu, nmap	merge	PRODUCTION
Transport	service.transport	naabu, nmap	merge	PRODUCTION
TLS observed version	tls.observed_version	tlsx	merge	PRODUCTION
TLS supported versions	tls.supported_versions	none	replace_set when future enumeration is explicit	UNSUPPORTED

WebApp

{
  "http": {
    "server": "nginx",
    "status_code": 200,
    "security_headers": {
      "strict_transport_security": "max-age=31536000",
      "x_content_type_options": "nosniff",
      "x_frame_options": "DENY",
      "referrer_policy": "strict-origin-when-cross-origin",
      "permissions_policy": "geolocation=()"
    },
    "csp": "default-src 'self'",
    "cookies": [
      {
        "name": "session",
        "secure": true,
        "http_only": true,
        "same_site": "Lax"
      }
    ]
  },
  "technologies": [
    { "name": "nginx", "version": "1.24.0" }
  ],
  "fingerprints": {
    "favicon_hash": "123456",
    "page_fuzzy_hash": "reserved"
  },
  "paths": ["/", "/login"]
}

Phase 4C production writers populate HTTPX status/server, selected security headers, CSP, safe cookie flags, technologies, favicon hash, and simhash page fuzzy hash where HTTPX emits them. Katana remains the canonical path producer. Plugins remain independently runnable; HTTPX does not require Katana, and Katana does not require HTTPX.

Field	Canonical path	Producer	Merge semantics	Status
HTTP server	http.server	httpx	merge	PRODUCTION
HTTP status	http.status_code	httpx	merge	PRODUCTION
Strict-Transport-Security	http.security_headers.strict_transport_security	httpx	merge	PRODUCTION
X-Content-Type-Options	http.security_headers.x_content_type_options	httpx	merge	PRODUCTION
X-Frame-Options	http.security_headers.x_frame_options	httpx	merge	PRODUCTION
Referrer-Policy	http.security_headers.referrer_policy	httpx	merge	PRODUCTION
Permissions-Policy	http.security_headers.permissions_policy	httpx	merge	PRODUCTION
CSP	http.csp	httpx	merge; multiple headers are joined deterministically	PRODUCTION
Cookie flags	http.cookies	httpx	replace_set only when Set-Cookie output is present; values are never stored	PRODUCTION
Technologies	technologies	httpx	merge; versioned entries are preserved over versionless observations	PRODUCTION
Favicon hash	fingerprints.favicon_hash	httpx	merge	PRODUCTION
Page fuzzy hash	fingerprints.page_fuzzy_hash	httpx	merge; currently HTTPX simhash	PRODUCTION
Paths	paths	katana	merge; non-authoritative crawl set	PRODUCTION

Certificate

{
  "certificate": {
    "fingerprint_sha256": "hex",
    "serial_number": "hex",
    "subject": {
      "common_name": "example.com",
      "organization": "Example Inc",
      "organizational_unit": "Security",
      "country": "US",
      "locality": "Example City",
      "province": "CA"
    },
    "issuer": {
      "common_name": "Example CA",
      "organization": "Example CA Inc",
      "country": "US"
    },
    "sans": ["example.com", "www.example.com"],
    "validity": {
      "not_before": "2026-01-01T00:00:00Z",
      "not_after": "2027-01-01T00:00:00Z",
      "expired": false
    },
    "signature": { "algorithm": "SHA256-RSA" },
    "public_key": { "algorithm": "RSA", "size": 2048 },
    "extensions": {
      "key_usage": ["Digital Signature"],
      "extended_key_usage": ["TLS Web Server Authentication"],
      "basic_constraints": {},
      "authority_key_identifier": "hex",
      "subject_key_identifier": "hex"
    }
  }
}

Phase 4D production writers populate TLSX SHA-256 fingerprint, serial number, subject common name and organization fields where emitted, issuer common name and organization fields where emitted, SANs, and validity. SANs are treated as an authoritative certificate set when TLSX returns them. Certificate identity remains the SHA-256 fingerprint; certificate replacement normally creates a new Certificate Asset rather than mutating identity.

Field	Canonical path	Producer	Merge semantics	Status
Fingerprint	certificate.fingerprint_sha256	tlsx	merge; identity value	PRODUCTION
Serial number	certificate.serial_number	tlsx	merge	PRODUCTION
Subject common name	certificate.subject.common_name	tlsx	merge	PRODUCTION
Subject organization	certificate.subject.organization	tlsx	merge when emitted	PARTIAL
Subject organizational unit	certificate.subject.organizational_unit	tlsx	merge when emitted	PARTIAL
Subject country/locality/province	certificate.subject.country, .locality, .province	tlsx	merge when emitted	PARTIAL
Issuer common name	certificate.issuer.common_name	tlsx	merge	PRODUCTION
Issuer organization	certificate.issuer.organization	tlsx	merge when emitted	PARTIAL
Issuer organizational unit/country/locality/province	issuer nested fields	tlsx	merge when emitted	PARTIAL
SANs	certificate.sans	tlsx	replace_set; authoritative certificate set	PRODUCTION
Validity	certificate.validity.not_before, .not_after, .expired	tlsx	merge	PRODUCTION
Signature algorithm	certificate.signature.algorithm	none	reserved	UNSUPPORTED
Public key algorithm/size	certificate.public_key.algorithm, .size	none	reserved	UNSUPPORTED
Key usage extensions	certificate.extensions.key_usage, .extended_key_usage	none	reserved	UNSUPPORTED
Basic constraints	certificate.extensions.basic_constraints	none	reserved	UNSUPPORTED
Authority/subject key identifiers	certificate.extensions.authority_key_identifier, .subject_key_identifier	none	reserved	UNSUPPORTED

Patch Contract

Observed-state updates use explicit patch semantics:

Operation	Meaning
merge	Update only the provided scalar/object/set paths. Omitted sibling fields remain unchanged.
replace_set	Replace one targeted set path with a complete authoritative set from the producer.
remove	Explicitly remove one field path or one element from a set path.
observed_paths	Record which paths the producer actually observed. This is provenance for future snapshot/change logic.

Same field path policy is last writer wins. A later patch to asset_info.service.version replaces the previous value of that exact path.

Omitted data is never removal. For example, if Nmap updates asset_info.service.version and does not report asset_info.tls.supported_versions, the TLS state remains unchanged.

Authoritative Replacement

replace_set is used only when a producer declares that the emitted value is the complete current set for one path. It does not replace parent objects or unrelated sibling paths.

Example:

replace_set(asset_info.dns.a, ["203.0.113.10"])

replaces only the A-record set. It does not remove asset_info.dns.aaaa, asset_info.dns.mx, or any other observed state.

Normalization

Set-like fields are normalized before storage:

• trim whitespace;
• drop empty values;
• deduplicate;
• sort deterministically.

V1 caps:

• asset_info.paths: 500 paths;
• asset_info.technologies: 200 technologies.

Paths use normalized path strings only. Phase 1 does not store per-path status codes, content metadata, request payloads, or response bodies.

Provenance

Patch application carries provenance for Scan History:

• organization ID;
• asset ID;
• scan ID when available;
• scan job ID when available;
• source plugin;
• observed timestamp.

Snapshots and changes persist this provenance where available. Manual/operator changes do not create Scan History.

Current Producer Support

Plugin	Canonical paths written
dnsx	asset_info.dns.a, asset_info.dns.aaaa, asset_info.dns.cname, asset_info.dns.ns, asset_info.dns.mx on Domain/Subdomain assets; asset_info.network.ptr on IP assets.
naabu	asset_info.service.state, asset_info.service.transport on Service assets.
nmap	asset_info.os.name, asset_info.os.family, asset_info.os.version, asset_info.os.accuracy on IP assets where present; asset_info.service.name, product, version, state, transport on Service assets.
httpx	asset_info.http.status_code, asset_info.http.server, selected asset_info.http.security_headers, asset_info.http.csp, safe asset_info.http.cookies, asset_info.technologies, and asset_info.fingerprints on WebApp assets.
katana	asset_info.paths on WebApp assets using non-authoritative merge semantics.
tlsx	asset_info.certificate.fingerprint_sha256, serial number, subject/issuer fields where emitted, SANs, validity dates, expired flag on Certificate assets; asset_info.tls.observed_version on Service assets when IP and port are known.

Scan Snapshots

asset_scan_snapshots stores the full canonical metadata.asset_info state after an observed-state patch is applied. The stored snapshot is not the plugin's partial payload.

Example:

previous current state:
  service.version = "1.22"
  tls.supported_versions = ["TLSv1.2"]

nmap patch:
  service.version = "1.24"

new snapshot:
  service.version = "1.24"
  tls.supported_versions = ["TLSv1.2"]

Snapshots are created when a canonical observed-state patch reaches an Asset. Managed Context, lifecycle state, criticality, and generic legacy metadata updates do not create scan snapshots.

Scan Changes

asset_scan_changes stores materialized machine-readable differences caused by one observation.

Change types:

Change type	Meaning
observed	First stored observation for a field or set element.
added	Later observation added a scalar or set element that was previously absent.
removed	Authoritative replacement or explicit removal removed a field or set element.
changed	A scalar value changed.

The first snapshot is not a silent baseline. Initial scalar values and set elements generate observed changes.

Scalar example:

service.version: "1.22" -> "1.24"
change_type: changed
field_path: service.version
old_value: "1.22"
new_value: "1.24"

Set example:

previous paths: ["/", "/login"]
current paths:  ["/", "/admin"]

removed:
  field_path: paths
  old_value: "/login"

added:
  field_path: paths
  new_value: "/admin"

Set diffing is element-level. Ordering-only changes produce no scan changes.

Patch-Aware Diffing

The scan-change engine is patch-aware. It does not diff the entire previous snapshot against the entire new snapshot after every plugin run.

Rules:

• merge can create observed, added, or changed only for paths present in the patch.
• replace_set can create element-level observed, added, and removed for that set path.
• remove can create removed for the explicitly removed field or set element.
• omitted fields are never removals.

This prevents false removals when different plugins own different observed-state paths.

Duplicate Observations

A canonical observed-state patch can create a new snapshot even when values are identical, preserving observation provenance. Identical values do not create duplicate asset_scan_changes.

Retention

No automatic retention is implemented yet. Existing normalization caps still apply before snapshot creation:

• paths: 500
• technologies: 200

Phase 2 also caps materialized changes per snapshot at 1000 to avoid pathological event volume while preserving the bounded canonical snapshot state.

System Changes, Scan Changes, and Exposure Changes

The three history domains are separate:

Domain	Storage	Meaning
System Changes	asset_history	Per-Asset lifecycle/operator timeline: discovery, manual creation, approval, disabled/stale, criticality, Managed Context.
Scan Changes	asset_scan_snapshots, asset_scan_changes	Fine-grained scanner-observed technical state over time.
Exposure Changes	exposure_changes	Organization-level attack-surface/security activity feed.

Scan Changes are not duplicated into Exposure Changes. Exposure Changes remain a higher-level feed for discovery/security events.

Graph topology also remains separate. Graph edges such as resolves_to, exposes, serves, and has_certificate remain canonical topology. Observed state may preserve DNS records, service banners, HTTP details, and certificate fields for technical-state history without replacing graph semantics.

Backup

Scan snapshots and scan changes are durable product data. They are included in backup/export when scan history is requested.

Phase 3: Read APIs and Asset Detail UI

Asset Detail now consumes Scan Changes without changing the storage contract:

• top change graph with exactly two categories: System Changes and Scan Changes;
• History tab toggle: System / Scan, defaulting to System;
• Recent Changes combines a small number of latest System and Scan changes;
• Scan Changes are rendered from asset_scan_changes, not snapshots;
• scan snapshots remain backend historical truth and are not directly exposed as a UI browser.

Read endpoints:

GET /api/v1/assets/{asset_id}/scan-changes
GET /api/v1/assets/{asset_id}/change-summary

scan-changes returns field-level changes sorted newest-first. change-summary returns day buckets containing collapsed System Change and Scan Change counts for the selected range.