Skip to content

Files Subsystem

The Files subsystem stores scan-generated binary artifacts in S3-compatible object storage and stores only metadata in PostgreSQL. Local development uses MinIO, but the backend treats it as S3-compatible storage.

Architecture

Frontend Files page
  -> API file endpoints
  -> PostgreSQL files metadata
  -> S3-compatible object storage / MinIO

Worker
  -> PluginResult.Artifacts
  -> Files service
  -> S3-compatible object storage / MinIO

Plugins do not write to PostgreSQL or S3 directly. A plugin returns artifact metadata and local temporary paths in PluginResult.Artifacts; the worker uploads those artifacts through internal/files and then removes the temporary files.

Metadata Table

The files table stores:

  • organization, scan, scan job, asset, vulnerability references
  • file type and source plugin/system source
  • original file name
  • S3 bucket and object key
  • content type, size, SHA-256
  • JSON metadata
  • creator and creation time

Binary file content is never stored in PostgreSQL.

File Types

Supported initial file types are:

Type Purpose
screenshot Web screenshots from httpx_screenshot
raw_output Raw scan output artifacts
report Generated report files
evidence Evidence/proof files
artifact Generic plugin artifact
other Fallback type

Object Keys

Scan-linked files use:

organizations/{org_id}/scans/{scan_id}/{file_type}/{uuid}-{safe_filename}

Non-scan files use:

organizations/{org_id}/files/{file_type}/{uuid}-{safe_filename}

File names are sanitized to avoid path traversal and unsafe object names.

Configuration

Backend storage config:

storage:
  provider: s3
  endpoint: http://minio:9000
  region: us-east-1
  bucket: easm-files
  access_key: easm
  secret_key: easm-password
  use_ssl: false
  public_base_url: http://localhost:9000

endpoint is used by API/worker containers for bucket creation, upload, delete, and internal object access. public_base_url is used to create a separate presigning client for browser-visible download URLs in local Docker Compose. Presigned URLs must be signed with the same host the browser will open; do not sign with minio:9000 and then rewrite to localhost:9000, because SigV4 includes the host in the signature. Do not put MinIO/S3 credentials in the frontend.

The API and worker call EnsureBucket during startup. Buckets are private; downloads use short-lived presigned URLs.

API

Endpoint Purpose
GET /api/v1/organizations/{org_id}/files List organization files
GET /api/v1/files/{file_id} Get file metadata
GET /api/v1/files/{file_id}/download Return a 5 minute presigned URL
DELETE /api/v1/files/{file_id} Delete object and metadata; admin only

List filters: file_type, scan_id, asset_id, source, limit, offset.

RBAC

All file access is organization-scoped. For file_id routes, the backend resolves organization_id from the file metadata before returning metadata or download URLs.

  • admin: read all organizations, delete files
  • hacker: read assigned organizations
  • client: read assigned organizations

Screenshot Collection

The httpx_screenshot plugin runs httpx in screenshot mode and returns screenshot artifacts. The worker uploads them as file_type=screenshot, source=httpx_screenshot, associates them with the scan and scan job, and links to the URL asset when resolvable.

Chromium dependency

Screenshot capture requires Chromium in the worker image. The local Docker worker image installs Alpine chromium plus runtime font/NSS packages and the plugin passes httpx -system-chrome so Chromium is not downloaded during scan execution. If Chromium is missing, the plugin fails fast with chromium browser not found in worker image.

Screenshot scan profile:

subfinder -> dnsx -> httpx -> httpx_screenshot

Run this profile to populate the Files page with screenshots.

Local Testing

Start the stack:

docker compose up -d --build

MinIO API: http://localhost:9000 MinIO console: http://localhost:9001

Credentials in local development:

user: easm
password: easm-password

The easm-files bucket is created by API/worker startup if it does not exist.

Presigned URL Hostnames

For Docker Compose, use separate internal and public endpoints:

storage:
  endpoint: http://minio:9000
  public_base_url: http://localhost:9000

The backend signs download URLs with public_base_url, so browser requests to localhost:9000 match the signed host and MinIO does not return SignatureDoesNotMatch.