The catalog

Hand-written archetypes and declarative profiles — the two ways Proteus knows how a workload behaves, and how to add a third AWS service.

The catalog knows how one kind of workload behaves: what resource attributes identify it, what metrics it reports, what spans it produces, what it logs.

There are two mechanisms, and the distinction matters when you extend it.

The two mechanisms#

Archetypes are hand-written generators. Twelve of them, one Python module each, for the workloads whose behaviour is idiosyncratic enough to be worth writing out: alb, nginx-web, app-service, lambda, rds-postgres, elasticache, kafka, opensearch, eks-workload, ecs-workload, bedrock-agent, browser-rum.

Profiles are data. A WorkloadProfile describes a workload as a list of metric specifications, each naming its shape, its unit, the AWS metric it corresponds to, and which FaultEffect field moves it. One generator (proteus/catalog/generic.py) renders any of them.

Verified on this build:

archetypes registered: 13 (12 workloads + custom)
profiles registered:   56
metric specs:          393
distinct aws_namespace values across profiles: 56
log specs:             80
metric specs with no confirmed aws_metric: 20

Twelve archetypes plus 56 profiles is 68 placeable workload types. The thirteenth registered archetype is CUSTOM, which is the delegation mechanism rather than a workload — it is deliberately absent from the archetypes array in GET /api/catalog, because probing it with no kind yields an empty entry.

Why both? Hand-writing does not scale to the hundred-odd AWS services that report to CloudWatch, and it is not what "let me place anything" needs. But the twelve archetypes carry behaviour a data format cannot express: a Bedrock agent's span tree, a browser app's RUM events, a Postgres instance's relationship to its connection pool. So the archetypes stayed, and everything else became data.

ArchetypeGenerator#

proteus/catalog/base.py is frozen. It holds the protocol:

@runtime_checkable
class ArchetypeGenerator(Protocol):
    archetype: Archetype

    def resource(self, service: Service, model: WorldModel) -> Resource: ...
    def metrics(self, tick: Tick) -> Iterator[MetricPoint]: ...
    def spans(self, tick: Tick) -> Iterator[SpanRecord]: ...
    def logs(self, tick: Tick) -> Iterator[LogRecord]: ...
    def rum(self, tick: Tick) -> Iterator[RumEvent]: ...

Every method may yield nothing. A managed RDS instance produces no RUM events; a browser app produces no container metrics. Returning an empty iterator is correct, not a stub — inherit NullSignals and override only the signals your archetype has.

resource() must include service.name. It should include every conventional OTel key that applies: the resource is the record's statement of its own identity, and those keys are load-bearing rather than decoration.

The registry#

register(generator)          # raises on a duplicate archetype
generator_for(archetype)     # or KeyError
registered()                 # a copy of the registry

Importing a module is the only thing that populates the registry. render/engine.py::load_catalog imports every public module under proteus.catalog — the names in CATALOG_MODULES plus anything else it discovers — and returns "<module>: <reason>" for each one that failed, so one broken module never prevents the others from loading. It catches every exception, not just ImportError, because a half-written module raises whatever its syntax happens to raise and that must not abort a render.

The capped-counter rule#

This is the most important paragraph in base.py, and it is in the docstring of metrics():

A capped archetype must scale its SUM counters from the sample back up to the modelled rate. Capping exists to bound payload size; it is never the reported rate. Emit spans, logs and RUM events for a sample of requests, then report counters as the rate the world model describes — base_rps × diurnal() × interval — not as the number of samples you drew. Gauges and histograms may legitimately describe only the sample; say so in their docstring when they do.

The rule exists because ignoring it produced a bug that survived every check in the suite. proteus.agent.invocations reported a flat 6 per minute — the sampling cap — against a modelled 1,001 to 3,079, and token counters under-reported by roughly 500x. It was deterministic, order-free, idempotent and reproducible across processes, so a golden file would have frozen the wrong value as correct.

Assert counters against the model, never against recorded output.

WorkloadProfile#

proteus/catalog/profile.py is frozen. A profile is a frozen dataclass:

@dataclass(frozen=True, slots=True)
class WorkloadProfile:
    kind: str                       # stable identifier, e.g. "sqs-queue"
    label: str
    runtime: Runtime
    metrics: tuple[MetricSpec, ...]
    aws_namespace: str | None = None   # display only, never emitted
    describe: str = ""
    logs: tuple[LogSpec, ...] = ()
    log_group: str | None = None       # default: /proteus/{model}/{service}
    resource_attrs: dict[str, str] = field(default_factory=dict)
    traced: bool = True
    infra_kind: str | None = None

traced=False is for a workload that is not on a request path — a static volume, a backup job. infra_kind names the InfraKind this workload naturally contends on, if any.

The registered sqs-queue profile, read off the live registry:

kind=sqs-queue  label="SQS queue"  runtime=managed
aws_namespace=AWS/SQS  traced=False

sqs.messages.visible       depth  {message}  7.0     queue_depth_add  ApproximateNumberOfMessagesVisible
sqs.messages.not_visible   depth  {message}  3.0     queue_depth_add  ApproximateNumberOfMessagesNotVisible   gain 0.3
sqs.message.oldest.age     age    s          4.0     queue_depth_add  ApproximateAgeOfOldestMessage           gain 0.05
sqs.messages.sent          count  {message}  1.0     rps_mult         NumberOfMessagesSent
sqs.messages.received      count  {message}  0.94    rps_mult         NumberOfMessagesReceived
sqs.messages.deleted       count  {message}  0.93    rps_mult         NumberOfMessagesDeleted
sqs.receives.empty         count  {message}  0.31    rps_mult         NumberOfEmptyReceives
sqs.message.sent.bytes     bytes  By         2640.0  rps_mult         (none)

log: "redrive: {count} messages exceeded maxReceiveCount"  WARN  async  driver=queue_depth_add  freq 0.2

Note sqs.message.sent.bytes: no aws_metric, because the confirmed counterpart was not established. That is deliberate, and it is the subject of the next section.

MetricSpec#

@dataclass(frozen=True, slots=True)
class MetricSpec:
    name: str
    shape: Shape
    unit: str
    baseline: float = 1.0
    driver: str | None = None
    aws_metric: str | None = None
    aws_namespace: str | None = None
    kind: MetricKind | None = None      # None = inferred from shape
    dimensions: tuple[str, ...] = ()
    capacity_key: str | None = None     # for CAPACITY
    threshold: float = 80.0             # for BINARY
    invert: bool = False                # for BINARY
    gain: float = 1.0

resolved_kind() infers the instrument type when kind is unset: COUNT and ERRORS become SUM, LATENCY becomes HISTOGRAM, everything else becomes GAUGE.

driver names a field on FaultEffect. When a fault is active the generator applies that field to this metric's baseline according to shape, so a profile author never writes fault logic. gain scales the driver's contribution, so two metrics can share a driver at different sensitivity — see sqs.message.oldest.age at gain 0.05 above.

Two rules that are not negotiable#

Never put an AWS/... string in a metric name. CloudWatch reserves any namespace beginning AWS/ and rejects the request outright. The AWS counterpart belongs in aws_metric / aws_namespace, which the generator emits as the attributes proteus.aws_metric and proteus.aws_namespace. That is what lets a UI say "this is your CPUUtilization" without publishing into a reserved namespace. The values are constant per metric name, so they add no series cardinality.

WorkloadProfile.validate() enforces the name half of this at import:

if spec.name.startswith("AWS/"):
    problems.append(
        f"{self.kind}: metric {spec.name!r} starts with the reserved "
        "AWS/ prefix and would be rejected by CloudWatch"
    )

Leave aws_metric as None when you are not sure. An invented AWS metric name reads as authoritative and sends an operator hunting an alarm that cannot exist. Untagged is honest; wrong is worse than missing. None means "no confirmed counterpart", never "not looked up yet", and GET /api/catalog reports the count per profile as unmapped_metrics so a UI can show it as unknown rather than fill it in. Twenty of the 393 specs are unmapped on this build.

validate() also refuses a half-mapping — aws_metric without aws_namespace is both or neither.

LATENCY is always milliseconds#

if spec.shape is Shape.LATENCY and spec.unit != "ms":
    problems.append(
        f"{self.kind}: {spec.name} is LATENCY with unit {spec.unit!r}; "
        "LATENCY is always rendered in milliseconds, so the unit must "
        "be 'ms' even when the AWS counterpart publishes seconds"
    )

This is a validation rule rather than a convention because three specs got it wrong, and they got it wrong for a defensible reason: AWS genuinely publishes Latency and ApplicationLatencyP50 in seconds, and their authors matched the counterpart. The renderer builds every LATENCY histogram over LATENCY_BOUNDS_MS and passes unit straight through, so a spec claiming seconds emits millisecond values labelled as seconds and reads 1000x fast. Name the AWS metric in aws_metric by all means — just keep the unit ms.

There is a second latency trap next door, in _profile_render.py: for a LATENCY spec, baseline is the p50 in milliseconds, not a multiplier on Service.base_latency_p50_ms. Reading it as a multiplier rendered an 86 ms SageMaker ModelLatency at 4,685 ms.

The Shape enum#

Eleven members. The generator owns the arithmetic; a profile only picks a shape and its driver. Keeping the vocabulary small is deliberate — a profile author should not be able to express something the generator cannot render deterministically.

invert, and why it exists#

A plain BINARY asserts a problem: 1 means the bad condition is present, which suits DDoSDetected or StatusCheckFailed. Several AWS health metrics use the opposite convention — HealthStatus, TunnelState and ConnectionState all publish 1 when healthy.

Without invert, those could only be modelled by renaming the metric to assert the negative (health.degraded rather than health.status), which then leaves the aws_metric mapping semantically inverted from the real metric — exactly the class of misleading mapping this module exists to prevent. invert=True keeps both the honest AWS name and the honest mapping.

Driver algebra#

MetricSpec.driver alone does not tell you what moves a metric. Two drivers apply whether or not a spec names them, and _profile_render.py owns the rule:

• request_rate already carries rps_mult, so every rate-derived shape moves with traffic.
• an ERRORS spec always adds error_rate_add, because an error counter that ignored injected errors would be useless.

A spec naming one of those as its driver therefore must not get it applied twice. The consequence: a null driver on a COUNT means "moves with traffic anyway", while a null driver on a GAUGE means "never moves".

RATE_IMPLICIT  = frozenset({"rps_mult"})
ERROR_IMPLICIT = frozenset({"rps_mult", "error_rate_add"})

generic.py::metric_detail(spec) resolves all of this into one dict for an API or GUI, so the surfacing happens once in the module that owns the arithmetic:

inert: true is correct for a static capacity reading and a bug for anything else. Any UI showing a driver should call effective_drivers, not read the field.

A driver naming a field that does not exist on FaultEffect raises AttributeError naming the field, rather than leaving the metric silently inert under every fault.

How to add an AWS service#

Adding a service is a data edit. Pick the module under proteus/catalog/profiles/ that matches the family — compute.py, datastores.py, messaging.py, network.py, analytics.py and their _extra siblings — and register a profile:

from proteus.catalog.profile import (
    LogSpec, MetricSpec, Shape, WorkloadProfile, register_profile,
)
from proteus.model.schema import Runtime

register_profile(WorkloadProfile(
    kind="example-widget",
    label="Example widget",
    runtime=Runtime.MANAGED,
    aws_namespace="AWS/Example",
    describe="One line for the palette.",
    traced=False,
    metrics=(
        MetricSpec(
            name="example.requests",
            shape=Shape.COUNT,
            unit="{request}",
            baseline=1.0,
            aws_metric="RequestCount",
            aws_namespace="AWS/Example",
        ),
        MetricSpec(
            name="example.latency",
            shape=Shape.LATENCY,
            unit="ms",              # always ms, even if AWS publishes seconds
            baseline=42.0,          # the p50 in ms, not a multiplier
            driver="latency_mult",
            aws_metric="Latency",
            aws_namespace="AWS/Example",
        ),
        MetricSpec(
            name="example.queue.depth",
            shape=Shape.DEPTH,
            unit="{item}",
            baseline=5.0,
            driver="queue_depth_add",
            # no confirmed AWS counterpart: leave both fields unset
        ),
    ),
    logs=(
        LogSpec(
            template="widget backlog growing: {count} items",
            severity="WARN",
            scope="async",           # async carries no trace id
            driver="queue_depth_add",
            frequency=0.2,
        ),
    ),
))

Five things to get right:

1. register_profile validates at import. A malformed profile fails loudly at startup rather than producing silently wrong telemetry. It also raises on a duplicate kind.
2. Give every metric that should move during an incident a driver. Check with metric_detail(spec)["inert"].
3. Set scope honestly on a LogSpec. "request" carries trace_id and span_id; "async" carries neither. Never label an asynchronous line as request-scoped to make it look correlated — that asserts a causal link to whichever request happened to be in flight.
4. Verify by rendering, not by reading. Render a tick and check the emitted MetricPoint.value against the world model and the unit the contract states. Every defect this layer has had was invisible in the spec and obvious in the output: gain discarded on 26 ERRORS specs so three counters with three gains rendered identical numbers; storage_gib read verbatim into a BYTES spec, giving 300 bytes for a 300 GiB volume; BINARY health flags reading 0 while healthy.
5. Fix the specs inside your test too. With the catalog under concurrent authorship, diffing against the live registry attributes someone else's profile edit to your change.

Then place it with archetype="custom", kind="example-widget". Nothing else needs to change: the GUI palette, GET /api/catalog and proteus plan all read the live registry.

A profile defined at runtime#

generic.py::profile_from_dict(raw) builds a WorkloadProfile from its dict form, validating before returning. It rejects unknown fields by name and raises ValueError listing every problem validate() found. This is the seam that lets the GUI define a workload kind with no code change — the same profile the hand-written modules declare, assembled at runtime.

When you need an archetype instead#

Write a hand-written generator only when the workload's structure is what matters and no list of metrics captures it: a span tree with a particular shape, a signal the profile renderer does not emit, or a relationship to an InfraNode that needs custom arithmetic. Adding an archetype means adding a member to the frozen Archetype enum, which is why the bar is high.

What the API reports#

GET /api/catalog is built by proteus/web/catalog_view.py entirely from the live registries — catalog.base.registered(), faults.library.known_kinds(), and the enums in model/schema.py. Nothing there is a hand-maintained list, because a palette that drifts from the registry offers the operator workloads the renderer cannot make.

One asymmetry is worth knowing when you read that payload: for an archetype, metric_detail is measured — a representative tick is rendered and the emitted MetricPoints inspected. For a profile it is declared, read straight off each MetricSpec, so it additionally carries shape and driver.

The probe is deliberately not wall-clock: PROBE_EPOCH is a fixed instant and PROBE_TICKS = 6 instants are sampled PROBE_STEP_HOURS = 4.0 apart, so the payload is identical from one request to the next and the UI can cache it. Spreading probes across a day matters because a generator that only logs at certain points on the diurnal curve would otherwise not report logs as one of its signals.