Provisioning

The CloudFormation stack — what it creates and why it is small, the fingerprint ledger, S3 template staging, drift, and the three fences on teardown.

Synthetic telemetry still needs some real AWS resources to land in. A log group that does not exist rejects every batch; a RUM app monitor that does not exist rejects every event. proteus/cfn/ provisions exactly that scaffolding and nothing else.

Why the stack is small#

CloudFormation can only manage real AWS resources. The services, pods and database instances in a WorldModel are not real — they exist only as OTel resource attributes on synthetic telemetry.

So the template creates four resource types, verified against a synthesised northwind-rx template:

AWS::Logs::LogGroup
AWS::RUM::AppMonitor
AWS::CloudWatch::Alarm
AWS::SNS::Topic

There is deliberately no RDS instance, EKS cluster, MSK cluster or OpenSearch domain. Creating them would cost real money, take many minutes per deploy and teardown, and buy nothing: the telemetry that claims to come from them is generated locally.

If you are tempted to add one, the thing you actually want is another InfraNode in the world model.

This is enforced rather than left to discipline. cfn/_names.py::FORBIDDEN_TYPES lists 15 resource types the template must never contain — the RDS, EKS, MSK, OpenSearch, ElastiCache, EC2, ECS, DynamoDB, ELBv2 and Redshift ones — and synth.validate() reports any that appear. The full list is in the Appendix.

Synthesis is pure#

def template(model: WorldModel, *, opts: SynthOptions | None = None) -> dict[str, Any]: ...

No AWS calls, no wall-clock reads, no randomness. The same model and options always produce a byte-identical template, which is what makes size_report and validate usable offline from proteus doctor.

SynthOptions carries the knobs. The ones with real consequences:

Retention versus the seed horizon#

CloudWatch Logs applies retention against the event timestamp, not the ingest time. A backdated event older than the retention period is accepted with HTTP 200 and then silently discarded — never retrievable, nothing reported.

Proven empirically: with 1-day retention, backdated events returned HTTP 200 and were then discarded, with no error anywhere.

So template() hard refuses rather than warning:

if o.seed_days is not None:
    shortfall = check_retention(o.retention_days, o.seed_days)
    if shortfall:
        raise ValueError(
            "refusing to synthesise a template that would silently lose telemetry. "
            + " ".join(shortfall)
        )

check_retention names the smallest value CloudWatch actually accepts that would cover the horizon, picked from RETENTION_ALLOWED (1, 3, 5, 7, 14, 30, 60, 90 … 3653 days):

LogRetentionDays=7 days is shorter than the 14-day backfill horizon; events older
than the retention period are accepted with HTTP 200 and then silently discarded.
Raise retention to at least 14 days.

The refusal can only fire when the caller declares seed_days. When it is None, the template records "SeedHorizonDays": "undeclared" in its Metadata so doctor can flag that the check never ran. The 14-day default merely happens to be generous; the declaration is the actual protection.

Retention is also a template parameter (LogRetentionDays), so the horizon can be raised on an existing stack without re-synthesising, and the parameter description repeats the warning for anyone reading it in the console.

Log groups come from the renderer, not from a convention#

SynthOptions.log_groups should always carry whatever proteus.render reports it writes to — even though the catalog currently follows /proteus/{model}/{service} and the names are therefore derivable from the service list.

if o.log_groups is not None:
    for name in dict.fromkeys(o.log_groups):        # dedupe, keep order
        problem = check_log_group_name(name)
        if problem is not None:
            raise ValueError(...)
        specs.append((name, {"Source": "renderer"}))

Deriving them here would be a second implementation of that convention, and two implementations are free to drift. Asking the renderer is a fact rather than an assumption, so the stack and the emitter's x-aws-log-group header cannot disagree — and an archetype that later adopts a different scheme stays correct by construction.

Names are used verbatim, never sanitised. An illegal name raises rather than being quietly rewritten, because a rewritten name no longer matches the header the emitter sends and the ingest call fails with HTTP 400 — the exact failure this is guarding against:

SynthOptions.log_groups is unusable: log group name '/proteus/my estate/api'
contains characters CloudWatch Logs rejects: [' ']. Allowed: letters, digits, '.',
'-', '_', '/', '#'. Fix the name at the source - proteus will not rewrite it,
because the stack must match the x-aws-log-group header the emitter sends.

The fallback path (when log_groups is None) derives one group per service and marks each "Source": "derived" in the resource metadata, so you can tell from a deployed stack which way it was built. That path is a guess about what the renderer does: correct only while every archetype follows the convention, and unable to notice when one stops.

How the renderer's set is computed — and why it is derived per service rather than sampled from a render — is in emitters. That is the defect where one group was created by coin flip.

Alarms are the template's bulk#

At most two alarms per service, error rate and p99 latency, kept to two so a 40-service estate produces tens of alarms rather than hundreds. alarm_plan(model, mode) reports what a mode will emit, and the CLI echoes it. Verified on northwind-rx:

alarms=all: 18 of 18 services, 27 alarms, 3 with no alarm this stack can make work
alarms=key: 14 of 18 services, 23 alarms, 1 with no alarm this stack can make work
alarms=none: 0 of 18 services, 0 alarms

"At most" and that third clause are the load-bearing parts. 27 rather than 36 because a service only gets an alarm the stack can actually make work — see metrics resolved per archetype. The count being reported rather than silently lower is the point: three services on this estate have no alarm, and alarm_plan says so rather than leaving you to count.

key means services something else depends on — a failure there is the one that propagates, so those are the alarms worth paging on. An estate with no modelled dependencies would yield nothing under that rule, which would silently mean none, so it falls back to every service and alarm_plan says so:

alarms=key: no dependencies are modelled, so every service is 'key' (18 services, 27 alarms, 3 with no alarm this stack can make work)

none removes the intent of the alarm set entirely: nothing is even configured to trip, and AWS DevOps Agent has no alert to investigate.

The alarms reference metric names and a namespace the renderer and emitter must agree with. The namespace and dimension are SynthOptions fields — dimension_name and metric_namespace — while the metric names are derived per archetype rather than taken from a global default.

Alarm metrics are resolved per archetype#

An alarm is only useful if it names a metric something actually emits, and there is no single metric name that fits an estate. Each archetype emits its own family, so one global name fits one archetype and misses every other. Rendering three-tier and grouping metric names by service shows five services and five disjoint prefixes:

import collections, datetime as dt
from proteus.model import presets
from proteus.render import engine
from proteus.render.timeline import Window
from proteus.render.records import MetricPoint

end = dt.datetime.now(dt.timezone.utc).replace(microsecond=0)
window = Window(start=end - dt.timedelta(minutes=2), end=end, interval_s=60.0)
model = presets.build("three-tier", "us-west-2")
per = collections.defaultdict(set)
for record in engine.render(model, window):
    if isinstance(record, MetricPoint):
        per[record.resource.service_name].add(record.name.split(".")[0])

edge-alb    alb            ['alb']
web-tier    nginx-web      ['nginx']
app-tier    app-service    ['db', 'executor', 'http', 'process']
app-cache   elasticache    ['elasticache']
data-tier   rds-postgres   ['postgres']

So alarm metric names are derived per archetype — an ALB alarm keys on alb.*, a cache on elasticache.*, an application service on http.server.*.

Error-rate alarms are metric math rather than a bare count, because an error count says nothing without the request count beside it, and the threshold is set against that service's own modeled baseline:

Expression: IF(requests > 0, 100 * errors / requests, 0)

Latency alarms use ExtendedStatistic: p99 against a threshold derived the same way, so an alarm description carries both numbers and a reader can see how much headroom the threshold allows.

What is established, and what is not#

This is the part to read before depending on an alarm.

Established, and machine-checked: every metric an alarm names is a metric the renderer emits. tests/test_cfn_alarm_metrics.py compares the synthesized template against a real render for every preset, so a name that drifts out of the emitted set fails the suite rather than deploying quietly.

Also established: no alarm carries a threshold above the widest bucket of the metric it names. That check across all seven presets returns nothing, which matters because a threshold above the top bucket boundary can never be reported at all — the same green-forever failure reached by a different route. ml-pipeline is where this bit: train-flow was alarmed at 112,500 ms against a bucket ladder topping out at 10,000 ms. That alarm is now suppressed rather than deployed dead, and ml-pipeline alarms only on lake and model-endpoint, at 350 ms and 1050 ms.

Not established: whether these alarms fire. Nobody on this project has an AWS account to verify it against, and three things stand between a correct name and a working alarm.

Whether the metrics arrive in the proteus/{model.name} namespace once CloudWatch has ingested them over OTLP. Whether they carry a service.name dimension matching what the alarm keys on. And whether ExtendedStatistic: p99 resolves against these metrics at all — which is where the intuition runs backwards, so it is worth following carefully.

Almost every latency metric here is an OTLP explicit-bucket histogram with delta temporality, carrying count, sum, bucket_counts and explicit_bounds, and no min or max — those fields are optional in OTLP and are never set. That is a real distribution rather than a four-number summary, so it looks like the strong case.

CloudWatch's rule points the other way. Its metrics documentation says percentiles need raw, unsummarized data points, and that summarized data supports them only if the sample count is 1 with minimum, maximum and sum all equal, or if minimum equals maximum. Neither can hold for a histogram spanning several buckets with a count in the hundreds. Whether CloudWatch's OTLP ingest preserves explicit_bounds as something a percentile can resolve against is undocumented and cannot be checked here — but the richer payload is the doubtful one, not the safe one.

Seven of the eight latency alarms sit on the doubtful side of that line. Their metrics — alb.target.response.time, elasticache.get.latency, http.server.request.duration, nginx.request.duration, opensearch.search.latency, proteus.agent.latency and proteus.rum.page_load.duration — are all histograms, and all use ExtendedStatistic: p99.

The eighth is the opposite shape, deliberately. orders-db alarms on postgres.read.latency, which is emitted as a gauge: one raw as_double per datapoint, which is the form the documentation describes as supported. So the poorest data on the estate sits on the firmest documented footing — and, more usefully, the two shapes now fail independently rather than sharing one unverified bet.

That alarm uses Statistic: Maximum rather than p99, and the reason is worth repeating because it is a naming argument rather than a statistical one. A gauge arrives once a minute, so over a 300-second period a percentile is computed from five readings and a p99 of five readings is their maximum. Both would compute the same number; only one of them says so. An alarm named -latency-p99 would invite a reader to think it described the slowest 1% of requests, which it would not.

Do not read either direction as proven. Neither has been tested against a live account.

Correct names are necessary and may not be sufficient. Confirm an alarm can reach ALARM on your own estate before you build a demo on it.

If you have an account and want this settled rather than disclosed, it is one run: deploy the stack, seed a short window, and look at a p99 latency alarm once its metric has data. An alarm still reporting INSUFFICIENT_DATA against a metric that plainly has datapoints answers the question outright. To answer it in the more useful direction, add a second alarm by hand on postgres.read.latency — a gauge, and therefore the documented supported case — and compare the two. If the gauge alarm evaluates and the histogram ones do not, that is the whole answer.

Warning Alarms set TreatMissingData: notBreaching, so an alarm with no data reports OK rather than INSUFFICIENT_DATA. An alarm on a metric that never arrives is indistinguishable from a healthy estate — which is why the earlier defect went unnoticed for so long, and why green is not evidence that anything is being measured.

Which fault reaches the threshold depends on the fault#

The postgres.read.latency threshold is 6 ms, an absolute number rather than one derived per estate. It is reachable on all four presets that carry an rds-postgres service — but not by every fault, and that distinction is easy to measure wrongly.

On minimal, whose database has the lowest baseline at 1.19 ms, slow-query peaks at 4.59 ms and does not trip the alarm, while dependency-timeout reaches 13.76 ms and cache-stampede 6.80 ms, and both do. So the database's own characteristic fault is not the one that trips its latency alarm there. That is a note about fault coverage rather than a defect in the threshold.

tests/test_cfn_alarm_thresholds.py sweeps every shipped fault at both severities against each service and its infra node, which is what it takes to make a claim like this safely. Two narrower sweeps preceded it and both produced confident false results: calibrating on orders-db alone, then measuring all four presets under slow-query alone. One fault across four estates is exactly as partial as one estate across all faults, and neither partial sweep looks partial — both return a clean, complete-looking table.

That is the same shape as the warning above. A flat result from a too-narrow query is indistinguishable from a real absence, just as an alarm with no data is indistinguishable from a healthy estate. In both cases the reassuring answer is the one to distrust, and the only remedy is widening the query until you know what it would take to see the thing you are claiming is not there.

How this was wrong before#

Worth keeping, because a reader running an older deployed stack needs to recognize it.

The alarm metric names used to be global: one triple, proteus.service.requests, proteus.service.errors and proteus.service.latency, applied to every alarmed service. The renderer emitted none of the three. So every alarm evaluated a metric that did not exist, sat at OK because of notBreaching, and presented a wall of green over an estate it was not measuring.

The scale of it is worth stating, because it explains how it survived. Measured at commit 3ed3525, the last revision before the fix: the seven presets synthesized 88 alarms between them, and all 88 named a metric their own service never published. Not a subset, not an edge case — every alarm the tool had ever deployed. Check out that revision and the figure re-derives; it is a measurement of a past state rather than a claim about the current one.

It was found from the outside, by someone testing alarm thresholds rather than reading the template — see Field notes from an SRE.

The first attempt at a fix repointed the names at http.server.*, which is what an app-service emits. That repaired one archetype and left the ALB, the nginx fleet, the cache and the database exactly as broken; the regression test caught it on the web-server preset, which is a single nginx-web service. Per-archetype derivation is the second attempt, and the alarms that could never have fired are now absent rather than present-and-dead — so a lower alarm count than an older stack is the fix working, not lost coverage.

An account of running into this from the outside, with the alarm arithmetic worked through, is in Field notes from an SRE.

Prefer raising the size ceiling over dropping alarms. S3 staging is automatic, so the size ceiling rarely binds in the first place.

Warning --alarms is not a CLI flag, despite two error messages in proteus/cfn/ advising you to pass it. proteus bootstrap --alarms key fails with No such option: --alarms. The alarms mode is a SynthOptions field reachable only from Python:

copy

from proteus.cfn.synth import SynthOptions, template
template(model, opts=SynthOptions(alarms="key"))

The advice in those messages is therefore unactionable as written. Reducing the number of services is the only remedy a command-line user has.

The stack as a state ledger#

WorldModel.fingerprint() is written into the template Metadata and exposed as the WorldModelFingerprint output, alongside WorldModelRevision and ProteusVersion. Verified Metadata for northwind-rx:

{
  "Proteus": {
    "WorldModelFingerprint": "d7caf912a5a9862f",
    "WorldModelRevision": 1,
    "ProteusVersion": "0.1.0",
    "WorldModelName": "northwind-rx",
    "WorldModelRegion": "us-west-2",
    "SchemaVersion": 1,
    "ServiceCount": 18,
    "InfraNodeCount": 8,
    "TenantCount": 3,
    "MetricNamespace": "proteus/northwind-rx",
    "RetentionDays": 14,
    "SeedHorizonDays": "undeclared",
    "GeneratedBy": "proteus.cfn.synth"
  }
}

That is what lets proteus doctor say "stack present, no drift, but your world model is ahead of what is deployed":

model vs stack   WARN   world model fingerprint d7caf912a5a9862f does not match the
                        deployed 4cfb59fa74ddf795 - the estate has changed since
                        bootstrap, so some services may have no log group or alarm
                 fix:   proteus bootstrap   (re-applies the stack)

The fingerprint deliberately excludes faults (see the world model), so injecting another fault does not make the stack look stale.

The full output set, verified:

AlarmTopicArn          LogGroupCount     LogGroupNames    LogGroupPrefix
MetricNamespace        ProteusVersion    RumAppMonitorId  RumAppMonitorName
WorldModelFingerprint  WorldModelName    WorldModelRevision

LogGroupNames is a comma-separated list, emitted only when it fits — stack output values cap at 4,096 bytes, so the code checks 4,000 first and simply omits the output otherwise.

LogGroupPrefix is the longest prefix shared by every group, trimmed to the last /. With renderer-supplied groups spanning /aws/..., /ecs/... and /proteus/... there is no useful shared prefix, and the code reports that honestly — "/", or a fallback to the derived proteus prefix — rather than a proteus-shaped prefix most groups are not under. Use LogGroupNames to enumerate, not LogGroupPrefix.

RumAppMonitorId and RumAppMonitorName appear only when the estate has at least one BROWSER_RUM service; the CLI needs both to sign RUM data-plane calls. With more than one monitor the unsuffixed pair points at the first BROWSER_RUM service in model order and each monitor also gets its own suffixed pair.

Template size and S3 staging#

CloudFormation accepts 51,200 bytes of TemplateBody inline but 1 MB via TemplateURL.

S3_THRESHOLD_BYTES   =    45_000   # where proteus switches to staging
MAX_INLINE_TEMPLATE_BYTES = 51_200 # CloudFormation's inline limit
MAX_S3_TEMPLATE_BYTES = 1_000_000  # CloudFormation's TemplateURL limit
ARTEFACT_EXPIRY_DAYS  =         7

Inline, the tool caps out at roughly 21 services — a 22-service estate synthesises to about 52 KB — which makes 56 workload types and a CUSTOM archetype a hollow promise. Staging through S3 raises the ceiling roughly 20x — several hundred services, running out at roughly 400 against CloudFormation's own 1 MB TemplateURL limit.

The 45,000-byte threshold sits below the hard limit by a deliberate margin: the 51,200-byte limit applies to the request, not just the body.

Verified for northwind-rx:

template 37,797 bytes across 47 resources, sent inline (74% of the 51,200-byte limit)

That sentence is TemplateSize.detail, and proteus doctor prints it verbatim. The check is offline — it synthesises the template in memory and measures it — so it runs before the credentials gate. A template that cannot deploy is worth knowing about even on a machine with no working profile.

If you measure the template yourself you will get a different number, and the spread straddles the limit. The figure depends on how the document is serialised, not only on what is in it. One unchanged northwind-rx template, against the 51,200-byte inline cap:

That is a 1.7x spread, and the widest reading exceeds the limit while the real one sits 26% below it. Pretty-printing is what anyone does to read a JSON document, so the plausible path is: dump the template, indent it, measure the file, conclude the estate cannot deploy inline. The conclusion is wrong and the margin looks decisive, which is worse than an ambiguous one.

Only the serialisation CloudFormation actually receives counts, and it is neither the prettiest nor the smallest. The size also moves with the options you pass — include_injector_role=True adds resources and changes the total again. So a size figure is comparable only against another produced the same way, and the number to trust for the inline limit is the one doctor prints.

doctor is also careful about whose fault a big template is. A large one is not a user error, so the fix line says so:

fix: no action needed: bootstrap stages large templates to S3 automatically

The size check uses default SynthOptions, which derives its own log groups rather than taking the ones bootstrap passes. That was verified conservative: the default build came out 1,113 bytes larger than the explicit-log-group build on the same 22-service estate, so the check never reports headroom that bootstrap will then consume.

Where a template body goes#

Two thresholds, not one, and the gap between them is where the interesting behaviour lives. _template_source in proteus/cfn/deploy.py decides:

flowchart TD
    SIZE["template body<br>size in bytes"]
    SMALL{"45,000 or less"}
    INLINE["TemplateBody<br>inline, no bucket needed"]
    FITS{"51,200 or less"}
    NOS3{"allow_s3"}
    TRY["stage to S3"]
    URL["TemplateURL<br>ceiling becomes 1 MB"]
    FAIL["DeployError<br>names the real constraint"]

    SIZE --> SMALL
    SMALL -- yes --> INLINE
    SMALL -- no --> FITS
    FITS -- no --> NOS3
    FITS -- "yes, the band" --> NOS3
    NOS3 -- "false, fits inline" --> INLINE
    NOS3 -- "false, too big" --> FAIL
    NOS3 -- true --> TRY
    TRY -- ok --> URL
    TRY -- "failed, fits inline" --> INLINE
    TRY -- "failed, too big" --> FAIL

S3_THRESHOLD_BYTES is 45,000 and MAX_INLINE_TEMPLATE_BYTES is 51,200, so between them a template still fits inline but without margin. S3 is preferred there, and a staging failure degrades to inline rather than failing — a caller with no S3 permissions must not lose an estate that deployed inline yesterday. Above 51,200 the same failure is fatal, because there is nothing to degrade to.

allow_s3=False forbids staging for a run, which turns the band into plain inline and anything above it into an error.

The degradation band#

Between the staging threshold and the hard inline limit, the template still fits inline — just without margin. S3 is preferred there, but a caller with no S3 permissions must not lose an estate that deployed inline yesterday, so a staging failure in that band degrades to inline rather than failing.

Above the inline limit, a staging failure raises the original size error with the S3 attempt appended, so the operator sees both what the real constraint is and why the workaround did not apply:

template is 78,400 bytes, above the 51,200-byte inline limit; reduce the number of
services, or pass --alarms key (or none) to drop per-service alarms. S3 staging was
attempted and failed: could not create bucket proteus-templates-...

That message is quoted as the tool really prints it, including the advice to pass --alarms key, which is not a flag. Read it as "reduce the number of services".

allow_s3=False forbids staging for a run.

The artefact bucket#

The bucket holds build artefacts, not state. Nothing in it is a source of truth: every object can be regenerated from the world model by re-running synth. That is why objects carry a 7-day lifecycle expiry and why losing the bucket costs nothing.

bucket_name(account_id, region) is proteus-templates-{account}-{region}, lowercased, with a sha256 fragment substituted for the region if that exceeds S3's 63-character limit. S3 bucket names are global, so embedding the account id is what stops two accounts contending for one name. If the name is nonetheless taken by someone else, that is reported rather than worked around:

bucket name proteus-templates-<your-account-id>-us-west-2 is taken by another AWS
account. proteus will not silently pick a different name, because the next run
would not find it. Deploy a smaller estate, or pass a bucket you control.

Object keys are content-hashed (templates/{stack}/{sha256[:16]}.json), so redeploying an unchanged template reuses the same object instead of accumulating near-identical copies.

On creation the bucket gets public access blocked, the proteus:managed tag, and a lifecycle rule expiring staged objects after 7 days. Tagging and lifecycle are best-effort; the public access block is not. If it cannot be applied the bucket is not safe to use, so that failure propagates:

created bucket proteus-templates-... but could not block public access
(AccessDenied). Refusing to stage templates in a bucket whose exposure is unknown;
delete it or grant s3:PutBucketPublicAccessBlock.

Deploy#

def deploy(session, stack_name, template, *, wait=True, parameters=None,
           timeout_seconds=1800, allow_s3=True) -> StackInfo: ...

Every function in cfn/deploy.py takes an explicit boto3 Session, so the caller decides which credentials are in play. This module never constructs one and never reads credentials itself.

Idempotent: creates when the stack is absent, updates when it is present, and treats CloudFormation's "No updates are to be performed" as success. A name not starting with proteus- is refused outright with UnsafeStackError.

CAPABILITY_IAM is added only when the template actually declares an AWS::IAM:: resource, rather than being passed unconditionally.

Three ways it refuses rather than guessing:

• the stack is mid-operation — wait for it to settle;
• the stack is in a state CloudFormation cannot update (ROLLBACK_COMPLETE, ROLLBACK_FAILED, REVIEW_IN_PROGRESS);
• the operation ends in a non-healthy terminal state.

A failed create is left at ROLLBACK, not deleted:

cfn.create_stack(**kwargs, OnFailure="ROLLBACK")

Proteus never deletes on its own. A failed create is left for the user to inspect and remove with an explicit teardown.

stack_name(model) slugifies the model name, prefixes proteus-, and truncates to 128 characters: northwind-rx becomes proteus-northwind-rx.

Logical ids, and the one thing not to change#

def logical_id(prefix: str, name: str) -> str:
    parts = [p for p in re.split(r"[^A-Za-z0-9]+", name) if p]
    cleaned = "".join(p[:1].upper() + p[1:] for p in parts)[:60] or "X"
    digest = hashlib.sha256(name.encode("utf-8")).hexdigest()[:6]
    return f"{prefix}{cleaned}{digest}"

Service names may contain characters CloudFormation forbids — logical ids are alphanumeric only — so the name is camel-cased with separators removed, and a 6-hex-character digest of the original name is appended so ids stay unique when sanitisation collides.

Do not change what is fed in here without a teardown first. A logical id is a resource's identity to CloudFormation, so changing the derivation — or changing which string is passed as name — is a replace, not an update. CloudFormation creates every new resource before deleting the old ones, and any physical name they share (a log group name, an alarm name) collides mid-update.

Observed in practice: switching log group ids from service-derived to group-name-derived produced "Validation failed with 13 error(s)" and a rolled-back update that only a teardown and clean recreate resolved. If a future change alters this derivation, ship it with a note telling operators to teardown first.

Drift#

def detect_drift(session, stack_name, *, timeout_seconds=300, poll_seconds=5) -> DriftReport: ...

Runs CloudFormation drift detection and returns per-resource drift. Read-only: drift detection never modifies a resource.

It polls until CloudFormation finishes or the timeout elapses, and a timeout is reported as DriftReport.timed_out, not raised — so a slow detection does not look like a failure. DETECTION_FAILED comes back as stack_drift_status: "UNKNOWN" with the reason in detail.

DriftReport.drifted filters to resources that are neither IN_SYNC nor NOT_CHECKED. Each ResourceDrift carries property_differences as readable strings:

/RetentionInDays: expected '14', actual '3'

Drift and the fingerprint answer different questions. Drift asks "has someone changed the deployed resources by hand?". The fingerprint asks "has the world model moved ahead of what was deployed?". doctor reports both.

Teardown, and the three fences#

teardown deletes real AWS resources, so it is fenced three ways and every fence is a hard refusal, not a warning:

1. the stack name must start with proteus-;
2. the stack must prove Proteus ownership;
3. confirm=True must be passed.

Without confirm, the function returns a TeardownPlan listing exactly what would be deleted and deletes nothing. teardown_plan is the read-only entry point and applies the same name and ownership fences, so a refusal surfaces before anything is confirmed.

There is deliberately no "force", no "delete anyway", and no way to skip a fence.

@dataclass(frozen=True, slots=True)
class TeardownPlan:
    stack_name: str
    stack_id: str
    model_name: str
    resources: tuple[PlannedDeletion, ...]
    executed: bool = False
    detail: str = ""
    staged_templates: tuple[str, ...] = ()

    @property
    def summary(self) -> str:
        """One line per resource type, e.g. `3x AWS::Logs::LogGroup`."""

Ownership proofs, including the pre-rename ones#

@property
def is_proteus_managed(self) -> bool:
    return (
        bool(self.outputs.get(VERSION_OUTPUT))          # ProteusVersion
        or bool(self.outputs.get(LEGACY_VERSION_OUTPUT))  # CwomniVersion
        or self.tags.get(MANAGED_TAG_KEY) == MANAGED_TAG_VALUE          # proteus:managed=true
        or self.tags.get(LEGACY_MANAGED_TAG_KEY) == MANAGED_TAG_VALUE   # cwomni:managed=true
    )

The current proofs are the ProteusVersion output and the proteus:managed=true tag. The pre-rename CwomniVersion and cwomni:managed=true forms are accepted for reading only and are never written.

The reason is worth stating, because accepting a legacy credential normally makes a safety check weaker. A stack deployed while this tool was called cwomni carries CwomniVersion and no ProteusVersion output. Without accepting the old form, the ownership check fails and teardown refuses to delete resources this tool genuinely created. Refusing is the safe direction in isolation — but it would leave a live stack that nothing can clean up, which is not a state to ship anyone into. Because the legacy forms are never written, the next deploy re-stamps the stack under the current names and the old proof stops being needed.

The same reasoning applies to the artefact bucket: _templates3._is_managed accepts either tag key, and stays fail-closed — an unreadable tag set returns False, because a bucket whose ownership cannot be established must not be deleted.

A refusal reads:

refusing to delete 'proteus-northwind-rx': it has neither the ProteusVersion output
nor the proteus:managed=true tag, so proteus did not create it

Staged templates and the bucket#

Staged template objects for the stack are deleted alongside it — they are build artefacts, so they go with the stack.

The bucket survives unless remove_bucket=True and it is empty afterwards and it carries a managed tag. A shared-name bucket someone else is using must not vanish because a demo was torn down, and leaving it costs nothing: the lifecycle rule expires stragglers. Even with all three conditions met, a failed delete_bucket is swallowed — it is left behind on purpose rather than forced.

Offline validation#

def validate(tpl, *, backfill_days=None) -> list[str]: ...
def size_report(tpl) -> TemplateSize: ...

validate lints a template offline and returns a list of problems; empty means clean. It catches the mistakes that would otherwise only surface as a failed deploy: illegal logical ids, forbidden resource types, dangling Ref and Fn::GetAtt targets, a missing fingerprint, and the 500-resource per-stack limit.

Pass backfill_days and the retention check runs too. Skipping it skips that check, so pass it whenever the seed window is known.

Verified on northwind-rx with backfill_days=14: [].

TemplateSize.verdict is ok, warn or fail. It warns at 80% of the staging threshold as well as on the staging path itself, and fails only above the 1 MB S3 ceiling — at which point the only remedies are fewer services or fewer alarms.

Both functions are what let proteus doctor find a size or retention problem before a deploy rather than during one. The failure mode they prevent is late and confusing: a 22-service estate exceeds the inline limit, bootstrap aborts, the new services' log groups are never created, and the next seed partially fails on log groups that do not exist. Discovering that at bootstrap time is bad; discovering it three commands later is worse.