How the AI behaves, and what it costs

The bridge between "we use AI" and exactly how it behaves, when it's trusted, and what it costs — the doc I'd defend line-by-line with an ML engineer.

Owner Product, with ML Engineering · shared pipeline for creative generation and performance recommendation.

Pipeline

Input→ Model call(s)→ Confidence scoring→ Lane assignment→ Human review*→ Action→ Feedback loop

Model routing — match cost to risk, not "best everywhere"

First-pass creativeLower-cost modelHigh volume, low stakes — filtered before a human sees it.

Top-N refinementHigher-capability modelLow volume, higher bar — reaches the reviewer.

Brand-safety checkDeterministic rulesCompliance-critical — never left to probabilistic behavior.

Confidence scoring

Confidence = f(model self-score, historical acceptance rate, data volume, signal strength)

Model self-score is one weak signal among several — never sufficient alone.
<7 days or <100 conversions caps confidence regardless of what the model claims.

Review lanes

Auto-apply

High confidence + low blast radius + established trust tier.

Suggest

Medium confidence, or newer account — most v1 traffic lands here by design.

Escalate

Low confidence, thin data, or compliance-sensitive action.

Cost controls

Monthly generation cap enforced at the request layer, before any model call.
Tiered routing: 8 first-pass + top-2 refined ≈ 25% the cost of "best model everywhere."
Cache key = feed-content hash + prompt template version.
Circuit breaker on provider errors → "generation delayed," not a retry storm.