How Mintok turns a workload into a cost.

0. Overview & principles

Mintok takes a Workload Spec (how many tokens, how often, how fast) and produces three artifacts: a Sizing recommendation (how many chips, of which kind), a Cost projection ($/M-token across relevant scenarios), and an Engagement Brief that synthesizes both into a customer-readable doc.

Each artifact has a clear formula. The full chain is:

Workload Spec  ──▶  Three-constraint solver  ──▶  Recommended chip count (N)
                            │                                 │
                            ▼                                 ▼
                  Binding constraint label           Throughput (tokens/sec)
                            │                                 │
                            └─────────────┐       ┌──────────┘
                                          ▼       ▼
                                Cost model (CapEx + Power + OpEx)
                                          │
                                          ▼
                                    $/M-token

1. The Workload Spec

The Spec captures what your workload does, separate from how it's implemented. Each field is a physical quantity with a unit. We don't store anyone's interpretation; we store the numbers.

Templates pre-fill defaults per archetype (chatbot, RAG, etc.) so you start from a sensible baseline. Every field is overridable; the brief records both the source (template default vs explicit FDE input) and which numbers the customer reviewed.

2. The Sizing algorithm

Sizing answers "how many chips of type X do I need to serve this workload?" We use a three-constraint solver: each chip count is the maximum across three independent physical limits. Whichever is largest binds — that label is surfaced on the Sizing tab as "Binding: Compute / Bandwidth / Memory" so you know which constraint to relax.

Notation used in §2. Symbols are kept consistent through every formula.

P       = active model parameters (count, not billions)        ▸ from ModelProfile.activeB × 1e9
bpp     = bytes per parameter                                  ▸ 2 for BF16, 1 for FP8/INT8, 0.5 for FP4
T_tgt   = target throughput (tokens/sec)                       ▸ derived from Spec eventsPerDay + tokensOut
peakF   = chip peak FLOPS                                      ▸ from ChipProfile.bf16Tflops × 1e12
mfu     = Model FLOPs Utilization (fraction, 0–1)              ▸ default 0.40 (see §4)
hbmBw   = chip HBM bandwidth (bytes/sec)                       ▸ from ChipProfile.hbmBwGbps × 1e9
hbmCap  = chip HBM capacity (bytes)                            ▸ from ChipProfile.hbmGb × 1e9
B       = serving batch size                                   ▸ from Spec or model default
S       = context length (tokens)                              ▸ from Spec contextMax
L       = model layers                                         ▸ from ModelProfile or estimated
D       = hidden dimension                                     ▸ from ModelProfile or estimated

2.1 Compute constraint

Each generated token costs roughly 2P FLOPs in forward pass (multiply-add per parameter, twice). To deliver T_tgt tokens/sec, the cluster needs at minimum:

chipsCompute = ⌈  (2 × T_tgt × P)  /  (peakF × mfu)  ⌉

physical reading:
  numerator   = total FLOPS/sec the workload demands
  denominator = effective FLOPS/sec each chip actually delivers under load

The 2 is the well-established forward-pass FLOPs-per-parameter constant (one multiply + one add per weight, per token). The × mfuacknowledges that real workloads never hit peak silicon spec; published benchmarks for H100 BF16 inference land 35–55% depending on batch and sequence — we default to 0.40 for conservative sizing. (Override per engagement when you have measured numbers.)

2.2 Bandwidth constraint

Decode is memory-bandwidth bound: every output token streams the model's weights from HBM into the compute units. With batch size B, those weights are reused B times per stream, so batching amortizes the cost.

chipsBandwidth = ⌈  (T_tgt × P × bpp)  /  (B × hbmBw × mbu)  ⌉

physical reading:
  numerator   = total bytes/sec the workload must read across all streams
  denominator = effective bytes/sec each chip's HBM delivers per batch slot

The × mbu is the bandwidth analogue of mfu on the compute side: Memory Bandwidth Utilization — the fraction of peak HBM bandwidth you actually sustain during decode. You never hit the spec number; real MBU on current accelerators lands roughly 50–65% depending on batch, sequence, and kernel quality. We default to 0.50 for conservative sizing (override per org in Finance settings, or per run). A lower MBU means more chips to hit the same token rate.

This is why short-context interactive workloads with tight latency targets (small batch sizes) often hit the bandwidth wall before the compute wall — and why long-context batch workloads can be compute-bound instead. Quantization (FP8 halves bpp) is one of the few levers that helps both compute and bandwidth simultaneously.

2.3 Memory constraint

The model's weights must fit in HBM. So must the KV-cache for every active sequence in the batch. The cluster must hold both, concurrently:

modelBytes = P × bpp                              ▸ weights, in bytes
kvBytes    = 2 × L × D × S × B × bpp              ▸ KV-cache for active batch

chipsMemory = ⌈  (modelBytes + kvBytes)  /  hbmCap  ⌉

The KV-cache formula comes from attention mechanics: each layer caches 2tensors (K and V), each of size D per token, for S tokens of context, across Bconcurrent sequences. Long-context (large S) workloads can have KV-cache that exceeds the model itself — a 200K-token context on a 70B model is comparable to the model's own weight footprint.

2.4 Binding constraint

The cluster needs to satisfy all three constraints, so the chip count is the max:

N = max(chipsCompute, chipsBandwidth, chipsMemory, 1)

binding =  "Compute"    if chipsCompute   is the max
           "Bandwidth"  if chipsBandwidth is the max (tie-breaking favors bandwidth over memory)
           "Memory"     otherwise

The binding label appears in the brief and the Sizing tab. It tells you what to relax to lower cost. If compute-bound: a smaller model or quantization. If bandwidth-bound: larger batches or higher-bandwidth silicon. If memory-bound: shorter contexts or model distillation.

After N is set, throughput is back-computed as the minimum of what compute and bandwidth can actually deliver at that count:

throughputCompute   = (N × peakF × mfu) / (2 × P)
throughputBandwidth = (N × hbmBw × B)   / (2 × P × bpp)

throughputActual    = min(throughputCompute, throughputBandwidth)

3. The Cost algorithm

Once the cluster is sized (N chips), we stack three cost streams: hardware, power, and overhead OpEx. The whole thing collapses into one number — $/M-token — so customers can compare like-for-like across chips, models, and providers.

3.1 Hardware cost

Hardware cost has two flavors depending on acquisition mode:

# CapEx mode: amortize the purchase over the depreciation horizon.
totalChipCost     = N × chipUnitPrice
totalRackInfra    = ⌈N / 32⌉ × rackInfraCost     ▸ 32 = 8 chips/node × 4 nodes/rack
totalCapEx        = totalChipCost + totalRackInfra
annualCapEx       = totalCapEx / amortYears       ▸ default 4 years

# Lease mode: monthly bill, no amortization.
monthlyHwLease    = N × chipLeasePerMonth         ▸ from vendor / colo provider
annualHwLease     = monthlyHwLease × 12

Rack infrastructure cost covers PDU, network switches, cabling, cold-aisle containment — the things you buy alongside the chips. Default is amortized over the same horizon as chips, unless the lease includes it (vendor-dependent).

3.2 Power cost

Each chip's nameplate TDP (Thermal Design Power) is its max sustained draw. Datacenter total draw is higher because cooling and conversion losses add overhead — captured by the PUE (Power Usage Effectiveness) multiplier.

clusterPowerKW = (N × chipTdpW × PUE) / 1000      ▸ PUE default 1.30

annualPowerCost = clusterPowerKW × 8,760 × kwhCost
                  └────────────┘   └─────┘   └──────┘
                  draw at the wall  hr/yr    $/kWh you pay

kwhCost is engagement-specific — typical US datacenter rates are $0.04–$0.12/kWh; renewable PPAs can go lower; constrained markets higher. We don't assume; the FDE inputs the customer's actual contracted rate (or industry average if unknown), and the brief notes which.

3.3 OpEx overhead

Real operations cost more than chips + power. Networking egress, software licenses, on-call engineering, monitoring, backup, datacenter staff — collectively the OpEx overhead. We bundle these as a single percentage on top of subtotal:

annualSubtotal     = annualHw + annualPowerCost
annualOpExOverhead = annualSubtotal × opexOverheadPct%
annualTotalCost    = annualSubtotal + annualOpExOverhead

Typical opexOverheadPct ranges 15–35% depending on customer maturity (in-house team with existing infra: low; greenfield deployment: high). The FDE sets this per engagement; the brief shows what was chosen.

3.4 $/M-token derivation

The unifying metric. Take the cluster's sustained throughput (from §2.4), project annual tokens at expected utilization, divide by total annual cost:

annualTokens   = throughputActual × 31,536,000 × utilizationPct%
                            (sec/year)  └────────────────┘
                                        fraction of time cluster is serving load,
                                        typically 60–80% for production inference

costPer1MToken = annualTotalCost × 1,000,000 / annualTokens

Lower utilization → fewer tokens generated per dollar of fixed cost → higher $/M-token. This is why a lightly-loaded reserved cluster looks expensive vs cloud-on-demand. The Cost tab renders both: CapEx-amortized $/M-token and equivalent cloud-rate $/M-token, so the customer sees the crossover.

4. Constants reference

Every named constant used above. These don't change per customer; they change only when the methodology version bumps (and you can compare deltas across versions).

Catalog values (chip TFLOPS, HBM GB, HBM bandwidth, TDP, list price, model active params, model layers, hidden dim) are not constants — they live in Mintok's chip and model catalog and update as vendors publish new specs. The brief always cites the catalog version it ran against.

5. Glossary

MFU (Model FLOPs Utilization)

The fraction of peak silicon FLOPS actually consumed by useful model computation. A 200-TFLOPS chip running at 0.40 MFU delivers 80 effective TFLOPS for the model.

MBU (Memory Bandwidth Utilization)

Analogous metric for HBM. Decode is bandwidth-bound; prefill is compute-bound. Our cost formula uses the binding-constraint approach to handle both.

KV-cache

Key-value tensors cached per layer per token for the attention mechanism. Size grows linearly with context length and batch size. Often the memory-binding constraint for long-context workloads.

Binding constraint

The physical limit that determines minimum chip count. If your model needs 10 chips for compute, 4 for bandwidth, and 8 for memory, you need 10 (the max). Compute is the binding constraint in that case.

$/M-token

"Dollars per million tokens." Mintok's unifying metric. All capex / power / opex collapse into this single number for like-for-like comparison.

Active parameters

For Mixture-of-Experts models, only a subset of params is active per token. We size against active params (e.g., DeepSeek-R1 = 671B total, 37B active).

Prefill vs decode

Prefill = processing the input prompt (one large parallel matmul). Decode = generating output tokens one at a time (autoregressive, smaller per-step compute, memory-bandwidth bound).

TCO horizon

The years over which total cost is summed. 3-year TCO is typical for cloud comparison; 5-year for CapEx amortization.

Sensitivity table

Cost projection rendered as a range across uncertain inputs (typically agent-calls-per-event). The customer sees a band, not a point estimate, because we refuse to fake precision we don't have.