Instant Sizing Report

Your customer support chatbot workload processes 100 events daily with 100 input tokens and 200 output tokens per event, targeting 1 token per second throughput. Across three evaluated self-hosted options on MI500 hardware—Qwen2.5 3B, Gemma 2 2B, and Llama 3.2 1B—bandwidth emerges as the binding constraint, limiting inference performance despite single-chip configurations. The annual token costs range from 0.0244 to 0.0733 dollars per million tokens depending on model selection. Against the hosted alternative (Kimi K3 at 10 dollars monthly), self-hosting requires approximately 24,907 events daily to break even; your current 100 daily events make hosted deployment substantially more economical at this scale. The practical takeaway is clear: remain with managed inference until event volume increases 249 times, at which point self-hosting becomes cost-competitive and delivers operational control over your latency-sensitive chatbot.