AI Inference
Run LLMs at scale
Dedicated NVIDIA clusters with sub-second API inference.
- Llama · Mistral · Mixtral
- vLLM · Ollama · TGI
- OpenAI-compatible endpoint
- LoRA & QLoRA fine-tuning
- GPU autoscaling
- Data stays on your server
Llama · Mistral · Mixtral · Gemma · Qwen · DeepSeek · Phi-4.
Paged-attention batching + streaming for high throughput.
LoRA & QLoRA workflows with your own datasets.
Replicas that follow traffic — no cold-start tax.
Currency
Choose billing period
Neuron
Fine-tune & serve small open-source models (up to 14B).
৳28,625/mo · save 20%
৳549,600 billed every 24 months
Choose plan- 1× RTX 4090 · 24 GB GDDR6X
- 36-core Dual Xeon · 256 GB RAM
- 240 GB SSD + 2 TB NVMe + 8 TB SATA
- Llama 3 8B · Mistral 7B · Phi-4 14B
- Ollama · vLLM · llama.cpp preinstalled
- OpenAI-compatible /v1/chat/completions
- 1 Gbps unmetered · 1 dedicated IPv4
Synapse
Serve mid-size 27B–34B models with room to breathe.
৳51,125/mo · save 20%
৳981,600 billed every 24 months
Choose plan- 1× RTX A6000 · 48 GB GDDR6
- 36-core Dual Xeon · 256 GB RAM
- 240 GB SSD + 2 TB NVMe + 8 TB SATA
- Gemma 2 27B · DeepSeek-R1 14B · Mixtral 8×7B
- vLLM auto-batching & streaming
- LoRA / QLoRA fine-tuning workspace
- Priority ML engineer support
Cortex
Production inference for 70B-class models on A100.
৳194,875/mo · save 20%
৳3,741,600 billed every 24 months
Choose plan- 1× NVIDIA A100 · 80 GB HBM2e
- 36-core Dual Xeon · 256 GB RAM
- Multi-model serving · replica reuse
- Llama 3.3 70B · Qwen 2.5 72B
- Redis token cache · Prometheus metrics
- Autoscaling replicas · JWT / OAuth2
- 99.9% uptime SLA
Hypermind
Frontier H100 clusters with InfiniBand interconnect.
Tailored to your requirement — pay only for what you need.
- H100 / H200 GPU clusters (1–8× per node)
- InfiniBand interconnect · NVLink
- Llama 3.1 405B · DeepSeek-V3 · custom weights
- TensorRT-LLM · DeepSpeed · multi-GPU tensor parallel
- Private endpoints · VPC peering · SOC 2
- Dedicated MLOps team · 24/7 AI support
- Enterprise SLA & compliance (HIPAA, GDPR)
GPU availability varies by region. 1 USD = 125 BDT · 1 EUR = 155 BDT. Hourly billing available on request.
Featured models
Top open-source LLMs, pre-installed.
Skip the download-and-configure ritual. Every server ships with your chosen model, runtime and API layer ready to serve traffic in about 15 minutes.
State-of-the-art 70B model — parity with Llama 3.1 405B for most tasks.
First-gen reasoning model with performance comparable to OpenAI o1.
High-performance efficient model available in 2B, 9B and 27B variants.
Sparse Mixture-of-Experts with strong reasoning at lower inference cost.
14B state-of-the-art open model tuned for reasoning and math.
Long-context multilingual model with excellent code + tool use.
Right-sizing
Pick a GPU by model size.
A rough rule: VRAM ≥ model size × 1.2 for Ollama / TGI, or × 1.5 for vLLM. Multi-GPU nodes unlock tensor parallelism for the largest frontier models.
- Ollama: ≥ Model × 1.2 VRAM
- vLLM: ≥ Model × 1.5 VRAM · strong multi-GPU
- TGI: ≥ Model × 1.2 · strong multi-GPU
- DeepSpeed: ≥ Model × 1.1 · super-strong (NVLink)
Architecture
Modular, production-ready inference stack.
Every deployment ships with the pieces you'd otherwise assemble by hand — load balancing, an OpenAI-compatible gateway, batched inference, Redis token cache and full observability.
- Load balancer · API gateway
- vLLM / TensorRT-LLM engine
- Redis token cache
- A100 / H100 GPU cluster
- Model registry (S3 / NFS)
- Prometheus + Grafana
Drop-in API
A drop-in replacement for the OpenAI API.
Point your existing SDK at your Host Tonic endpoint and keep your data, your weights, and your latency budget. No refactor required.
- OpenAI /v1/chat/completions
- Streaming & auto-batching
- LoRA / QLoRA fine-tuning
- Private model registry
- Multi-model serving
- Autoscaling replicas
- Token & latency dashboards
- API keys · OAuth2 · JWT
from openai import OpenAI
client = OpenAI(
base_url="https://llm.your-domain.com/v1",
api_key="ht_live_••••••••",
)
resp = client.chat.completions.create(
model="llama-3.3-70b",
messages=[{"role": "user",
"content": "Ship it."}],
stream=True,
)
for chunk in resp:
print(chunk.choices[0].delta.content or "", end="")Why self-host
The case for a dedicated LLM server.
Full data residency, on-prem-style compliance (HIPAA, GDPR), controlled logs and audit trails.
Dedicated GPUs, Redis token caching and load balancing — no noisy neighbours.
Pay for the server; run as many tokens as it can produce. Zero surprise invoices.
You control the model version. No silent upgrades, no behaviour drift.
Tensor + pipeline parallelism for 30B–70B+ models with NVLink / InfiniBand.
Break free from API rate limits and cloud dependencies — your endpoint, forever.
Runtimes & tools
Every inference stack you'd reach for.
Paged-attention batching · streaming · high throughput
One-command model management · perfect for dev + prototypes
GGUF quantization · CPU + GPU hybrid inference
Hugging Face Text-Generation-Inference · tensor parallel
NVIDIA-optimised kernels for H100 / A100
Beautiful chat UI on top of your private endpoint
Frequently asked questions
How fast can I deploy an LLM server?+
Around 15 minutes. Our auto-provisioner installs the GPU driver stack, your chosen runtime (vLLM / Ollama / TGI) and pre-pulls the model weights so you can hit /v1/chat/completions on first boot.
Which GPUs are available?+
RTX 4090 (24 GB), RTX A6000 (48 GB), A100 (80 GB HBM2e) and H100 (80 GB) — single or multi-GPU nodes with NVLink / InfiniBand for the larger clusters.
Is the endpoint really OpenAI-compatible?+
Yes. /v1/chat/completions, /v1/completions and /v1/embeddings match the OpenAI schema, including streaming and function calling. Point any OpenAI SDK at your endpoint.
Do I pay per token?+
No. You rent the server; run as many tokens as the hardware can produce. Billing is either monthly or hourly — pick whichever fits your workload.
Can I fine-tune models on my own data?+
Yes — Neuron and above ship with a LoRA / QLoRA workspace. Bring a JSONL dataset, kick off a run, and load the adapter into vLLM without redeploying.
Where does my data live?+
On the server you rent. Weights, prompts and completions never leave your instance. Storage is encrypted at rest and the network is protected end-to-end.