Key Takeaways
- Self-hosting an AI model gives you data privacy, cost control at scale, and the ability to customise model behaviour — but requires hardware that matches your model’s size
- A 7B-parameter model (e.g. Llama 3, Mistral) runs on a single consumer GPU with 16GB VRAM; a 70B model requires 40–80GB VRAM across one or more high-end cards
- GPU dedicated servers offer maximum performance and fixed monthly costs; cloud VPS with GPU access offers flexibility and lower entry cost but is more expensive at sustained load
- Tools like Ollama and vLLM make self-hosting accessible without deep infrastructure expertise
- For Singapore businesses: local AI hosting simplifies PDPA compliance by keeping customer data within Singapore’s jurisdiction
Paying per API call to OpenAI or Anthropic works well at low volume. At scale — thousands of queries per day, sensitive customer data, or heavily customised behaviour — self-hosting an AI model becomes both viable and economically competitive.
The decision point is infrastructure: what hardware does your chosen model actually need, and which hosting model fits your cost structure and compliance requirements?
Why Businesses Are Self-Hosting AI Models
Three business pressures are driving organisations to run their own models:
Data privacy and compliance.
When you send queries to a third-party API, your data leaves your infrastructure. For businesses handling sensitive customer information — healthcare records, financial data, legal documents — this creates compliance risk. Self-hosting keeps all inference local.
Cost control at scale.
API pricing is per token. At low volumes this is convenient; at high volumes the cumulative cost exceeds self-hosting infrastructure within months. Businesses running AI across internal workflows, customer support, or document processing often reach this crossover point within 6–12 months.
Model customisation and control.
Self-hosted models can be fine-tuned on proprietary data, run with custom system prompts that cannot be overridden, and updated on your timeline — not the API provider’s.
What Hardware Does an AI Model Actually Need?
The primary constraint in AI model hosting is GPU VRAM (video memory). Model parameters must fit into GPU memory to run efficiently. Running inference from CPU RAM alone is possible but 10–50x slower — not viable for production use.
| Model Size | VRAM Required | Example Models | Suitable Hardware |
| 7B parameters | 8–16GB | Llama 3.1 8B, Mistral 7B, Gemma 2 9B | RTX 4090 (24GB), RTX 4080 (16GB) |
| 13B parameters | 16–24GB | Llama 2 13B, Qwen 2.5 14B | RTX 4090, A10G |
| 34B parameters | 24–40GB | CodeLlama 34B | 2× RTX 4090, A100 40GB |
| 70B parameters | 40–80GB | Llama 3.1 70B, Qwen 2.5 72B | A100 80GB, 2× A100 40GB |
| 405B parameters | 200GB+ | Llama 3.1 405B | 4–8× A100 or H100 |
For most Singapore SMEs and mid-market businesses, a 7B or 13B model covers the majority of use cases — document summarisation, internal chatbots, customer support drafts, content generation — and runs on a single mid-range GPU server.
GPU Dedicated Server vs Cloud VPS: Which Is Right for You?
| Factor | GPU Dedicated Server | Cloud VPS with GPU |
| Setup time | 1–3 days provisioning | Minutes |
| Cost model | Fixed monthly fee | Pay-per-use or monthly |
| Performance | Maximum (no shared resources) | Variable (shared infrastructure) |
| Sustained load cost | Lower at high utilisation | Higher at high utilisation |
| Flexibility | Lower (fixed hardware) | Higher (scale up/down) |
| Data sovereignty | Full control | Depends on provider region |
| Best for | Production AI workloads, high query volume | Testing, variable or low query loads |
GPU Dedicated Server is the right choice when your AI workload runs continuously or at high volume, you need maximum performance per query, data sovereignty requires infrastructure you fully control, or you want predictable monthly costs regardless of usage spikes.
Cloud VPS with GPU is the right choice when you are testing a self-hosted AI deployment before committing to dedicated hardware, your AI queries are intermittent, or you need to scale GPU capacity dynamically.
Tools for Self-Hosting an AI Model
You do not need to build a model inference server from scratch. Mature open-source tools handle the infrastructure so you can focus on your application layer.
Ollama — The most accessible starting point. Install it on a Linux server, pull any supported model with a single command, and your model is serving a local API endpoint immediately. Ideal for businesses deploying AI for internal tools and team use. Free and open-source.
vLLM — Production-grade inference server built for throughput. Handles concurrent requests efficiently with PagedAttention, making it suitable for customer-facing applications with multiple simultaneous users. Higher configuration overhead than Ollama but significantly higher throughput at scale.
LM Studio — Desktop application for testing models locally before deploying on a server. Useful for evaluation and development but not suitable for production serving.
Hugging Face Text Generation Inference (TGI) — Enterprise-grade serving framework used by major AI companies. Supports model quantisation (reducing VRAM requirements by 30–50%), streaming responses, and advanced batching strategies for high concurrency.
How to Get Started: Deploying Your First Model
- Choose your model — Start with a 7B or 8B model (Llama 3.1 8B or Mistral 7B) to validate your infrastructure before scaling to larger models
- Provision your server — A dedicated server or VPS with an NVIDIA GPU (at least 16 GB of VRAM is recommended for production use), Ubuntu 22.04 LTS, and CUDA drivers installed
- Install Ollama — Run the installer script; it handles CUDA detection and configuration automatically
- Pull your model — ollama pull llama3.1 downloads and quantises the model for your hardware specifications
- Expose the API — Ollama serves a REST API compatible with OpenAI’s API format, so existing integrations work without modification
- Secure your endpoint — Use a reverse proxy (Nginx) with authentication before exposing your model API to your applications or users
Singapore-Specific Considerations
PDPA compliance is simplified by local hosting. Singapore’s Personal Data Protection Act (PDPA) places obligations on businesses regarding cross-border data transfers. When your AI inference runs on infrastructure in Singapore, customer data processed by the model does not leave the jurisdiction — removing the need for cross-border transfer safeguards that apply to third-party API usage.
Latency matters for real-time applications. Inference requests routed to overseas API providers introduce 100–300ms of network latency before model processing even begins. Singapore-hosted models can serve local users with only milliseconds of network latency — critical for customer-facing applications like chatbots where response speed affects user experience.
Data centre reliability in Singapore. Singapore’s data centre infrastructure is among the most reliable in the Asia-Pacific region, with carrier-grade power redundancy and connectivity. For AI workloads that must stay online, co-locating or renting dedicated servers in a Singapore facility offers both performance and compliance advantages over self-managed office hosting.
When to Self-Host vs Use an API
Self-hosting is not always the right answer. Use this framework to decide:
Self-host when: you process sensitive customer data that cannot leave your infrastructure; your query volume is high enough that API costs exceed hosting costs (typically 1,000+ queries/day for 7B models); you need to fine-tune the model on proprietary data; or you require guaranteed availability independent of third-party provider uptime.
Use a third-party API when: you are in early experimentation or have low query volume; you need access to frontier models (GPT-4o, Claude, Gemini) that cannot be self-hosted; infrastructure management is not a core business capability; or time-to-deployment matters more than long-term cost optimisation.
Many businesses use a hybrid approach: third-party APIs for complex reasoning tasks and frontier models, self-hosted models for high-volume, privacy-sensitive, or cost-sensitive workloads.
Frequently Asked Questions
What is the cheapest way to self-host an AI model?
Start with a 7B-parameter model on a server with a single consumer GPU (RTX 4080 or 4090 with 16–24GB VRAM). A dedicated server with these specs typically costs SGD 200–400/month — often less than API costs at production query volumes.
Can I run an AI model on a standard VPS without a GPU?
Yes, but CPU-only inference is 10–50x slower than GPU inference. For internal tools with low concurrency requirements, a high-RAM CPU server can work. For customer-facing applications, CPU inference is generally too slow for production use.
What open-source models are best for business use in 2025?
Llama 3.1 (Meta) and Qwen 2.5 (Alibaba) are the strongest general-purpose options across multiple sizes, both with commercially permissive licences. Mistral 7B remains popular for its efficiency relative to its parameter count.
Do I need a data centre or can I host at my office?
Office hosting introduces risks: power interruptions, cooling requirements, network reliability, and physical security concerns. For production AI infrastructure, a data centre or managed server provider is strongly recommended.
How does PDPA apply to AI model inference?
Under PDPA, personal data processed by an AI model — customer names in documents, support queries, transaction records — may be subject to data protection obligations. Hosting your model infrastructure in Singapore means processing occurs domestically, simplifying compliance relative to routing data to overseas API providers.
Ready to Run AI on Your Own Infrastructure?
Self-hosting an open-source AI model is now within reach for most Singapore businesses — the hardware costs have dropped, the tooling is mature, and the compliance case is compelling.
Exabytes Singapore offers dedicated server configurations for your specific workloads, with Singapore-based data centre hosting, flexible hardware specifications, and local technical support. Contact our team to discuss the right server configuration for your model size and query volume.
