On-Premise LLM for Arabic: A Practical Guide for Saudi Enterprises

An on-premise LLM is a large language model that runs on servers your organisation controls — in your own data centre or a sovereign cloud inside the Kingdom — instead of through a public AI API. For Saudi enterprises, an Arabic-capable on-premise LLM keeps sensitive data under your control while handling Modern Standard Arabic, Gulf dialect and bilingual content.

Large language models have moved from experiments to everyday business tools: drafting correspondence, summarising reports, answering staff questions and searching internal documents. For Saudi organisations, two requirements shape how these models can be used. They must work well in Arabic, and sensitive data must stay under the organisation's control in line with the Personal Data Protection Law (PDPL) and national data-governance rules. Running an Arabic LLM on-premise, or in an in-Kingdom sovereign cloud, meets both. This guide covers model choice, sizing, infrastructure, evaluation and governance — the decisions a CIO needs to make before the first GPU is ordered.

What is an on-premise LLM, and how is it different from a cloud AI API?

With a public AI API, every prompt — and every document you attach to it — is sent to a provider's infrastructure, processed there and often logged for a retention period you do not set. With an on-premise LLM, the model weights sit on your hardware, inference happens inside your network, and prompts, outputs and logs never leave your environment. A sovereign cloud deployment sits between the two: the model runs on dedicated capacity hosted inside Saudi Arabia, under contracts and keys you control.

Factor	Public AI API	In-Kingdom sovereign cloud	On-premise LLM
Where prompts are processed	Provider's region, often outside KSA	Data centre inside KSA	Your own data centre
PDPL cross-border transfer exposure	Likely, needs a transfer mechanism	Low if all processing stays in KSA	None from inference itself
Cost model	Per token, rises with usage	Reserved capacity	Capital plus operations, flat at volume
Fine-tuning on internal data	Limited, provider-controlled	Yes	Yes, full control
Operational effort	Lowest	Medium	Highest — needs MLOps skills

No option is right for every workload. Many organisations use a public API for non-sensitive drafting and an on-premise LLM for anything that touches personal, financial or classified data. The point is to make that split deliberately, by data class, rather than by default. Our article on data sovereignty and on-premise AI in the GCC covers the regulatory side of this decision in more depth.

Which Arabic LLMs can you run on-premise?

Not every model handles Arabic equally. Some open models were trained mostly on English and treat Arabic as an afterthought; others have been built or fine-tuned specifically for Arabic. The right choice depends on the task: a model that drafts formal letters needs strong Modern Standard Arabic, while a model that reads customer messages must cope with dialect, mixed Arabic-English text, Arabizi and spelling variation.

Arabic-first models

ALLaM is the Arabic LLM family developed by the National Center for Artificial Intelligence at the Saudi Data and AI Authority (SDAIA). An instruction-tuned 7-billion-parameter preview, ALLaM-7B-Instruct-preview, is published with open weights on Hugging Face. According to its model card, it was trained from scratch on 4 trillion English tokens followed by 1.2 trillion mixed Arabic and English tokens, with a 4,096-token context window — which means it can run on your own servers. The family is now developed by HUMAIN, the PIF company launched in May 2025, whose HUMAIN Chat service runs on the larger ALLaM 34B model hosted inside the Kingdom. The Jais family, developed in the UAE, is another Arabic-centric option with open weights. For a business-level view of ALLaM, see our ALLaM guide for businesses.

Multilingual open models

Several general-purpose open-weight model families now include meaningful Arabic data. They are often stronger at reasoning, coding and tool use than smaller Arabic-first models, but their Arabic quality varies widely by version and size. Treat vendor claims of "Arabic support" as a hypothesis to test, not a fact.

Match the model to the job

• Formal drafting and translation: prioritise Modern Standard Arabic fluency and terminology accuracy.
• Customer service and chat: prioritise Gulf dialect understanding and robustness to typos and mixed scripts. Our Arabic chatbot guide covers this use case.
• Search, extraction and classification: a mid-sized model plus good retrieval usually beats a much larger model on its own.

How large does the model need to be?

Model size is the main cost driver. As a rule of thumb, a model needs roughly two bytes of GPU memory per parameter at 16-bit precision just to hold its weights, so a 7-billion-parameter model needs around 14 GB before you account for the working memory each concurrent user consumes. Quantisation — storing weights at 8-bit or 4-bit precision — can cut that substantially with a modest quality trade-off that you must measure on your own Arabic test set.

For many enterprise tasks — classification, extraction, search and summarisation — a well-chosen mid-sized model combined with retrieval-augmented generation (RAG) over your own documents performs as well as a much larger model at a fraction of the infrastructure cost. Start with the smallest model that passes your evaluation, and scale up only when a measured gap justifies it.

What infrastructure does an on-premise LLM need?

• GPU servers sized for the model, the context length you need and the number of concurrent users, with headroom for growth.
• An inference server that batches requests efficiently and exposes the model to internal applications through an authenticated API.
• Retrieval pipelines that index approved internal documents, with Arabic text normalisation (alef and hamza forms, taa marbuta, diacritics) so search works on real Arabic content.
• Identity and access control — role-based access so the model only retrieves documents the user is already entitled to see.
• Audit logging and data classification controls covering prompts, retrieved sources and outputs.
• Monitoring for latency, throughput, accuracy and usage, with alerts when quality drifts.

Security belongs in the design from the start. Government entities and operators of critical national infrastructure are within the scope of the National Cybersecurity Authority's Essential Cybersecurity Controls (ECC-2:2024), and an AI platform is simply another system those controls must cover.

How do you evaluate an Arabic LLM before deployment?

Benchmarks published in English say little about how a model will perform on your Arabic content. Use a structured evaluation instead:

1. Build a test set of 100–300 real, anonymised examples of the work the model will do, in Arabic and English, including dialect and mixed-language inputs.
2. Define what "good" means for each task: factual accuracy, correct terminology, tone, citation of sources, refusal when the answer is not in the documents.
3. Run every candidate model with the same prompts and retrieval settings, so the comparison is fair.
4. Have Arabic-speaking subject experts score the outputs — automated metrics alone miss awkward phrasing and subtle errors.
5. Test for failure modes: hallucinated figures, leakage of restricted documents, prompt injection and responses that breach your policies.
6. Repeat the evaluation whenever the model, prompts or document set change, so quality does not quietly drift.

Which Saudi regulations apply to an on-premise LLM?

Private deployment keeps data inside your environment, but it does not remove your obligations. Three frameworks matter most:

• PDPL. The Personal Data Protection Law, issued by Royal Decree M/19 in September 2021, came into force on 14 September 2023 and became fully enforceable on 14 September 2024 after a one-year transition. SDAIA supervises it through the National Data Governance Platform. If your model processes personal data — HR files, customer records, patient notes — purpose limitation, data minimisation and breach notification apply regardless of where the model runs. Our PDPL compliance guide explains the obligations.
• SDAIA AI Ethics Principles. Published in September 2023, they set out seven principles — fairness, privacy and security, humanity, social and environmental benefits, reliability and safety, transparency and explainability, and accountability and responsibility — and a risk-based approach to AI systems.
• NCA controls. ECC-2:2024 for in-scope entities, plus the NCA's Data Cybersecurity Controls for protecting data across its lifecycle.

Governance questions to settle before launch: who can use the model, which document collections it can see, how outputs are reviewed before they reach customers or decisions, and how usage is logged and retained. Aligning these controls with PDPL and the AI Ethics Principles from day one avoids costly rework later.

What does a practical deployment roadmap look like?

1. Pick one high-value use case — internal policy search or contract summarisation are common starting points.
2. Classify the data the use case touches and decide which deployment model each class allows.
3. Shortlist two or three models and run the evaluation above.
4. Size and build the infrastructure for the winning model, with access control and logging in place before users arrive.
5. Pilot with a small group, measure time saved and error rates, and refine prompts and retrieval.
6. Scale to further use cases on the same platform, so each new one costs less than the last.

This is the approach behind our private AI deployment for a KSA government entity: fully on-premise inference, internal data pipelines, role-based access and Saudi data residency, delivering 100% independence from external AI APIs and predictable cost at enterprise volume. Amwaj Solutions designs and deploys private, Arabic-capable AI infrastructure for Saudi organisations — see our private AI and chatbot service for how we approach it, or contact us to discuss your first use case.