Workshops: AI Tools at Work
Local LLMs & Data Sovereignty: Ollama, Mistral, Llama Workshop
In this workshop IT and engineering teams learn to run open-weight language models locally or in their own cloud for maximum control over sensitive data. You install models with Ollama, compare open-weight models, build a simple document search and assess realistically when self-hosting pays off.
Who is this workshop for?
- IT infrastructure, platform and DevOps teams
- Engineering teams and AI engineers
- Organisations handling highly sensitive data, e.g. healthcare, defence, research
- Public sector and critical infrastructure operators
- CTOs and architects shaping an AI platform strategy
What you will take away
- You understand the difference between open-weight, open-source and proprietary API models, including licensing questions.
- You install and run models with Ollama and know alternatives such as LM Studio, llama.cpp and vLLM.
- You estimate hardware needs realistically based on model size and quantisation.
- You build a simple RAG application that answers questions about internal documents locally.
- You compare cost, quality and operational effort against cloud offerings.
Agenda
Module 1
Why local?
- Data sovereignty, confidentiality, offline capability
- Limits: model quality, operations, cost
- Hybrid scenarios with cloud models
Module 2
Models and licences
- Mistral, Llama, Qwen, Gemma and other open-weight families
- Licence terms and commercial use
- Benchmarks vs your own tests
Module 3
Hands-on with Ollama
- Installation and first models
- Quantisation and hardware requirements
- API access and tool integration
Module 4
Local document search (RAG)
- Embeddings and vector databases
- A pipeline for internal documents
- Measuring and improving quality
Module 5
Running in production
- Serving with vLLM and GPU planning
- Access control, logging and updates
- Security risks such as prompt injection, locally too
When local models pay off
Local models make sense when data must not leave your premises, when you want independence from vendors, or when you process large volumes of similar tasks such as classification or extraction. For demanding knowledge work, the strongest cloud models such as Claude or GPT are often still ahead.
We help you make this trade-off soberly, based on your use cases.
Local does not automatically mean secure
A locally hosted model prevents data from flowing to third parties, but not every risk. Exposed API endpoints, missing access control or prompt injection through documents are real threats on-premise too.
- Never expose endpoints unprotected on the network
- Log access and assign roles
- Keep models and runtimes up to date
Hybrid strategies
Many companies combine local models for sensitive data with cloud services such as Claude, ChatGPT or Mistral Le Chat for general tasks. Our AI Solutions Architects and AI Code Engineers support you with architecture, selection and implementation.
Frequently asked questions
What hardware do we need?
Small models already run on powerful laptops, while larger ones need GPUs with plenty of memory. In the workshop we estimate requirements for your specific use cases.
Are local models as good as ChatGPT or Claude?
They deliver good results for many well-defined tasks. For complex analysis, long contexts or demanding coding, leading cloud models are usually stronger.
Do participants need coding skills?
Basic command-line skills and ideally some Python are helpful, as the workshop is very hands-on.
Workshop
Get in touch
Want to find out whether local LLMs are the right answer for your sensitive data?
Tell us a little about your team – we will get back to you within 24 hours with a proposal for content, format and dates.
Prefer to talk?
More workshops
- Workshops: AI Tools at WorkUsing AI Meeting Assistants in a GDPR-Compliant Way: Workshop
- Workshops: AI Tools at WorkAI Translation & Writing Assistants: DeepL, ChatGPT, Claude
- Workshops: AI Tools at WorkAI Basics for All Employees: AI Literacy Training
- Deploy, Convert & ScaleHow to Deploy a Replit App to Production (Custom Domain, Secrets)
