Ever wanted to run a powerful language model locally on your own PC? Good news — with open-source tools and Local AI models, it's totally possible. In this easy tutorial, we'll walk through how to run Gemma 3, an open source LLM, on a Windows computer using Kolosal AI. Kolosal AI is a lightweight app (only ~20 MB) that lets you run Local LLM models with no internet required. That means no cloud needed — everything runs offline on your device, keeping your data private and secure.
Step 1: Download Kolosal AI
Go to https://kolosal.ai and click the "Download for Windows" button. You'll get a small (~20 MB) KolosalAI_Installer.exe file. Kolosal supports both CPU and GPU by default.
Step 2: Install Kolosal AI
Double-click the installer to begin setup. If SmartScreen blocks it, click "More Info" → "Run Anyway". Complete the wizard like any typical Windows app.
Step 3: Load Gemma 3 and Start Chatting
Open Kolosal AI from your Start Menu. The app will open a clean UI — go to the Model Selector, find Gemma 3 (e.g., "Gemma 3 1B"), and click Download. Once downloaded, select it to load into memory.
That's it! Now you can type prompts and chat with Gemma 3 — fully offline, right on your device.
Local AI Example
Try something like: "Suggest a unique pasta recipe using mushrooms and lemon."
Gemma 3 will generate a response like:
"Sure! How about a zesty Mushroom Lemon Herb Pasta? Start by sautéing mushrooms in olive oil with garlic..."
Explore More Models
Kolosal AI supports many other open LLMs like Mistral, Qwen, and LLaMA. You can easily switch models, test responses, or run it as a local API server.