An AI assistant on your server
You want an AI assistant on your server, without handing it the keys to the house. Let's install it on its own account, with a backup behind us, then look at the alternative that runs at home.
- Course
- Tutorial 28 of 35
- Time
- 25 min to read
- Tested with
- Debian 13.7, Claude Code 2.1.318
What you will be able to do
- Choose between an online assistant and a local model.
- Create an account just for the assistant and back up before you start.
- Install the assistant, then launch it and log in.
- Pick the assistant's permission mode, and know what hardware a local model needs.
Before you start: Your first SSH connection, Secure SSH: turn off passwords, Users, sudo and file permissions.
Contents
How do you host an AI assistant on your server? You have a Debian machine and plenty of commands you don't know by heart yet. An assistant reads your files and suggests the right lines, on one condition: it can only break what you planned for. This tutorial installs the assistant in a closed-off corner of the server, then explains when a local model is the better idea. One warning up front: the online assistant needs a paid online account, covered in step 4. The local model needs none.
Two ways to get an assistant on your server
There are two options. The online assistant is installed on the server, but it does its thinking at the company that makes it. The local model does its thinking on the server itself. Here, the online assistant is Claude Code, a program made by Anthropic. This is what sets them apart:
- An online assistant is a program you start in the terminal and talk to in plain language. It reads your files, suggests commands and runs them if you agree. It is light, and it needs a paid online account, because the thinking happens on the maker's machines, over the Internet.
- A local model is the "brain" of the AI, a big file that your server runs itself. Nothing leaves your home, but it needs memory and, ideally, a graphics card.
We start with the online assistant, because the computing happens at the maker's end: your server only needs 4 GB of memory. The local model gets its own chapter further down. A Linux account is an identity on the machine, with its own home folder and its own rights. The assistant runs its commands with the rights of the account that started it. So let's build it an account of the right size.
Alternative text: your computer connects over SSH to mon-serveur. On mon-serveur, there are two accounts: alice with sudo, and assistant without sudo. The assistant account sends its questions to an online service over the Internet.
1Create an account just for the assistant
Create a separate account with no administrator rights. The sudo prefix lends you those rights for one command, and this account won't be allowed to use it. So the assistant can't change the system by itself. The full refresher is in Users, sudo and file permissions. First, log in over SSH, as in Your first SSH connection. Here, alice is an example name: replace it with your own user. Run these two commands. The first creates the assistant account without a password: until you add a key, nobody can log in to it over SSH. The second, id, proves the account exists by printing its numbers. You'll enter it from alice in step 3.
sudo adduser --disabled-password --gecos "" assistant
id assistantuid=1001(assistant) gid=1001(assistant) groups=1001(assistant),100(users)Your numbers may differ. Now check that it has no administrator rights. This command asks sudo (-l lists the rights, -U names the account) what the Linux account assistant is allowed to do.
sudo -l -U assistantUser assistant is not allowed to run sudo on mon-serveur.Check that its home folder is closed to other Linux accounts. The -d option of ls shows the folder itself, not its contents. The rights drwx------ mean that only the owner can go in.
ls -ld /home/assistant /home/alicedrwx------ 2 alice alice 4096 Nov 6 09:19 /home/alice
drwx------ 2 assistant assistant 4096 Nov 6 09:19 /home/assistantYour dates will be your own. So the other accounts can't enter alice's folder. What's left is protecting yourself from the assistant's own mistakes.
2Back up first
Make a copy of /etc, the folder of system settings, before you let the assistant near anything. A backup is a spare copy, so you can go back if a step goes wrong. Here, the copy is a single file, an archive, stored in alice's home, a folder closed to other accounts since the previous step. It holds your settings, including the accounts' encrypted passwords: keep it there and nowhere else.
mkdir -p ~/sauvegardes
sudo tar -czf ~/sauvegardes/etc-avant-assistant.tar.gz -C / etc
sudo chown alice: ~/sauvegardes/etc-avant-assistant.tar.gz
chmod 600 ~/sauvegardes/etc-avant-assistant.tar.gz
ls -lh ~/sauvegardestotal 96K
-rw------- 1 alice alice 94K Nov 6 10:51 etc-avant-assistant.tar.gzYour size and date will be your own. tar packs a whole folder into one file: -c creates the archive, -z compresses it, -f gives its name and -C / first moves to the root of the system, the / folder. Then chown gives the file to alice, and chmod 600 reserves it for her alone, since it holds encrypted passwords. This archive protects your settings; for your data, use Your first backup with rsync, which copies to another machine.
Get one file back from the archive
To get a file back, extract it into a separate folder, without touching /etc. Here, -x extracts and etc/hostname names the file you want. It appears in ~/restauration/etc/hostname: open it next to the current file in /etc, then copy over only the line you want to restore.
mkdir -p ~/restauration
tar -xzf ~/sauvegardes/etc-avant-assistant.tar.gz -C ~/restauration etc/hostname
ls ~/restauration/etc3Install the assistant
Install Claude Code from the assistant account. Claude Code needs Debian 10 or later and 4 GB of RAM, the machine's working memory. It also needs curl, the program that downloads a web address. From alice, the next command installs curl, or tells you it's already there.
sudo apt install -y curlTo work inside the assistant account, open a session in it from alice. The -u assistant option names the account to open, and -i sets up the session as if the assistant had just logged in. You'll leave it with exit, which closes the session and brings you back to alice.
sudo -iu assistantYou are now in the assistant account. The installer is a script from the maker: a file of commands that bash, the terminal's program, reads line by line. Download it into a file first, read it with less (q to quit), then run it. Mostly you'll see downloads from the maker's addresses and copies into ~/.local. If the text looks unreadable or mentions other sites, stop and ask for advice.
curl -fsSL https://claude.ai/install.sh -o install.sh
less install.sh
bash install.shSetting up Claude Code...
Checking installation status...
Installing Claude Code native build latest...
Setting up launcher and shell integration...
Setup notes:
* Native installation exists but ~/.local/bin is not in your PATH. Run:
echo 'export PATH="$HOME/.local/bin:$PATH"' >> ~/.bashrc && source ~/.bashrc
Claude Code successfully installed!
Version: 2.1.318
Location: ~/.local/bin/claude
Next: Run claude --help to get started
Setup notes:
* Native installation exists but ~/.local/bin is not in your PATH. Run:
echo 'export PATH="$HOME/.local/bin:$PATH"' >> ~/.bashrc && source ~/.bashrc
Installation complete!Your version number is probably newer. The installer says the PATH, the list of folders where the terminal looks for programs, doesn't contain ~/.local/bin yet. It suggests a command to add it, which you can ignore: at the next login, Debian adds that folder by itself. The next command checks exactly that. You stay in the assistant account, but exec bash -l swaps your terminal for a fresh one, as if you had just logged in (-l stands for "login"). Then you ask for the version.
exec bash -l
claude --version2.1.318 (Claude Code)This installation updates itself in the background. For an update right now, type claude update.
4The first launch
The assistant connects to your online account with the maker, not to be confused with the Linux account. A free account isn't enough. There are two plans:
- A paid subscription with the maker, which includes Claude Code use. The price is the same every month, and it's what I recommend for starting out.
- A Console account, the maker's developer space, where you pay in a sum in advance. Each question uses a little of it: you pay for what you use, so the bill varies.
Prices change: read the official plans page on the maker's website before you pay. You are still in the Linux account assistant. Create a working folder, then start claude inside it.
mkdir -p ~/travail
cd ~/travail
claude --permission-mode manualThe --permission-mode option picks the permission mode, explained below. You can list its possible values with claude --help.
The assistant asks whether you trust this folder: say yes, it's yours. Then it offers to log you in. Your server has no browser, so it prints an address. Copy it, open it on your computer and log in to your online account. The browser then shows a code. Paste it if the assistant asks, at the Paste code here if prompted prompt. The terminal announces Login successful.
To check the connection, quit the assistant with /exit (it closes the assistant only, while exit closes the account's session) and query the tool. It prints a few lines of settings: look for the loggedIn line, which should end with true. This command is only useful if you doubt your login. With false, see "If it fails".
claude auth statusStart claude --permission-mode manual again in ~/travail, then ask a first harmless question: "Which Debian version is running here, and how much disk space is left?" If it wants to run a command, it has to ask permission: you read the command, then accept or refuse. If it asks nothing, see "If it fails". Let's see why this permission matters.
What it can run, and how to limit it
The assistant can run any command the Linux account assistant is allowed to run. The permission mode sets what it does without asking you. Remember this one:
manual: it reads on its own, but asks before changing or running anything. It's the mode from the step 4 launch, and the right one for starting out.
The documentation describes others, like acceptEdits (it edits files without asking) or auto, which let the assistant act more and more freely. Keep them for later. The riskiest one, bypassPermissions, removes every safeguard. It's meant for containers, which are disposable mini-machines, isolated from the rest of yours.
Four habits complete this setting:
- No sudo for this Linux account. When a command needs administrator rights, you run it yourself as alice, after reading it.
- No secrets within reach. It reads everything its account reads (its own folders, but also most of the settings in
/etc), and what it reads goes to the online service. Keep passwords and keys out of~/travail, for example in alice's folder. - No unknown content pasted as it is. A booby-trapped text can try to give it instructions, which is called a prompt injection. Read what you give it to read.
- No mode without safeguards. The maker keeps it for containers, and the tool refuses to start it as root, the administrator.
With a limited Linux account and manual mode, the assistant can only change what its account touches. The backup limits nothing: it lets you go back. The assistant can still read what its account reads, and that goes to the maker. If you'd rather nothing goes to a third party, look at the other option.
The alternative: a local model
A local model keeps your data on the server, but it needs more hardware and answers less well than a big online service. The table compares the two options.
| Criterion | Online assistant | Local model |
|---|---|---|
| Where it thinks | At the maker's | On your server |
| What leaves the server | What it reads and your questions | Nothing |
| Hardware | 4 GB of RAM at least | Enough free memory to hold the model, plus some room |
| Cost | A paid online account | The hardware and the electricity |
| Acting on the server | It suggests and runs commands | It answers your questions, without acting |
Ollama is a program that downloads and runs a local model. You can use it instead of Claude Code, or on top of it: it's an assistant that answers your questions without touching the server. You install and start it from alice, because its installation needs sudo, which the assistant account doesn't have. What decides it: the server's memory needs enough free room to hold the model while it runs. The name llama3.2:1b means a small model (1b: one billion internal settings), which is 1.3 GB to download. The model works in memory with extra data on top of its own weight: plan for at least double 1.3 GB, so 2.6 GB here. Read the available column of this command: it's the memory that programs can still get.
free -h total used free shared buff/cache available
Mem: 62Gi 11Gi 1.8Gi 753Mi 51Gi 51Gi
Swap: 8.0Gi 136Mi 7.9GiThe installation and a first try take five commands:
apt installaddszstd, which the installer needs to unpack its files.curldownloads Ollama's script into a file, without running it.lessopens it: arrows to scroll,qto quit. You'll see downloads from ollama.com; if the text mentions other sites, stop.shruns it (this script is written forsh, unlike the one in step 3). It may ask for your sudo password along the way.ollama rundownloads the model, so be patient, and opens a text conversation;/byeleaves it.
sudo apt install -y zstd
curl -fsSL https://ollama.com/install.sh -o install.sh
less install.sh
sh install.sh
ollama run llama3.2:1bGraphics card and model memory
A graphics card (GPU) computes much faster than the processor. Ollama uses one with a compatible NVIDIA card and a driver at version 550 or later (the NVIDIA program that lets the system talk to the card). Without a card, it uses the processor, more slowly. Once the model is loaded, the ollama ps command shows the memory it takes.
If it fails
Four situations come up often:
claude: command not found. You didn't reopen the account after the installation, so the terminal isn't looking in~/.local/binyet. Typeexitto go back to alice, then reopen the account withsudo -iu assistant. Without the-i, a plainsudo -u assistantdoesn't read the profile, the settings file the terminal reads at login, and you get the same error.Not logged in · Please run /login,loggedInset tofalse, or a refused code. The tool is installed but not logged in. Startclaude, type/loginand follow the address it prints. If the address doesn't open or the code is refused, run/loginagain to get a new one.curl: command not found. The install script can't be downloaded, so nothing gets installed. Do this in order. Typeexitto go back to alice. Installcurlwithsudo apt install -y curl. Reopen the account withsudo -iu assistant. Then pick up step 3 again at the download ofinstall.sh.- The assistant runs commands without asking. It isn't in manual mode. Quit it with
/exit, then startclaude --permission-mode manualagain: the option goes at the end of the launch line.
What's next
Your assistant now lives in a limited Linux account, and a copy of /etc sits in alice's home. It helps you while you're at the screen, but it won't warn you about anything while you sleep. The next article, Simple monitoring, sets up the monitoring that alerts you when the server is unwell.