NixOS2 - Secondary/Backup Server

A fully declarative, reproducible NixOS server configured as a secondary/backup server with failover capability. This is the companion to the primary nixos-config server.

Server Role

This is NIXOS2 - the secondary server in a two-server setup:

• Primary Git Server: Gitea runs HERE (mirrors to Codeberg)
• Backup/Failover: Most services are disabled but fully configured for quick failover
• Data Replication: Syncthing mirrors data from the primary server
• Redundant DNS: AdGuard Home provides backup DNS filtering with keepalived VRRP failover (virtual IP 192.168.50.10)

Security Warning

This configuration uses a temporary, publicly-known password for initial convenience:

Default Password: nixos

After installation, you MUST:

1. SSH into the system: ssh ppb1701@YOUR_IP (password: nixos)
2. Change your password: passwd
3. Edit /etc/nixos/configuration.nix:
   • Remove: initialPassword = "nixos";
   • Change: security.sudo.wheelNeedsPassword = true;
4. Rebuild: sudo nixos-rebuild switch

DO NOT expose this system to the internet before changing the password!

Service States

Enabled Services (Running)

• AdGuard Home: Network-wide ad blocking and DNS filtering
  • Backup DNS server for the network
  • Web UI at http://adguard2.home or port 3000
• Homepage Dashboard: Centralized service dashboard
  • Auto-discovers enabled services via NixOS module system
  • Real-time CPU, memory, and disk usage widgets
  • Web UI at http://home2.home or port 8582
• Gitea: Self-hosted Git server (PRIMARY INSTANCE)
  • This is the main Git server, mirrors to Codeberg
  • Web UI at http://git.home or port 3300
• Syncthing: Cross-platform file synchronization
  • Mirrors data from primary server for disaster recovery
  • Web UI at http://syncthing2.home or port 8384
• Tailscale: Secure mesh VPN
  • Remote access to this server
• Nginx: Reverse proxy for clean local URLs
• x11vnc + noVNC: Remote desktop access via browser
  • x11vnc connects to the running LXQT/X11 session (localhost:5910 — offset from primary's 5900)
  • noVNC serves a browser-based VNC client via websockify (port 6080)
  • Access via http://YOUR_IP:6080/vnc.html — password-protected via /etc/nixos/private/vncpasswd
  • Configured in modules/services.nix
• XRDP: Windows-compatible remote desktop (port 3389)
  • Backed by XFCE session; accessible from Windows (mstsc), macOS, or Linux (remmina)
  • Firewall allows LAN (192.168.1.0/24) and Tailscale; all other sources dropped
  • Configured in modules/services.nix and modules/networking.nix
• QEMU/libvirt VM host: Hosts the iso-builder VM (nixos-config vm branch)
  • VM produces custom NixOS ISOs, copies them to /mnt/nextcloud-data/isos/ via virtiofs
  • ISOs available on the network immediately via the isos Samba share
  • Configured in modules/vm.nix — see docs/VM-SETUP.md for full setup
• Samba: File shares accessible from any machine on the network
  • \\nixos2\isos — ISO builds from the iso-builder VM (configured in services.nix)
  • \\nixos2\timemachine — macOS Time Machine target on the 6TB SSD, capped at 2TB (configured in timemachine.nix)
  • Global Samba config (workgroup, Apple extensions, security) lives in services.nix; the timemachine share and its dedicated user live in timemachine.nix
• Keepalived: VRRP BACKUP node for DNS high availability
  • Holds virtual IP 192.168.50.10 when the primary (nixos) goes down
  • Priority 50 (BACKUP); primary nixos has priority 100 (MASTER)
  • Configured in modules/networking.nix
• Restic Backup + Failover Restore System:
  • Backs up Gitea (daily at 02:00) and private configs (daily at 03:15) to /var/local/backups/restic
  • After each backup, repo is shared to primary via Syncthing (chgrp/chmod to syncthing group)
  • Runs a Vaultwarden restore job hourly, pulling the latest snapshot from primary's repo — keeps data warm for fast failover
  • Toggle jobs via enableBackups / enableRestores flags in modules/backups.nix (ebu)

Disabled Services (Failover-Ready)

These services are fully configured but disabled. Enable for failover if the primary server fails:

• Nextcloud: Private cloud storage (runs on primary)
• Collabora Online: Document editing for Nextcloud (runs on primary, requires unstable channel)
• Vaultwarden: Password manager (runs on primary)
• SearX: Self-hosted search (runs on primary)
• Linkwarden: Bookmark manager (runs on primary)
• NoteDiscovery: Knowledge base (runs on primary)
• ntfy-sh: Push notifications (runs on primary)
• PostgreSQL: Database (enable if running Linkwarden/Nextcloud)

Enabling a Failover Service

# 1. Edit services.nix
sudo micro /etc/nixos/modules/services.nix

# 2. Find the service and change:
#    enable = false;  -->  enable = true;

# 3. Configure required secrets in /etc/nixos/private/

# 4. Rebuild
sudo nixos-rebuild switch

Quick Start

Manual Installation

On an existing NixOS system:

git clone https://codeberg.org/ppb1701/nixos2-config /etc/nixos
cd /etc/nixos
sudo nixos-rebuild switch

Note: You'll need to adjust hardware-configuration.nix for your hardware.

From ISO

Download the latest ISO:

https://codeberg.org/ppb1701/nixos-config/releases/tag/nixos

1. Boot from USB
2. Login (user: nixos, password: nixos)
3. Run: sudo /etc/nixos-config/install-nixos.sh
4. Follow prompts
5. Reboot

Configuration

Network Settings

Edit modules/networking.nix:

networking = {
  useDHCP = false;
  interfaces.eno1 = {
    ipv4.addresses = [{
      address = "192.168.50.218";  # Your secondary server IP
      prefixLength = 24;
    }];
  };
  defaultGateway = "192.168.50.1";
  nameservers = [ "127.0.0.1" ];
};

Gitea (Primary Git Server)

Gitea is enabled and configured as the primary Git hosting server:

services.gitea = {
  enable = true;
  settings.server = {
    DOMAIN = "git.home";
    ROOT_URL = "http://git.home";
    HTTP_PORT = 3300;
  };
};

Access: http://git.home or http://YOUR_IP:3300

Required secrets in /etc/nixos/private/secrets.nix:

{
  giteaSecret = "your-gitea-secret-key";
  giteaInternalToken = "your-gitea-internal-token";
  grafanaSecretKey = "your-grafana-secret-key";  # openssl rand -hex 32
}

Syncthing (Data Replication)

Syncthing mirrors data from the primary server:

1. Configure devices in /etc/nixos/private/syncthing-devices.nix
2. Add the primary server as a device
3. Share folders for replication

Access: http://syncthing2.home or http://YOUR_IP:8384

Repository Structure

nixos2-config/
├── configuration.nix              # Main system configuration
├── configuration-bios.nix         # BIOS/Legacy boot variant
├── configuration-uefi.nix         # UEFI boot variant (used by install-nixos.sh for UEFI installs)
├── hardware-configuration.nix     # Hardware-specific settings
├── build-iso.sh                   # ISO build script
├── install-nixos.sh               # Automated installation script
├── iso-config.nix                 # Custom ISO configuration
├── modules/
│   ├── services.nix              # Service configurations (Gitea, x11vnc, noVNC, Samba global+isos, failover-ready services)
│   ├── timemachine.nix           # Samba Time Machine share, tmuser, and directory setup
│   ├── vm.nix                    # QEMU/libvirt VM host (iso-builder), virtiofsd, storage pools
│   ├── homepage.nix              # Homepage Dashboard (service landing page with system monitoring)
│   ├── nginx-virtualhosts.nix    # Nginx reverse proxy virtual hosts
│   ├── monitoring.nix            # Prometheus, Grafana, Alertmanager, Loki, Promtail
│   ├── backups.nix               # Restic backup configuration
│   ├── networking.nix            # Network & firewall settings
│   ├── system.nix                # System packages, users, desktop, SSH
│   ├── boot-bios.nix             # BIOS/GRUB boot configuration
│   └── boot-uefi.nix             # UEFI/systemd-boot configuration
├── home/
│   └── ppb1701.nix               # User environment (ZSH, aliases)
├── private/                       # Private config (gitignored)
│   ├── secrets.nix               # Service passwords, Gitea secrets
│   ├── ssh-keys.nix              # SSH authorized keys
│   ├── syncthing-devices.nix     # Syncthing device IDs
│   ├── vncpasswd                 # VNC password file (for x11vnc)
│   └── alertmanager.env          # SMTP credentials
├── private-example/               # Example templates
├── docs/
│   ├── VM-SETUP.md               # iso-builder VM creation and virtiofs setup
│   ├── SERVICES.md               # Service guide including x11vnc/noVNC and Samba
│   └── TROUBLESHOOTING.md        # Common issues and solutions
└── README.md                      # This file

Failover Procedures

If Primary Server Fails

1. DNS Failover:

   • AdGuard Home is already running on this server
   • Update DHCP to point clients to this server's IP for DNS

2. Nextcloud Failover:

   # Enable Nextcloud
   sudo micro /etc/nixos/modules/services.nix
   # Change: services.nextcloud.enable = true;
   # Also enable PostgreSQL if not already
   sudo nixos-rebuild switch
   

3. Vaultwarden Failover:

   # Data is already warm — the hourly restore job keeps /var/local/vaultwarden current
   # Just enable the service and rebuild
   sudo micro /etc/nixos/modules/services.nix
   # Change: services.vaultwarden.enable = true;
   sudo nixos-rebuild switch
   # Update Tailscale Funnel to point to this server
   sudo tailscale funnel --bg --https=443 http://127.0.0.1:8222
   

4. Other Services:

   • Follow same pattern: edit services.nix, enable service, rebuild

Returning to Primary

1. Ensure primary server is back online
2. Sync any data changes back to primary
3. Disable failover services on this server
4. Update DNS/DHCP to point back to primary

System Maintenance

Shell Aliases

rebuild       # Rebuild and switch configuration
rebuild-safe  # Rebuild, auto-reboot if activation hangs
cleanup       # Clean old generations and optimize store
diskspace     # Check disk usage

# Service management
eaf          # Edit modules/adguard-filters.nix
ags/agr/agl  # AdGuard status/restart/logs
sts/str/stl  # Syncthing status/restart/logs
gts/gtr/gtl  # Gitea status/restart/logs
ncs/ncr/ncl  # Nextcloud status/restart/logs
rds/rdr/rdl  # Redis status/restart/logs
cos/cor/col  # Collabora (coolwsd) status/restart/logs
ncocc        # Run nextcloud-occ command

Cleaning Up Old Generations

cleanup  # Or manually:
sudo nix-collect-garbage -d
sudo nix-store --optimize

Related Repositories

• Primary Server: nixos-config - Main production server
• Blog: https://blog.ppb1701.com/building-a-resilient-home-server-series

License

MIT License

---

Built with NixOS - Secondary/Backup Server Configuration