Initial commit
This commit is contained in:
sarodz
284
README.md
Normal file
284
README.md
Normal file
@@ -0,0 +1,284 @@
|
||||
# Homelab Starter
|
||||
|
||||
A minimal GitOps homelab setup using **K3s** and **Flux CD**. This starter gives you a working Kubernetes cluster with core infrastructure and one example app (Gitea), all managed declaratively through Git. Secrets are encrypted at rest in Git using **SOPS** with **age**.
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
┌──────────────────────────────────────────────────────────┐
|
||||
│ Git Repository │
|
||||
│ │
|
||||
│ bootstrap/ Infrastructure & apps/ │
|
||||
│ ├── ns/ app definitions └── gitea │
|
||||
│ ├── repositories/ are organized │ │
|
||||
│ └── kustomization/ into layers │ │
|
||||
└──────────┬─────────────────────────────────────┘ │
|
||||
│ │
|
||||
▼ │
|
||||
┌──────────────────┐ Flux watches your Git repo │
|
||||
│ Flux CD │ and continuously reconciles │
|
||||
│ (flux-system) │ the cluster state to match. │
|
||||
└──────────┬───────┘ │
|
||||
│ │
|
||||
▼ │
|
||||
┌──────────────────────────────────────────────────────────┐
|
||||
│ Kubernetes Cluster (K3s) │
|
||||
│ │
|
||||
│ Infrastructure layer Application layer │
|
||||
│ ┌────────────┐ ┌────────────┐ │
|
||||
│ │ MetalLB │─── IPs ──────▶ │ Gitea │ │
|
||||
│ │ Traefik │─── Routing ──▶ │ PostgreSQL │ │
|
||||
│ │ Cert-Mgr │─── TLS ─────▶ │ │ │
|
||||
│ └────────────┘ └────────────┘ │
|
||||
└──────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
## What's Included
|
||||
|
||||
| Component | Purpose |
|
||||
|-----------|---------|
|
||||
| **MetalLB** | Assigns real LAN IPs to LoadBalancer services |
|
||||
| **Traefik** | Ingress controller — routes HTTP/HTTPS traffic to apps |
|
||||
| **Cert-Manager** | Automatically provisions TLS certificates via Let's Encrypt |
|
||||
| **Gitea** | Self-hosted Git service (example app) with PostgreSQL |
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- A Linux machine (or VM) with [K3s](https://k3s.io) installed
|
||||
- `kubectl` configured to talk to your cluster
|
||||
- [Flux CLI](https://fluxcd.io/flux/installation/) installed locally
|
||||
- [SOPS](https://github.com/getsops/sops) and [age](https://github.com/FiloSottile/age) installed locally
|
||||
- A Git repository (GitHub, Gitea, GitLab, etc.) to host this code
|
||||
- A domain name pointed at your cluster (for TLS certificates)
|
||||
- A Cloudflare account (or other DNS provider) for DNS-01 challenges
|
||||
|
||||
## Quick Start
|
||||
|
||||
### 1. Copy this starter into your own repository
|
||||
|
||||
```bash
|
||||
# Create a new repo and copy the starter contents into it
|
||||
# (include hidden files like .sops.yaml)
|
||||
cp -r starter/{.,}* ~/my-homelab/
|
||||
cd ~/my-homelab
|
||||
git init && git add -A && git commit -m "Initial homelab setup"
|
||||
git remote add origin <YOUR_GIT_REPO_URL>
|
||||
git push -u origin main
|
||||
```
|
||||
|
||||
### 2. Replace placeholder values
|
||||
|
||||
Search for all `<YOUR_...>` placeholders and replace them with your values:
|
||||
|
||||
| Placeholder | File(s) | Description |
|
||||
|---|---|---|
|
||||
| `<YOUR_IP_RANGE>` | `infrastructure/metallb-config/ipaddresspool.yaml` | LAN IP range for LoadBalancer services, e.g. `192.168.1.200-192.168.1.210` |
|
||||
| `<YOUR_LAN_CIDR>` | `infrastructure/traefik-install/traefik-override.yaml` | Your local network CIDR, e.g. `192.168.1.0/24` |
|
||||
| `<YOUR_EMAIL>` | `infrastructure/cert-manager-issuer/issuer.yaml` | Email for Let's Encrypt notifications |
|
||||
| `<YOUR_DNS_API_TOKEN>` | `infrastructure/cert-manager-issuer/secrets.yaml` | Cloudflare API token (see below) |
|
||||
| `<YOUR_DOMAIN>` | `infrastructure/routes/gitea.yaml`, `apps/gitea/install/gitea.yaml` | e.g. `git.example.com` |
|
||||
| `<YOUR_DB_PASSWORD>` | `apps/gitea/install/secrets.yaml`, `apps/gitea/install/gitea.yaml` | A strong password for PostgreSQL |
|
||||
|
||||
```bash
|
||||
# Quick way to find all placeholders
|
||||
grep -r '<YOUR_' --include='*.yaml' .
|
||||
```
|
||||
|
||||
**Why is a Cloudflare API token needed?** Cert-manager uses [DNS-01 challenges](https://cert-manager.io/docs/configuration/acme/dns01/) to prove domain ownership to Let's Encrypt. Unlike HTTP-01 challenges, DNS-01 doesn't require your server to be publicly reachable — ideal for homelabs behind NAT. It works by automatically creating a DNS TXT record on your domain, which requires API access to your DNS provider. Create a Cloudflare API token with **Zone > DNS > Edit** permission for your domain at [dash.cloudflare.com/profile/api-tokens](https://dash.cloudflare.com/profile/api-tokens).
|
||||
|
||||
### 3. Set up secret encryption (SOPS)
|
||||
|
||||
Secrets are stored in dedicated `secrets.yaml` files and must be encrypted before committing to Git. This starter uses [SOPS](https://github.com/getsops/sops) with [age](https://github.com/FiloSottile/age) encryption so that Flux can decrypt them in-cluster while they stay encrypted at rest in your repository.
|
||||
|
||||
#### Generate an age key pair
|
||||
|
||||
```bash
|
||||
age-keygen -o age.agekey
|
||||
# Output: Public key: age1xxxxxxxxx...
|
||||
```
|
||||
|
||||
> **Important:** Add `age.agekey` to `.gitignore` — never commit your private key.
|
||||
|
||||
#### Update `.sops.yaml`
|
||||
|
||||
Replace `<YOUR_AGE_PUBLIC_KEY>` in `.sops.yaml` with the public key from the previous step. This file tells SOPS which files to encrypt and which YAML fields to target:
|
||||
|
||||
```yaml
|
||||
creation_rules:
|
||||
- path_regex: infrastructure/.*/secrets\.yaml$
|
||||
age: age1xxxxxxxxx... # your public key
|
||||
encrypted_regex: "^(data|stringData)$"
|
||||
- path_regex: apps/.*/secrets\.yaml$
|
||||
age: age1xxxxxxxxx... # your public key
|
||||
encrypted_regex: "^(data|stringData)$"
|
||||
```
|
||||
|
||||
Only `data` and `stringData` fields are encrypted — metadata stays readable so Git diffs remain useful.
|
||||
|
||||
#### Encrypt your secret files
|
||||
|
||||
After filling in your actual values (step 2), encrypt each secrets file:
|
||||
|
||||
```bash
|
||||
sops --encrypt --in-place infrastructure/cert-manager-issuer/secrets.yaml
|
||||
sops --encrypt --in-place apps/gitea/install/secrets.yaml
|
||||
```
|
||||
|
||||
To edit an encrypted file later:
|
||||
|
||||
```bash
|
||||
sops infrastructure/cert-manager-issuer/secrets.yaml
|
||||
```
|
||||
|
||||
#### Load the private key into the cluster
|
||||
|
||||
Flux needs the age private key to decrypt secrets during reconciliation:
|
||||
|
||||
```bash
|
||||
kubectl create namespace flux-system
|
||||
|
||||
kubectl create secret generic sops-age \
|
||||
--namespace=flux-system \
|
||||
--from-file=age.agekey=age.agekey
|
||||
```
|
||||
|
||||
The Flux Kustomizations in this starter already reference this secret via `decryption.secretRef.name: sops-age`.
|
||||
|
||||
### 4. Bootstrap Flux
|
||||
|
||||
```bash
|
||||
flux bootstrap git \
|
||||
--url=<YOUR_GIT_REPO_URL> \
|
||||
--branch=main \
|
||||
--path=bootstrap
|
||||
```
|
||||
|
||||
This command:
|
||||
- Installs Flux components into the `flux-system` namespace
|
||||
- Creates `bootstrap/flux-system/` with auto-generated Flux manifests
|
||||
- Commits and pushes these files to your repo
|
||||
- Starts the reconciliation loop
|
||||
|
||||
### 5. Watch Flux reconcile
|
||||
|
||||
```bash
|
||||
# Watch all kustomizations converge
|
||||
flux get kustomizations --watch
|
||||
|
||||
# Check pod status across all namespaces
|
||||
kubectl get pods -A
|
||||
```
|
||||
|
||||
### 6. Access Gitea
|
||||
|
||||
Once everything is reconciled and the TLS certificate is issued:
|
||||
|
||||
```bash
|
||||
# Check the certificate status
|
||||
kubectl get certificate -n gitea
|
||||
|
||||
# Open Gitea in your browser
|
||||
echo "https://<YOUR_DOMAIN>"
|
||||
```
|
||||
|
||||
## Directory Structure
|
||||
|
||||
```
|
||||
.sops.yaml # SOPS encryption rules (which files, which fields)
|
||||
|
||||
bootstrap/ # Flux bootstrap layer
|
||||
├── flux-system/ # Auto-generated by `flux bootstrap` (don't edit)
|
||||
├── kustomization/ # Flux Kustomization CRs — tell Flux what to deploy
|
||||
│ ├── infrastructure/ # One file per infrastructure component
|
||||
│ └── apps/ # One directory per app
|
||||
│ └── gitea/
|
||||
├── ns/ # Namespace definitions
|
||||
└── repositories/ # Helm chart repository sources
|
||||
|
||||
infrastructure/ # Core cluster services
|
||||
├── metallb-install/ # MetalLB Helm release
|
||||
├── metallb-config/ # MetalLB IP pool and L2 advertisement
|
||||
├── traefik-install/ # Traefik Helm release + config overrides
|
||||
├── cert-manager-install/ # cert-manager Helm release + config overrides
|
||||
├── cert-manager-issuer/ # Let's Encrypt ClusterIssuer + DNS credentials
|
||||
│ ├── issuer.yaml
|
||||
│ └── secrets.yaml # encrypted with SOPS
|
||||
└── routes/ # Traefik IngressRoutes (one per app)
|
||||
|
||||
apps/ # Application deployments
|
||||
└── gitea/
|
||||
└── install/ # Gitea + PostgreSQL manifests
|
||||
├── gitea.yaml
|
||||
├── postgresql.yaml
|
||||
└── secrets.yaml # encrypted with SOPS
|
||||
```
|
||||
|
||||
## How Flux Works
|
||||
|
||||
Flux follows a **reconciliation loop**:
|
||||
|
||||
1. **Source Controller** watches your Git repository for changes
|
||||
2. **Kustomize Controller** applies Kustomization CRs (in `bootstrap/kustomization/`)
|
||||
3. **Helm Controller** installs/upgrades Helm releases (MetalLB, Traefik, Cert-Manager)
|
||||
4. If you push a change to Git, Flux detects it and updates the cluster automatically
|
||||
|
||||
The `dependsOn` fields in the Flux Kustomizations ensure things install in the right order:
|
||||
|
||||
```
|
||||
MetalLB Install → MetalLB Config → Traefik Install → Routes
|
||||
Cert-Manager Install → Cert-Manager Issuer
|
||||
Traefik Install → Gitea Install
|
||||
```
|
||||
|
||||
Kustomizations that reference encrypted secrets include a `decryption` block pointing to the `sops-age` secret, so Flux can decrypt them transparently during apply.
|
||||
|
||||
## Next Steps
|
||||
|
||||
- **Add a backup system** — Set up Restic + Rclone to back up your PVCs and databases to cloud storage.
|
||||
- **Add your second app** — See [`docs/adding-an-app.md`](docs/adding-an-app.md) for a step-by-step walkthrough.
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
**Flux not reconciling?**
|
||||
```bash
|
||||
flux logs --level=error
|
||||
flux get sources git # Check if Flux can reach your repo
|
||||
flux reconcile source git flux-system # Force a sync
|
||||
```
|
||||
|
||||
**MetalLB not assigning IPs?**
|
||||
```bash
|
||||
kubectl get ipaddresspool -n infrastructure
|
||||
kubectl get svc -n infrastructure # Check if Traefik has an EXTERNAL-IP
|
||||
```
|
||||
|
||||
**Traefik not routing?**
|
||||
```bash
|
||||
kubectl get ingressroute -A # Check if routes exist
|
||||
kubectl logs -n infrastructure -l app.kubernetes.io/name=traefik
|
||||
```
|
||||
|
||||
**TLS certificate not issuing?**
|
||||
```bash
|
||||
kubectl get certificate -A # Check certificate status
|
||||
kubectl get certificaterequest -A # Check pending requests
|
||||
kubectl describe clusterissuer cert-issuer # Check issuer health
|
||||
kubectl logs -n infrastructure -l app=cert-manager
|
||||
```
|
||||
|
||||
**SOPS decryption failing?**
|
||||
```bash
|
||||
# Check if the sops-age secret exists
|
||||
kubectl get secret sops-age -n flux-system
|
||||
|
||||
# Check Flux kustomization status for decryption errors
|
||||
flux get kustomizations
|
||||
kubectl describe kustomization cert-manager-issuer--infra -n flux-system
|
||||
```
|
||||
|
||||
**Gitea not starting?**
|
||||
```bash
|
||||
kubectl get pods -n gitea # Check pod status
|
||||
kubectl logs -n gitea -l app=gitea # Check Gitea logs
|
||||
kubectl logs -n gitea -l app=postgresql # Check DB logs
|
||||
```
|
||||
Reference in New Issue
Block a user