--- ![[IMG-My $130 Year Evernote Replacement - Paperelss-ngx + GitOps-20250825193543039.png|500]] --- I’ve been an Evernote user since 2010. Back then, a yearly subscription was about $45. It was indispensable while I was backpacking across Asia, giving me instant access to scanned passports, receipts, and tickets in India, Singapore, and Japan. Having critical documents on hand, anywhere in the world, was a lifesaver. ![[IMG-My $130 Year Evernote Replacement - Paperelss-ngx + GitOps-20250825193543070.png|400]] But over the years, the service changed. The app felt slower, features I relied on were deprecated, and the price climbed dramatically. Today, a personal plan costs nearly $130 a year. I decided there had to be a better way to regain control over my data and of course save money. I learned about **Paperless-ngx** through two main sources: [/r/selfhost](https://www.reddit.com/r/selfhosted/comments/1lg8ydc/whats_the_benefit_of_paperlessngx/) and also from coming across DB Tech’s video [Transform Your Chaos into Order: Quick Paperless-NGX Setup with Docker!](https://www.youtube.com/watch?v=2UdlUYi0bmk) **Paperless-ngx** is a powerful, private, and free alternative to Evernote. The initial setup doesn’t take much work but I had plans to automate things in the program such as automating backups and automating different configurations. --- ## The Setup Environment Over the years, I've gradually assembled my homelab with various devices. The NAS, for instance, was purchased to replace an outdated PLEX server that consisted of a 2010 Mac Mini and a tangled array of external hard drives. The upgrade became necessary when the Mac Mini could no longer run the latest software updates. Remarkably, the 2010 Mac Mini is still operational, though it's currently stored in a closet, awaiting its next project. ### The Brains (NAS) ![[IMG-My $130 Year Evernote Replacement - Paperelss-ngx + GitOps-20250825193543135.png|250]] * **Model:** QNAP TS-464 * **Specs:** Intel Celeron N5105 @ 2.00GHz, 16GB RAM, 32TB Storage * **Role:** Runs Docker containers, stores all documents, and executes scheduled automation scripts. ### The Workstation ![[IMG-My $130 Year Evernote Replacement - Paperelss-ngx + GitOps-20250825193543186.jpg|500]] * **Model:** Mac Mini M2 * **Role:** The development and management hub for writing code, managing the Git repository, and connecting to the NAS via SSH. * **Tools:** Visual Studio Code, Terminal. ### The Control Center ![[IMG-My $130 Year Evernote Replacement - Paperelss-ngx + GitOps-20250825193543223.png|500]] * **Service:** GitHub * **Role:** The single source of truth for all automation scripts and the engine for our **CI/CD pipeline** via GitHub Actions. --- ## Part I: Deploying Paperless-ngx ![[IMG-My $130 Year Evernote Replacement - Paperelss-ngx + GitOps-20250825193543289.png]] The QNAP comes with a "Container Station" application to install docker containers. I’ve sidestepped this application in favor of [Komodo](https://github.com/moghtech/komodo), which has a lot more options and has a much pretty UI. Docker compose files are hosted on a private GitHub repository that is then linked to Komodo. Komodo is able to pull docker-compose.yaml for `Paperless-ngx` and build and run it with no problems. It even allows for automatic docker image pull/updates. Here’s an example of the docker compose file I use with my setup. >[!note]- Paperless-ngx Docker Compose >```yaml ># paperless-ngx/compose.yaml ># Paperless-NGX configuration for Docker ># Self-hosted document management system for managing and archiving documents > >services: > broker: > image: docker.io/library/redis:8 > restart: unless-stopped > volumes: > - redisdata:/data > >db: > image: docker.io/library/postgres:17 > restart: unless-stopped > volumes: > - pgdata:/var/lib/postgresql/data > environment: > POSTGRES_DB: ${POSTGRES_DB} > POSTGRES_USER: ${POSTGRES_USER} > POSTGRES_PASSWORD: ${POSTGRES_PASSWORD} > healthcheck: > test: ["CMD-SHELL", "pg_isready -U $POSTGRES_USER -d $POSTGRES_DB"] > interval: 10s > timeout: 5s > retries: 5 > > webserver: > image: ghcr.io/paperless-ngx/paperless-ngx:latest > container_name: paperless-webserver > restart: unless-stopped > depends_on: > db: > condition: service_healthy # Waits for the database to be ready > broker: > condition: service_started > ports: > - "8000:8000" > volumes: > - ${CONTAINER_DIR}/data:/usr/src/paperless/data > - ${CONTAINER_DIR}/media:/usr/src/paperless/media > - ${CONTAINER_DIR}/export:/usr/src/paperless/export > - ${CONTAINER_DIR}/consume:/usr/src/paperless/consume > # env_file: docker-compose.env > environment: > # --- Connection Settings --- > PAPERLESS_REDIS: redis://broker:6379 > PAPERLESS_DBHOST: db > # Pull DB credentials > PAPERLESS_DBUSER: ${POSTGRES_USER} > PAPERLESS_DBPASS: ${POSTGRES_PASSWORD} > PAPERLESS_DBNAME: ${POSTGRES_DB} > > PAPERLESS_CONSUMER_POLLING: 60 > PAPERLESS_CONSUMER_USE_INOTIFY: false > PAPERLESS_URL: ${PAPERLESS_URL} > PAPERLESS_CSRF_TRUSTED_ORIGINS: ${PAPERLESS_CSRF_TRUSTED_ORIGINS} > > # PAPERLESS_FILENAME_FORMAT: "{created_year}-{created_month}-{created_day} - {correspondent} - {title} - " > > # --- User-defined Settings from Komodo --- > # Sets timezone, OCR language, and file ownership > PAPERLESS_TIME_ZONE: ${PAPERLESS_TIME_ZONE} > PAPERLESS_OCR_LANGUAGE: ${PAPERLESS_OCR_LANGUAGE} > USERMAP_UID: ${USERMAP_UID} > USERMAP_GID: ${USERMAP_GID} > healthcheck: > test: ["CMD-SHELL", "curl -f http://localhost:8000 || exit 1"] > interval: 30s > timeout: 10s > retries: 5 > > > networks: > - proxy-network # Connect to the same network as cloudflared > - default # Keep it on the default network for other paperless services if any > ># ... other services like Mealie, SQLpage, Paperless database, etc. > >networks: > proxy-network: > external: true > default: # Default network for internal communication > >volumes: > pgdata: > redisdata: > ``` --- ## Part II: The Backup System ![[IMG-My $130 Year Evernote Replacement - Paperelss-ngx + GitOps-20250825193856508.png|500]] Automating my paperless system required scripting, which raised concerns about the need for backup and version control. Without proper documentation and backups, troubleshooting these scripts becomes nearly impossible, as I tend to forget their existence once they are set up. The solution involves two core components: 1. **Shell Scripts:** A few simple shell scripts handle the logic. One script, `paperless_doc_exporter.sh`, executes the `document_exporter` command inside the Paperless container. A second script, `reclone_back.sh` uses [`rclone`](https://rclone.org/) to upload the backups to an offsite location like Google Drive and also prunes old backup copies. >[!note]- paperless_doc_exporter.sh >```bash >#!/bin/sh > > # This directive tells the linter the full path to the config file from the repo root > . "$(dirname "$0")/backup.conf" > > # --- Script Logic --- > echo "--- Starting Paperless Exporter Backup on $(date) ---" | tee -a "${EXPORT_LOG_FILE}" > > echo "Found container: ${WEBSERVER_CONTAINER_NAME}. Starting export..." | tee -a "${EXPORT_LOG_FILE}" > > # Execute the exporter using variables from the config file. > /share/CACHEDEV1_DATA/.qpkg/container-station/bin/docker exec "${WEBSERVER_CONTAINER_NAME}" \ > document_exporter ../export --zip --zip-name "paperless_full_export" > > echo "Export complete. Output file is paperless_full_export.zip" | tee -a "${EXPORT_LOG_FILE}" > echo "--- Export Backup Complete ---" | tee -a "${EXPORT_LOG_FILE}" >``` >[!note]- reclone_backup.sh >```bash >#!/bin/sh > ># This directive tells the linter the full path to the config file from the repo root ># shellcheck source=nas-automation/scripts/backup.conf >. "$(dirname "$0")/backup.conf" > ># --- Script Logic --- >echo "--- Starting Rclone Backup Process on $(date) ---" > ># --- SAFETY CHECK 1: Check if local backup file exists --- >if [ ! -f "$LOCAL_BACKUP_PATH" ]; then > echo "!!! DANGER: Local backup file not found at $LOCAL_BACKUP_PATH. Aborting." > exit 1 >fi > ># --- SAFETY CHECK 2: Check if local backup file is too small --- >CURRENT_SIZE=$(stat -c%s "$LOCAL_BACKUP_PATH") >if [ "$CURRENT_SIZE" -lt "$MINIMUM_SIZE_BYTES" ]; then > echo "!!! DANGER: New backup size ($CURRENT_SIZE bytes) is below the minimum threshold. Aborting." > exit 1 >fi > >echo "Local backup is valid (Size: $CURRENT_SIZE bytes). Proceeding with upload." > ># Get the Paperless-ngx version from the running container. >PAPERLESS_VERSION=$(docker exec "${WEBSERVER_CONTAINER_NAME}" /bin/sh -c "cat /app/VERSION" 2>/dev/null) >if [ -z "$PAPERLESS_VERSION" ]; then > PAPERLESS_VERSION="unknown" >fi >echo "Detected Paperless-NGX version: v${PAPERLESS_VERSION}" > ># Generate the new filename. >DATED_FILENAME="paperless_backup - $(date +'%Y-%m-%d') - v${PAPERLESS_VERSION}.zip" >echo "Uploading to remote file: ${GDRIVE_REMOTE}/${DATED_FILENAME}" > ># --- SAFETY CHECK 3: Check if the upload was successful --- ># SC2181: Check exit code directly instead of using $? >if ! /share/CACHEDEV1_DATA/.qpkg/container-station/bin/docker run --rm \ > -v /share/CACHEDEV1_DATA/Container/rclone/config:/config/rclone \ > -v /share/CACHEDEV1_DATA/Container/paperless:/data/paperless \ > rclone/rclone:latest copyto \ > /data/paperless/export/paperless_full_export.zip \ > "${GDRIVE_REMOTE}/${DATED_FILENAME}" -v; then > echo "!!! DANGER: rclone upload failed. Aborting cleanup to protect old backups." > exit 1 >fi > >echo "Upload complete. Now cleaning up old backups older than ${RETENTION_DAYS}..." > ># Use rclone's built-in delete functionality for efficiency and safety. >/share/CACHEDEV1_DATA/.qpkg/container-station/bin/docker run --rm \ > -v /share/CACHEDEV1_DATA/Container/rclone/config:/config/rclone \ > rclone/rclone:latest delete "${GDRIVE_REMOTE}" --min-age "${RETENTION_DAYS}" -v > >echo "Cleanup complete." >echo "--- Backup process finished successfully ---" >exit 0 >``` 2. **Persistent Scheduling:** QNAP systems present a unique challenge: `cron` jobs created via SSH do not persist after a reboot. The solution (see [Qnap: Running Your Own Application at Startup](https://www.qnap.com/en/how-to/faq/article/running-your-own-application-at-startup)) is to use a special boot script, `autorun.sh`, located on a persistent system partition. This script runs once at startup, verifies the schedule, and writes the correct `cron` jobs to the system's `crontab`, ensuring the automation is always active. >[!note]- autorun.sh >```bash >#!/bin/sh > ># The single source of truth for all backup-related paths and settings. >CONF_FILE="/share/CACHEDEV1_DATA/Container/homelab-scheduler/nas-automation/scripts/backup.conf" > ># --- SAFETY CHECK: Ensure config file exists before proceeding --- >if [ ! -f "$CONF_FILE" ]; then > # Since this runs at boot, logging to the system log is a good idea. > logger "AUTORUN ERROR: Backup config file not found at ${CONF_FILE}. Cannot update cron." > exit 1 >fi > ># shellcheck disable=SC1090 >. "$CONF_FILE" > ># --- Main Logic: Add/Update the crontab if our unique job ID isn't present --- >CRONTAB_FILE="/etc/config/crontab" > >if ! grep -q "$JOB_ID" "$CRONTAB_FILE"; then > logger "Updating Paperless-NGX backup schedule from config file..." > > # --- CLEANUP: Remove all known old versions of the job to ensure a clean state --- > sed -i '/paperless_export_backup.sh/d' "$CRONTAB_FILE" > sed -i '/rclone_paperless_backup.sh/d' "$CRONTAB_FILE" > sed -i '/gdrive paperless:PaperlessBackup/d' "$CRONTAB_FILE" > sed -i '/gdrive_encrypted/d' "$CRONTAB_FILE" > sed -i '/paperless_sync_gdrive.sh/d' "$CRONTAB_FILE" > sed -i '/# PAPERLESS_BACKUP/d' "$CRONTAB_FILE" # Removes old version markers > > # --- ADD NEW JOBS: Use a 'here document' for improved readability --- > # All paths and identifiers are now pulled directly from the backup.conf file. > cat << EOF >> "$CRONTAB_FILE" > >${JOB_ID_BLOCK} ># 1. Import new documents from GDrive every 15 minutes >*/15 * * * * ${REPO_PATH}/nas-automation/scripts/import_from_gdrive.sh >> ${LOG_BASE_PATH}/import_activity.log 2>&1 ># 2. Create the local backup archive every night at 2:05 AM >5 2 * * * ${EXPORT_SCRIPT_PATH} >> ${EXPORT_LOG_FILE} 2>&1 ># 3. Upload to GDrive and prune old backups at 3:05 AM >5 3 * * * ${RCLONE_SCRIPT_PATH} >> ${RCLONE_LOG_FILE} 2>&1 ># 4. Run the AI processing script every night at 4:05 AM >5 4 * * * PAPERLESS_TOKEN='${PAPERLESS_TOKEN}' /opt/QPython312/bin/python3 ${REPO_PATH}/nas-automation/scripts/process_ai.py >> ${LOG_BASE_PATH}/ai_process_activity.log 2>&1 >EOF > > > # Apply the new crontab and restart the cron daemon to load the new schedule. > crontab "$CRONTAB_FILE" > /etc/init.d/crond.sh restart > logger "Paperless-NGX backup schedule updated successfully." >else > # Log that the check was performed and no action was needed. > logger "Paperless-NGX backup schedule is already up-to-date. No changes made." >fi > >exit 0 >``` --- ## Part III: The "Pro" Upgrade - Managing Your System with GitOps and CI/CD This is where the project evolves from a simple set of scripts into a professional-grade system. The final repository is organized with a clear separation between the workflow definitions and the automation scripts themselves. The `deploy-on-nas.yml` acts as the main instructions that Github uses to process files. >[!note]- deploy-on-nas.yml >```yaml >name: CI Checks and Deploy to NAS > >on: > push: > branches: > - main > pull_request: > branches: > - main > >jobs: > # Job 1: Check all shell scripts for errors > shellcheck: > name: Lint Shell Scripts > runs-on: ubuntu-latest > steps: > - name: Checkout code > uses: actions/checkout@v4 > > - name: Run ShellCheck > uses: ludeeus/action-shellcheck@master > env: > # This tells ShellCheck to follow sourced files (-x) > SHELLCHECK_OPIS: "-x" > > # Job 2: Check for spelling errors > spellcheck: > name: Spell Check > runs-on: ubuntu-latest > steps: > - name: Checkout code > uses: actions/checkout@v4 > > - name: Run cspell > uses: streetsidesoftware/cspell-action@v6 > with: > # Assumes a cspell.json config file exists in the repo root > files: '**/*.{md,sh,yml}' > > # Job 3: Scan for any accidentally committed secrets > gitleaks: > name: Scan for Secrets > runs-on: ubuntu-latest > steps: > - name: Checkout code > uses: actions/checkout@v4 > with: > fetch-depth: 0 # Required to scan the entire history > > - name: Run Gitleaks > uses: gitleaks/gitleaks-action@v2 > > # Job 4: The deployment job (runs only if the above jobs succeed) > deploy: > name: Deploy to NAS > # This 'needs' clause is the key to the quality gate > needs: [shellcheck, spellcheck, gitleaks] > runs-on: self-hosted # This targets your on-premise runner > > steps: > - name: 1. Check out repository to the runner's workspace > uses: actions/checkout@v4 > > - name: 2. Sync files from workspace to the NAS directory > run: | > # The destination path should match the volume mount in your runner's docker-compose.yml > rsync -av --delete ${{ github.workspace }}/ /path/on/nas/to/your/scripts/ > > - name: 3. Generate Job Summary > if: success() > run: | > echo "### ✅ Deployment to NAS Successful" >> $GITHUB_STEP_SUMMARY > echo "The latest changes from commit \`[${{ github.sha }}](${{ github.server_url }}/${{ github.repository }}/commit/${{ github.sha }})\` have been synced." >> $GITHUB_STEP_SUMMARY >``` >[!note]- folder structure >```txt >homelab-scheduler/ >├── .github/ >│ └── workflows/ >│ └── deploy-on-nas.yml >├── nas-automation/ >│ └── scripts/ >│ ├── autorun.sh >│ ├── backup.conf >│ └── ... (other scripts) ... >├── cspell.json >└── README.md >``` ### Introducing GitOps ![[IMG-My $130 Year Evernote Replacement - Paperelss-ngx + GitOps-20250825193543417.png|500]] The core principle of GitOps is treating your infrastructure configuration as code. Instead of SSHing into the NAS to edit a script, you make changes in your local Git repository and push them to GitHub. The system then automatically updates itself to match the "desired state" defined in the repository. For my setup I simply edit files locally in VS Code the push the changes to Github. ### The CI/CD Pipeline This automation is handled by a **GitHub Actions** workflow. * **The Self-Hosted Runner:** A key component is a self-hosted runner, a small agent that runs in a Docker container on the NAS. It securely connects *outbound* to GitHub to listen for jobs. This is highly secure, as it requires no open firewall ports on your home network. * **The Workflow:** The process is defined in `.github/workflows/deploy-on-nas.yml` and consists of two stages: 1. **CI (Continuous Integration):** When you push a change, a series of "quality gate" jobs run first in the cloud. These jobs use tools like `shellcheck` to find bugs in scripts and `gitleaks` to scan for accidentally committed secrets. If any of these checks fail, the process stops. 2. **CD (Continuous Deployment):** If all CI checks pass, the final `deploy` job is sent to your self-hosted runner on the NAS. This job uses `rsync` to sync the validated files from the repository to the active script directory on the NAS. The result is a push-to-deploy system. A `git push` from the Mac automatically and safely updates the scripts running on the NAS. ## Conclusion Paperless-ngx now automatically backs up every night and pushes copies to a remote location. The next step is to use AI to automate the creation of note titles and other metadata.