LeadWin System
    • Home
    • Projects
    • Tech Stack
    • Skills
    • About
    • Contact
    • Peyvor
    • BRO
    • VideoLab
    • Baguio Outage Radar
    • Systems & Infrastructure
    • Toggle light or dark theme
    • GitHub organization (opens in a new tab)

Up Down to move, Enter to open, Esc to close

Operations practice case study

Systems & Infrastructure

How I approach running software: hardened services, scripted installs, rollback-first change management and quiet, useful alerting. Qualitative, drawn from the other projects.

Status
Practice summary. Not a single deployment.
Source
Source: private

The problem

Small systems fail in ordinary ways: a bad release, a crashed process, a feed that goes quiet, a restart that quietly changes behaviour.

The aim is a repeatable way to change things safely and to know when something stops working.

The approach

Services run as unprivileged users under a service manager that restarts them, with write access limited to one data directory.

Releases are immutable directories. A symlink points at the live one, so switching forward or back is one atomic step.

Every change starts with a backup and a recorded state, such as deployed file hashes, so that a rollback path exists before the change.

Windows services are handled with scheduled tasks that have a restart policy, scripted deployment from a manifest, and restarts kept as a separate, deliberate step.

Monitoring is deliberately small: an alert after repeated failures, a notice when things recover, and logs that make a dry run reviewable.

Architecture

Rollback-first release flowA release is built, backed up against, uploaded as a new directory, verified, and activated by switching a symlink. Rollback switches the symlink back to the previous directory.Build from a git reftests and checks firstBackup + record statehashes, data backupUpload new release dirimmutable, timestampedSwitch symlinkatomic activateVerifylogs, health, dry-run outputRollbackpoint symlink at previous
Solid outline: component. Amber fill: control or guard. Dashed outline: external input or output. Dotted outline: deliberately blocked.
Text alternative: the flow in words
  1. Build from a specific git ref after tests and checks pass.
  2. Take a backup and record the current state before changing anything.
  3. Upload the new release into its own timestamped directory.
  4. Activate it by switching a symlink in one atomic step.
  5. Verify with logs and health checks. If anything is wrong, point the symlink back at the previous release.

Key technical decisions

  • Rollback-first

    A change is not ready until the way back is written and, where possible, tested. Recovery artifacts are recorded next to the release.

  • Dry run before live

    New automation runs without side effects first. Its output is read by a person before anything is switched on.

  • Separate deploy from restart

    Copying files and restarting processes are different steps with different approvals, which keeps a deployment from changing behaviour by surprise.

  • Least privilege by default

    Loopback-only listeners, key-only access, no secrets in repositories or CI, and read-only CI tokens.

  • Static, boring web hosting

    This site is built to static files and designed for a minimal web server with a strict content security policy and atomic releases. The deploy files in its repository are written but not yet used.

Safety and security notes

  • Secrets live outside repositories. Release signing keys, bot tokens and passwords are never committed.
  • Continuous integration uses a read-only token and has no deploy step.
  • No hostnames, addresses, accounts or employer details appear here.

Limits and honest caveats

  • Familiar with, but not claimed as project evidence: Cloudflare services, n8n workflow automation, Docker and Tailscale. See the tech stack page for evidence status.