Architecture

Overview

pkgprobe is structured as a Cargo workspace with a single business-logic crate (core) consumed by three thin shells: a CLI, a C FFI layer, and a native macOS SwiftUI app. The GUI links the FFI layer statically — there is no daemon, no socket, and no IPC.

Dependency discovery runs entirely in-process via native Rust catalogers — there is no bundled external scanner and no subprocess. pkgprobe layers its own value on top: registry-provenance validation, OSV vulnerability enrichment, policy enforcement, license conflict detection, audit logging, and a native macOS GUI.

pkgprobe/
├── core/       # Discovery dispatch, mapping, enrichment, policy, SBOM
├── ffi/        # C ABI wrapper — produces libpkgprobe_ffi.a + .dylib
├── cli/        # clap CLI — delegates entirely to core
└── gui-macos/  # SwiftUI app — links ffi statically, calls core in-process

---

System architecture

graph TB
    %% ── End users ──────────────────────────────────────────────────
    DEV([Developer])
    SEC([Security Team])
    CI([CI Runner])

    %% ── Desktop & CLI ──────────────────────────────────────────────
    subgraph desktop["Desktop App (macOS)"]
        GUI[SwiftUI GUI]
        FFI[pkgprobe-ffi]
        CORE["pkgprobe-core\n(native catalogers, in-process)"]
        GUI --> FFI --> CORE
    end

    subgraph cli_box["CLI"]
        CLI[pkgprobe-cli]
        CLI --> CORE
    end

    DEV --> GUI
    SEC --> GUI
    CI --> CLI

    %% ── Self-hosted infrastructure (docker-compose) ────────────────
    subgraph infra["Self-Hosted Infrastructure (docker-compose)"]
        subgraph services["Application Services"]
            LICENSE["license-server\n:8083\nLicense mgmt + LS webhooks"]
            VP["vuln-proxy\n:8082\nOSV caching gateway"]
            VP_CACHE[(In-Memory Cache)]
            VP --> VP_CACHE
            FLEET["fleet\n:8081\nDashboard + API"]
        end

        subgraph storage["Storage"]
            PG[(PostgreSQL\n:5432)]
            SEAWEED["SeaweedFS\n:8333 S3 API\n:8888 Filer\nRelease artifacts +\nSparkle appcast"]
        end

        LICENSE --> PG
        FLEET --> PG
        FLEET -->|"license ops\n(Bearer admin key)"| LICENSE
    end

    CORE -->|"POST /api/v1/licenses/validate-key\nPOST /machines\nPOST /checkout"| LICENSE
    CORE -->|"POST /api/v1/query\n(Bearer auth)"| VP
    CORE -->|"POST /api/v1/report\n(Bearer auth)"| FLEET
    SEC -->|HTMX Web UI| FLEET

    %% ── External APIs ──────────────────────────────────────────────
    subgraph external["External APIs"]
        OSV["OSV API\nosv.dev"]
        CLERK["Clerk\n(SSO/JWT, optional)"]
    end

    VP -->|"POST /v1/querybatch\nGET /v1/vulns/{id}"| OSV
    FLEET -.->|"JWKS + webhooks\n(optional)"| CLERK

    %% ── Billing ────────────────────────────────────────────────────
    LEMON["Lemon Squeezy\n(payments)"]
    LEMON -->|"POST /webhook/lemonsqueezy\norder/subscription events"| LICENSE

    %% ── Alerting destinations ──────────────────────────────────────
    subgraph alerts["Alerting Destinations"]
        SLACK["Slack\nIncoming Webhook"]
        JIRA["Jira Cloud\nREST API"]
        SMTP["SMTP\nEmail Server"]
        WEBHOOK["Generic Webhooks\n(HMAC-signed)"]
    end

    CORE -->|Block Kit JSON| SLACK
    CORE -->|"Basic Auth\nCreate issues"| JIRA
    CORE -->|STARTTLS| SMTP
    CORE -->|"HMAC-SHA256\nX-PkgProbe-Signature"| WEBHOOK

    %% ── Distribution & Updates ─────────────────────────────────────
    subgraph dist["Distribution & Updates"]
        GH_REL["GitHub Releases\n(backup)"]
        BREW["Homebrew Tap\npkgprobe/homebrew-tap"]
        SPARKLE["Sparkle Appcast\nappcast.xml\n(Ed25519 signed)"]
        SCAN_ACTION["GitHub Action\npkgprobe/scan-action"]
    end

    SEAWEED -->|DMG download| GUI
    SEAWEED -->|CLI binary| CLI
    SEAWEED -->|appcast.xml| SPARKLE
    SPARKLE -->|auto-update check| GUI
    BREW -->|"brew install"| CLI
    BREW -->|"brew install --cask"| GUI
    CI -->|"uses: pkgprobe/scan-action"| SCAN_ACTION
    SCAN_ACTION -->|downloads CLI from| SEAWEED

    %% ── CI/CD Workflows ────────────────────────────────────────────
    subgraph cicd["CI/CD (GitHub Actions)"]
        CI_WF["ci.yml\nfmt + clippy + test\n+ Go vet/test"]
        REL_WF["release.yml\nbuild + sign + notarize\n+ upload + tap update"]
    end

    REL_WF -->|"upload via S3 API"| SEAWEED
    REL_WF -->|create release| GH_REL
    REL_WF -->|update formulae| BREW
    REL_WF -->|generate| SPARKLE

    %% ── Local storage ──────────────────────────────────────────────
    subgraph local["Local Storage (~/.pkgprobe/)"]
        CONFIG["config.yaml"]
        VCACHE["vuln-cache/\n(per-PURL SHA-256)"]
        SCACHE["scan-cache.json"]
        AUDIT["logs/audit.jsonl"]
        LIC_FILE["license.json\n(Ed25519 signed)"]
    end

    CORE --> CONFIG
    CORE --> VCACHE
    CORE --> SCACHE
    CORE --> AUDIT
    CORE --> LIC_FILE

    %% ── Admin policy ──────────────────────────────────────────────
    ADMIN_POLICY["/Library/Application Support/pkgprobe/policy.yaml\n(admin-managed, overrides user config)"]
    CORE -->|merge at scan time| ADMIN_POLICY

---

Docker infrastructure

All backend services run via a single docker-compose.yml at the repo root:

docker-compose up -d

Service	Port	Purpose
postgres	5432	Shared PostgreSQL (separate DBs: license_server, fleet)
license-server	8083	License management, Ed25519 signing, Lemon Squeezy webhooks
vuln-proxy	8082	OSV vulnerability proxy with shared in-memory cache
fleet	8081	Enterprise security posture dashboard
seaweedfs-master	—	SeaweedFS topology master
seaweedfs-volume	—	SeaweedFS data storage
seaweedfs-filer	8333 (S3), 8888 (HTTP)	S3-compatible API for release artifacts + Sparkle appcast

---

Data flow: scan with vulnerabilities

sequenceDiagram
    participant Caller as SwiftUI / CLI
    participant FFI as pkgprobe-ffi
    participant Lib as lib.rs
    participant Roots as project_roots
    participant Cats as catalogers
    participant Reg as registry parsers
    participant VCache as vuln-cache (disk)
    participant Proxy as vuln-proxy
    participant ProxyCache as proxy cache (memory)
    participant OSV as osv.dev

    Caller->>FFI: pkgprobe_scan_with_vulns(config_path, ttl)
    FFI->>Lib: scan_with_vulns()
    Lib->>Lib: Config::load()
    Lib->>Roots: find_project_roots(scan_path)
    Roots-->>Lib: Vec<DiscoveredProject>
    loop each project root
        Lib->>Cats: default_registry().dispatch(root)
        Cats->>Cats: run matching catalogers (tier precedence), merge + dedupe
        Cats-->>Lib: PackageMetadata
        Lib->>Reg: attach_ecosystem_registries(meta, root)
        Reg-->>Lib: meta.registries populated
    end
    Lib->>VCache: check per-PURL entries (TTL)
    VCache-->>Lib: hits (skip) / misses
    Lib->>Proxy: POST /api/v1/query (cache misses)
    Proxy->>ProxyCache: check in-memory cache
    ProxyCache-->>Proxy: hits / misses
    Proxy->>OSV: POST /v1/querybatch (Phase 1)
    OSV-->>Proxy: vuln IDs per package
    Proxy->>OSV: GET /v1/vulns/{id} (Phase 2, concurrent)
    OSV-->>Proxy: full vuln records
    Proxy->>Proxy: map severity (CVSS v3) + filter fix versions
    Proxy->>ProxyCache: write results
    Proxy-->>Lib: ProxyResponse {results, errors}
    Lib->>VCache: write per-PURL cache entries
    Lib->>Lib: evaluate_policy()
    Lib-->>FFI: ScanEnvelope as JSON
    FFI-->>Caller: *char (heap-allocated UTF-8 JSON)

---

Alerting flow

sequenceDiagram
    participant Core as pkgprobe-core
    participant Diff as diff.rs
    participant Fleet as fleet service
    participant Slack as Slack
    participant Jira as Jira Cloud
    participant Email as SMTP Server
    participant WH as Generic Webhooks

    Core->>Core: scan_with_vulns() completes
    Core->>Diff: compare with previous scan
    Diff-->>Core: new vulns, resolved vulns, policy changes

    par Alert dispatch
        Core->>Fleet: POST /api/v1/report (ScanReport, Bearer auth)
        Core->>Slack: POST webhook_url (Block Kit JSON)
        Core->>Jira: POST /rest/api/2/issue (deduplicated, Basic auth)
        Core->>Email: SMTP STARTTLS (HTML vulnerability table)
        Core->>WH: POST url (HMAC-SHA256 signed)
    end

---

Component diagram

graph TD
    subgraph gui-macos
        UI[SwiftUI Views]
        Store[PkgProbeStore\n@MainActor ObservableObject]
        UI --> Store
    end

    subgraph ffi["pkgprobe-ffi (staticlib + cdylib)"]
        FFI["extern C\npkgprobe_scan()\npkgprobe_scan_with_vulns()\npkgprobe_load_cached()\npkgprobe_generate_sbom()\npkgprobe_free_string()"]
    end

    subgraph cli["pkgprobe-cli (clap)"]
        CLI["main.rs\nscan / sbom / fix / config\ntrial / login / logout"]
    end

    subgraph core["pkgprobe-core"]
        LIB[lib.rs\nscan_from_config()\nscan_with_vulns()\npreview_fixes()\napply_fix()]
        ROOTS[project_roots.rs\nfind project directories]
        CATS[catalogers.rs\nCataloger trait + dispatch\n(~20 native catalogers)]
        REG[registry parsers\n.npmrc / NuGet.Config /\ncomposer.json / auth.json]
        VULN[vuln.rs\nproxy client]
        POLICY[policy.rs\nevaluate rules]
        CACHE[scan_cache.rs]
        SBOM[sbom.rs\nhand-rolled CycloneDX 1.6\n+ SPDX 2.3]
        DOTNET[dotnet.rs\n.csproj fallback]

        LIB --> ROOTS
        ROOTS --> CATS
        CATS -->|PackageMetadata| REG
        REG --> VULN
        VULN --> POLICY
        LIB --> CACHE
        LIB --> SBOM
        CATS -.->|.NET, no lockfile| DOTNET
    end

    Store -->|Task.detached| FFI
    CLI --> LIB
    FFI --> LIB

---

FFI boundary

The macOS GUI links libpkgprobe_ffi.a (static archive) at compile time. The C header is auto-generated by cbindgen:

// All returned strings are heap-allocated UTF-8 JSON.
// Caller must free with pkgprobe_free_string().
char    *pkgprobe_scan(const char *config_path);
char    *pkgprobe_scan_with_vulns(const char *config_path, uint64_t cache_ttl_hours);
char    *pkgprobe_load_cached(void);
char    *pkgprobe_read_config(const char *config_path);
uint8_t  pkgprobe_write_config(const char *config_path, const char *json);
char    *pkgprobe_generate_sbom(const char *project_name, const char *format);
char    *pkgprobe_check_pm_available(const char *project_path);
char    *pkgprobe_install_deps(const char *project_path);
char    *pkgprobe_preview_fix(const char *project_path, const char *cve_id);
char    *pkgprobe_apply_fix(const char *project_path, const char *dep_name, const char *target_version);
void     pkgprobe_free_string(char *ptr);

• Passing NULL as config_path uses ~/.pkgprobe/config.yaml.
• pkgprobe_generate_sbom: format is "cyclonedx" (CycloneDX 1.6 JSON) or "spdx" (SPDX 2.3 JSON). Built by hand-rolled writers directly from the PackageMetadata in the on-disk scan cache — no external generator.

On macOS the GUI calls FFI from a Task.detached block so the main thread stays free while scanning. The returned JSON is decoded in Swift into ScanEnvelope -> [ProjectSummary] (sidebar) and [String: ProjectDetails] (detail panel).

---

Discovery details

Project-root detection

discovery::project_roots::find_project_roots walks the configured scan paths, skipping SKIP_DIRS (node_modules, .git, vendor, target, etc.), and emits one DiscoveredProject per directory containing a recognized marker file (package.json, Cargo.toml, pom.xml, go.mod, pubspec.yaml, etc.).

Native cataloger dispatch

For each project root, discovery::catalogers::default_registry().dispatch(root) runs every cataloger whose matches(root) returns true — all in-process, no subprocess. Tier precedence (Lockfile > InstallTreeOnly > Naked) suppresses lower tiers when a higher one matches the same root, and the output is a single PackageMetadata per root with merged, de-duplicated dependencies. JS devDeps are included by default. See ADR-0001 for the migration retrospective.

Registry augmentation

attach_ecosystem_registries in lib.rs reads per-ecosystem config files and attaches RegistryConfig records to the PackageMetadata. The catalogers focus on dependency extraction; registry resolution is pkgprobe’s owned layer on top. See the Registry Conventions page for the full spec.