commit 86901d816e82792dd59bb01344240d566efa2741 Author: Tony Date: Thu Aug 6 19:21:08 2026 +0200 init: specifications diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 0000000..9675ba9 --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,38 @@ +# Instructions for Codex and automated contributors + +Read `README.md` and the relevant documents under `docs/` before changing implementation or contracts. + +## Non-negotiable rules + +1. Preserve the product invariants in `README.md`. +2. Do not give the main application direct Docker-socket access. +3. Do not implement integrations as native plugins, host executables, scripts, or sidecar containers. Integrations are WebAssembly adapters only. +4. Do not add arbitrary command execution, arbitrary Docker API proxying, arbitrary host paths, or unrestricted network access. +5. Treat templates and module manifests as untrusted input. Validate them against `specs/*.schema.json` before persistence or execution. +6. Keep secrets out of API responses, logs, audit payloads, exports, error messages, and test fixtures. +7. Preserve player data and backups by default in every deletion, update, restore, and migration workflow. +8. Keep `compose.yaml` minimal. Product settings belong in the database and web interface unless they are bootstrap secrets, bind roots, or network/listen settings required before startup. +9. SQLite is the V1 database. Do not introduce an external database, message broker, Kubernetes, or distributed-node design without an accepted architecture decision. +10. Palworld is the reference integration. Any contract change affecting templates, modules, backups, permissions, or instance lifecycles must be checked against both Palworld examples. + +## Change workflow + +- Locate the normative document first. +- State assumptions when requirements are ambiguous; do not silently invent security-sensitive behavior. +- Update documentation, schema, example, implementation, and tests together when a contract changes. +- Prefer small Go packages with explicit interfaces and dependency direction. +- Add migrations for persisted data changes. Never edit an already released migration. +- Use deterministic serialization and stable identifiers. +- Validate JSON Schemas and YAML examples in automated checks. +- Add negative tests for authorization, path validation, module capabilities, archive extraction, and agent operation scope. +- Report what was validated and what still needs physical or integration testing. + +## Definition of done for a change + +- Relevant requirements and acceptance criteria are satisfied. +- Authorization is enforced in the backend, not only hidden in the UI. +- Audit and notification behavior is deliberate. +- Failure and rollback behavior is covered. +- Documentation and machine-readable examples agree. +- Tests cover success, denial, and interruption paths. + diff --git a/README.md b/README.md new file mode 100644 index 0000000..6d05579 --- /dev/null +++ b/README.md @@ -0,0 +1,79 @@ +# DoGaMa + +DoGaMa is a lightweight, self-hosted manager for private game servers running as Docker containers. It is designed for families and small groups of friends, not for commercial hosting or general Docker administration. + +This repository currently contains the normative product and engineering specification. Implementation must follow the documents and machine-readable contracts linked below. + +## Product invariants + +- DoGaMa only displays and operates game-server containers that it created or explicitly adopted through a controlled administrator workflow. +- The main application never mounts the Docker socket. A separate, private, restricted agent is the only component allowed to reach Docker. +- The main application is a small Go service with an embedded web UI and SQLite. +- Game-specific integrations are exclusively lightweight WebAssembly adapters. They never run as privileged host processes or sidecar containers. +- A module may contact only the API endpoint of its assigned instance, through host-provided functions and declared ports. +- Templates are declarative, versioned YAML documents validated against a JSON Schema. +- Almost all operational configuration is performed in the web interface. `compose.yaml` only bootstraps DoGaMa itself. +- Secrets are never returned after submission, logged, audited, or included in normal exports. +- Destructive operations preserve player data and backups by default. + +## Documentation map + +### Product + +- [Vision and scope](docs/product/vision-and-scope.md) +- [V1 acceptance criteria](docs/product/acceptance-criteria.md) +- [Roadmap](docs/product/roadmap.md) + +### Architecture and domain + +- [System architecture](docs/architecture/system-architecture.md) +- [Main application](docs/architecture/main-application.md) +- [Restricted Docker agent](docs/architecture/docker-agent.md) +- [WebAssembly module runtime](docs/architecture/wasm-modules.md) +- [Data model](docs/domain/data-model.md) +- [Roles and permissions](docs/domain/authorization.md) +- [Instance lifecycle](docs/domain/instance-lifecycle.md) + +### Operations and security + +- [Backups, import, restore and export](docs/operations/backups-import-export.md) +- [Resources, ports, storage, mods and updates](docs/operations/instance-operations.md) +- [Notifications and audit](docs/operations/notifications-and-audit.md) +- [Security and threat model](docs/security/security-and-threat-model.md) +- [Administration and manager interfaces](docs/ux/interfaces.md) + +### Contributor contracts + +- [Development conventions](docs/contributing/development.md) +- [AI and Codex contributor guide](docs/contributing/ai-codex-guide.md) +- [Template schema](specs/template.schema.json) +- [Module manifest schema](specs/module-manifest.schema.json) +- [Normalized module API](specs/normalized-module-api.md) +- [Palworld reference template](catalog/palworld/template.yaml) +- [Palworld reference module manifest](modules/palworld-rest/manifest.yaml) + +## Intended deployment + +```text +Browser + | + v +DoGaMa main application ---- SQLite / catalog / backups + | + | private authenticated API + v +Restricted Docker agent ---- Docker socket + | + v +Managed game-server containers +``` + +Only the main application's HTTP port is published. The agent and game-management APIs remain on private Docker networks. Individual game ports are published by the managed instances according to approved templates and administrator configuration. + +## Status + +Specification baseline. The roadmap and V1 acceptance criteria define the implementation order and completion boundary. + +## Validate the specification + +Install the temporary validation dependencies from `tools/requirements-validation.txt`, then run `python tools/validate_spec.py`. The check validates both JSON Schemas, YAML examples, cross-referenced ports/mounts/capabilities, packaged-asset checksums, JSON fixtures, requirement coverage and internal Markdown links. diff --git a/catalog/palworld/README.md b/catalog/palworld/README.md new file mode 100644 index 0000000..d644cca --- /dev/null +++ b/catalog/palworld/README.md @@ -0,0 +1,21 @@ +# Palworld reference template + +This template demonstrates every important V1 contract: official image, public UDP game port, private REST integration port, persistent save data, write-only credentials, online-save backup, import and an independently versioned WebAssembly adapter. + +## Verified upstream facts + +- Pocketpair publishes the official image and example Compose deployment at [pocketpairjp/palworld-dedicated-server-docker](https://github.com/pocketpairjp/palworld-dedicated-server-docker). +- The current official example at specification time uses `ghcr.io/pocketpairjp/palserver:v1.0.2.101103`, UDP 8211, `/pal/Package/Pal/Saved` and the packaged `helper.sh` pattern. +- The [official requirements](https://docs.palworldgame.com/0.7.3/getting-started/requirements/) specify four or more CPU cores, 16 GB memory with over 32 GB recommended for stability, and UDP 8211. The template's 8 GB minimum is an explicit lower bootable boundary mentioned upstream, not a stability recommendation; the UI must warn below 16 GB. +- The [configuration reference](https://docs.palworldgame.com/settings-and-operation/configuration/) documents `AdminPassword`, `RESTAPIEnabled`, `RESTAPIPort`, server name/password and maximum players. +- The [REST API documentation](https://docs.palworldgame.com/category/rest-api/) documents information, players, metrics, announce, save, shutdown and moderation operations. REST API credentials and port must remain private. + +The image tag and upstream API may change. Catalog maintainers must verify and release a new immutable template version; existing instances remain pinned. + +## Reference limitations + +- Artwork URLs are intentionally absent until a catalog maintainer selects suitable source files and attribution. DoGaMa caches and converts accepted raster artwork locally. +- The schema's storage sizes are conservative product defaults because upstream specifies SSD performance but not a fixed disk-size requirement. +- Local hosted-world migration may require player identity conversion. The generic V1 importer detects structure and warns; it does not silently convert identities. +- The checked-in module manifest is a source example. It becomes installable only after `module.wasm` is built and its real SHA-256 replaces the all-zero placeholder. + diff --git a/catalog/palworld/assets/helper.sh b/catalog/palworld/assets/helper.sh new file mode 100644 index 0000000..2f0ee07 --- /dev/null +++ b/catalog/palworld/assets/helper.sh @@ -0,0 +1,3 @@ +#!/bin/sh +sudo chown -R user:usergroup /pal/Package/Pal/Saved +exec /bin/sh /pal/Package/PalServer.sh "$@" diff --git a/catalog/palworld/template.yaml b/catalog/palworld/template.yaml new file mode 100644 index 0000000..cd1e474 --- /dev/null +++ b/catalog/palworld/template.yaml @@ -0,0 +1,198 @@ +schema_version: 1 +id: palworld-official +version: 1.0.0 + +source: + type: official + url: https://github.com/pocketpairjp/palworld-dedicated-server-docker + +game: + id: palworld + name: Palworld + description: Official Palworld dedicated server reference for DoGaMa. + website: https://www.palworldgame.com/ + artwork: + attribution: Palworld and related artwork are property of Pocketpair, Inc.; no affiliation is implied. + +requirements: + minimum: + cpu_cores: 4 + memory_mb: 8192 + storage_gb: 30 + recommended: + cpu_cores: 4 + memory_mb: 32768 + storage_gb: 50 + +container: + image: ghcr.io/pocketpairjp/palserver + tag: v1.0.2.101103 + entrypoint: + - /pal/helper.sh + arguments: + - -port=8211 + - -useperfthreads + - -NoAsyncLoadingThread + - -UseMultithreadForDS + assets: + - source: assets/helper.sh + destination: /pal/helper.sh + sha256: 52e58fe4e22654d0d312fe2bb6195db61dad5ffce78e0440a951d33d20f2c36f + read_only: true + stop_timeout_seconds: 120 + ports: + - id: game + container_port: 8211 + protocol: udp + purpose: game + publish: true + required: true + - id: rest_api + container_port: 8212 + protocol: tcp + purpose: integration + publish: false + required: true + +storage: + mounts: + - id: saved + container_path: /pal/Package/Pal/Saved + category: player_data + backup: true + user_configurable: true + read_only: false + +configuration: + fields: + - id: server_name + label: Server name + type: string + target: + kind: ini + name: ServerName + apply: restart_required + visibility: members + required: true + default: DoGaMa Palworld Server + - id: server_description + label: Server description + type: string + target: + kind: ini + name: ServerDescription + apply: restart_required + visibility: members + required: false + default: Managed by DoGaMa + - id: max_players + label: Maximum players + type: integer + target: + kind: ini + name: ServerPlayerMaxNum + apply: restart_required + visibility: members + required: true + default: 16 + minimum: 1 + maximum: 32 + - id: server_password + label: Player password + type: secret + target: + kind: ini + name: ServerPassword + apply: restart_required + visibility: secret + required: false + - id: admin_password + label: Administrator and REST API password + type: secret + target: + kind: ini + name: AdminPassword + apply: restart_required + visibility: secret + required: true + - id: rest_api_enabled + label: Enable private REST API + type: boolean + target: + kind: ini + name: RESTAPIEnabled + apply: restart_required + visibility: admin + required: true + default: true + - id: rest_api_port + label: Private REST API port + type: integer + target: + kind: ini + name: RESTAPIPort + apply: restart_required + visibility: admin + required: true + default: 8212 + minimum: 1 + maximum: 65535 + +capabilities: + - server_info + - metrics + - player_list + - online_save + - graceful_shutdown + - announcement + - kick + - ban + - unban + +integration: + module_id: palworld-rest + version_range: ">=1.0.0 <2.0.0" + port_id: rest_api + required: false + +backup: + strategy: online_save + source_mounts: + - saved + restart_after_backup: true + +healthcheck: + type: module + port_id: rest_api + startup_timeout_seconds: 300 + interval_seconds: 10 + +mods: + supported: false + +imports: + supported: true + accepted_formats: + - zip + - tar + - tar.gz + - tar.zst + - directory + max_extracted_size_gb: 50 + required_paths: + - Level.sav + - Players + destination_mount: saved + destination_relative_path: SaveGames/0 + requires_stopped_server: true + +updates: + backup_before_update: true + automatic_default: false + rollback_on_failure: true + health_timeout_seconds: 300 + +compatibility: + minimum_manager_version: 1.0.0 + requires_instance_migration: false + diff --git a/compose.yaml b/compose.yaml new file mode 100644 index 0000000..9faa839 --- /dev/null +++ b/compose.yaml @@ -0,0 +1,54 @@ +services: + dogama: + image: ghcr.io/dogama/dogama:${DOGAMA_VERSION:-latest} + restart: unless-stopped + ports: + - "${DOGAMA_HTTP_PORT:-8080}:8080" + environment: + TZ: ${TZ:-UTC} + DOGAMA_AGENT_URL: http://agent:8081 + DOGAMA_AGENT_TOKEN_FILE: /run/secrets/agent_token + DOGAMA_MASTER_KEY_FILE: /run/secrets/master_key + secrets: + - agent_token + - master_key + volumes: + - ./data:/var/lib/dogama + - ${DOGAMA_SERVERS_ROOT:-/srv/game-servers}:/srv/game-servers + - ${DOGAMA_BACKUPS_ROOT:-/srv/game-backups}:/srv/game-backups + networks: + - frontend + - control + - games + depends_on: + - agent + + agent: + image: ghcr.io/dogama/dogama-agent:${DOGAMA_VERSION:-latest} + restart: unless-stopped + environment: + TZ: ${TZ:-UTC} + DOGAMA_AGENT_TOKEN_FILE: /run/secrets/agent_token + DOGAMA_ALLOWED_SERVER_ROOT: /srv/game-servers + DOGAMA_ALLOWED_BACKUP_ROOT: /srv/game-backups + secrets: + - agent_token + volumes: + - /var/run/docker.sock:/var/run/docker.sock + - ${DOGAMA_SERVERS_ROOT:-/srv/game-servers}:/srv/game-servers + - ${DOGAMA_BACKUPS_ROOT:-/srv/game-backups}:/srv/game-backups + networks: + - control + - games + +networks: + frontend: + control: + internal: true + games: + +secrets: + agent_token: + file: ./secrets/agent_token + master_key: + file: ./secrets/master_key diff --git a/docs/architecture/docker-agent.md b/docs/architecture/docker-agent.md new file mode 100644 index 0000000..955d0c4 --- /dev/null +++ b/docs/architecture/docker-agent.md @@ -0,0 +1,72 @@ +# Restricted Docker agent + +## Security objective + +The agent reduces the chance that an application bug becomes arbitrary Docker control. Because Docker socket access is effectively host-level privilege, the agent is small, independently testable and deny-by-default. + +## Allowed V1 operations + +- `CreateInstance(plan)` +- `StartInstance(instance_id)` +- `StopInstance(instance_id, timeout)` +- `RestartInstance(instance_id, timeout)` +- `ReplaceInstance(plan, expected_revision)` for configuration/update recreation +- `DeleteContainer(instance_id)` without deleting host data +- `InspectInstance(instance_id)` +- `GetInstanceStats(instance_id)` +- `CheckPorts(bindings)` without disclosing unrelated container details +- `CheckDisk(paths)` +- `ListRegisteredInstances()` only + +There is no generic Docker request, arbitrary command, arbitrary `exec`, arbitrary container ID, image-build endpoint, host filesystem browser or list-all-containers endpoint. + +## Registration binding + +Each managed container has labels such as: + +```text +io.dogama.managed=true +io.dogama.instance-id= +io.dogama.template-id= +io.dogama.template-version= +io.dogama.plan-digest= +``` + +Labels alone are insufficient. The agent keeps a durable registry of instance ID, expected container identity and plan digest, authenticated by an agent-local key or MAC. An operation succeeds only when request, registry and inspected labels agree. + +## Deployment-plan validation + +Before create or replace, the agent verifies: + +- instance and template identifiers have valid syntax; +- image reference matches the canonical approved plan and preferably a resolved digest; +- command, entrypoint, capabilities, devices and security options exactly match allowed template fields; +- no privileged mode, host PID/IPC/network namespace, device mount or Docker socket mount; +- every bind source resolves below an allowed root after symlink-aware canonicalization; +- every container destination is declared by the template; +- port protocols and container ports match the template and host ports do not conflict; +- resource limits are present and within administrator limits; +- labels use the reserved namespace and cannot be overridden; +- only approved DoGaMa networks are attached. + +V1 templates do not expose arbitrary Docker security options. The agent applies a secure fixed baseline: no-new-privileges where compatible, dropped capabilities by default, bounded PIDs and a non-host network mode. + +## Authentication and replay defense + +Requests use a shared secret read from a Docker secret file. Sign method, path, body digest, timestamp and nonce. Reject clock-skewed or reused nonces. Use constant-time comparison, small body limits and short timeouts. Rotate the token through an explicit maintenance workflow. + +The agent listens only on the internal control network and publishes no host port. Authentication remains mandatory even on that network. + +## Failure semantics + +- Validate the full plan before pulling or mutating anything. +- Return typed, non-sensitive errors. +- Create with a deterministic name only after registration intent is persisted. +- On partial create, remove the incomplete container but never delete bind-mounted data. +- On replace failure, retain or recreate the prior container plan when safe; otherwise leave the instance stopped and report intervention required. +- Agent startup reconciles its registry with Docker but never adopts unknown containers automatically. + +## Testing focus + +Negative integration tests must cover forged labels, unknown IDs, path traversal, symlink escape, reserved-label overrides, host networking, privileged flags, extra mounts, image substitution, conflicting ports, replayed requests and attempts to target unrelated containers. + diff --git a/docs/architecture/main-application.md b/docs/architecture/main-application.md new file mode 100644 index 0000000..c4bc975 --- /dev/null +++ b/docs/architecture/main-application.md @@ -0,0 +1,77 @@ +# Main application + +## Responsibilities + +- Serve the embedded responsive web UI and versioned HTTP API. +- Authenticate users and enforce global plus instance-scoped authorization. +- Own catalog, immutable template snapshots and module metadata. +- Validate configuration and construct canonical deployment plans. +- Orchestrate lifecycle, backup, import, restore, export and update workflows. +- Persist state and durable jobs in SQLite. +- Run sandboxed WebAssembly adapters. +- Schedule recurring work. +- Deliver notifications and maintain the light audit trail. + +It must not call the Docker socket, run arbitrary commands, trust client-side authorization, or let modules handle files and backups. + +## Proposed Go boundaries + +```text +cmd/dogama +internal/ + auth + catalog + instance + backup + importexport + update + module + notify + audit + jobs + persistence/sqlite + agentclient + web +web/ embedded production assets +migrations/ +``` + +Package names express business capabilities. Avoid a generic `utils` package and avoid passing database handles into HTTP handlers. + +## API rules + +- Version public routes under `/api/v1`. +- Use opaque stable IDs, explicit request/response structs and consistent problem details. +- Require idempotency keys for creation and destructive job submission. +- Use optimistic revision numbers for editable instance configuration. +- Never return stored secret values. Secret fields return only `configured: true|false`. +- Paginate catalog, audit, backups and operations. +- Filter every instance query by the authenticated principal before loading sensitive details. + +## SQLite rules + +- Enable foreign keys, WAL mode and a busy timeout. +- Keep transactions short; filesystem and network operations happen outside transactions. +- Persist workflow intent before external action and result afterward. +- Use append-only migrations with a schema-version table. +- Back up the SQLite database consistently as part of system backup guidance, separate from game backups. +- Keep timestamps in UTC and store IANA timezone names for schedules. + +## Configuration precedence + +1. Bootstrap-only environment or secret files: listen address, database/data root, agent endpoint/token file, master-key file and allowed bind roots. +2. Global administrator settings in SQLite: public game address, defaults, audit retention, notification channels, upload limits and safety policies. +3. Template defaults. +4. Per-instance administrator settings. + +Runtime environment variables must not become a second hidden configuration interface for ordinary product options. + +## Web UI embedding + +V1 uses server-rendered Go `html/template` views with progressive enhancement, a small bundled JavaScript/TypeScript layer and Server-Sent Events for operation/status updates. This keeps the browser payload and build surface small while preserving accessible forms and a stable JSON API. A large client-side framework is not required for V1. + +Production CSS and script assets are compiled before the Go build and embedded. The main binary serves hashed assets with immutable caching and renders application pages without shadowing `/api/` routes. Core administration workflows remain usable when optional real-time enhancement is unavailable. + +## Secret handling + +Encrypt secret values with an authenticated encryption algorithm using a master key external to SQLite. Store key version and nonce with ciphertext. Support key rotation as a maintenance workflow. Decrypt only at the last responsible moment, keep plaintext lifetimes short and redact structured errors. diff --git a/docs/architecture/system-architecture.md b/docs/architecture/system-architecture.md new file mode 100644 index 0000000..a84e7c3 --- /dev/null +++ b/docs/architecture/system-architecture.md @@ -0,0 +1,85 @@ +# System architecture + +## Components + +### Main application + +A statically deployable Go service owns the HTTP API, embedded web UI, authentication, authorization, SQLite data, catalog, workflows, scheduler, backups, notifications, audit and WebAssembly runtime. It has access only to configured DoGaMa data, server-data roots and backup roots. It does not mount the Docker socket. + +### Restricted Docker agent + +A separate Go service is the only DoGaMa component with Docker socket access. It is not published on the host. It accepts a small typed operation set, validates every plan against approved roots and signed/registered instance state, and never exposes a generic Docker proxy. + +### Managed game servers + +Each instance is a Docker container created from an approved template snapshot. Persistent data uses bind mounts below administrator-approved roots. Game ports may be published; management API ports remain private whenever the game permits it. + +### WebAssembly adapters + +Optional modules translate game-specific APIs to DoGaMa's normalized game API. Modules run inside the main application under a deny-by-default sandbox. They do not orchestrate Docker or files. + +## Trust boundaries + +```text +Untrusted browser and uploads + | + v +[Main application boundary] + AuthN/AuthZ, workflows, DB, archive validation, WASM runtime + | + | authenticated, private, typed plans + v +[Agent privilege boundary] + plan verification, path/image/port/label enforcement + | + v +[Docker daemon / host boundary] + | + v +[Game container boundary] <--- instance-scoped WASM host networking +``` + +Templates, manifests, modules, catalog artwork, webhooks, imports and game API responses are untrusted data even when an administrator supplied them. + +## Dependency direction + +- HTTP and scheduled jobs call application use cases. +- Use cases depend on domain interfaces, not Docker, SQLite or WASM implementations. +- Infrastructure adapters implement persistence, agent client, archive, notification and WASM interfaces. +- The agent has its own domain model and does not import main-application persistence code. +- Game-specific behavior crosses only the normalized module contract. + +## Persistence and filesystem layout + +Suggested container paths: + +```text +/var/lib/dogama/ + dogama.db + catalog/ + modules/ + imports/staging/ + cache/ +/srv/game-servers// + config/ + player-data/ + mods/ + logs/ +/srv/game-backups// +``` + +Paths stored in SQLite use stable instance and mount identifiers. User-supplied display names never become paths without normalized slug generation and collision checks. + +## Communication + +- Browser to main application: HTTP(S), JSON API, secure cookie session. +- Main application to agent: private Docker network, request authentication, timestamp/nonce replay protection and bounded request bodies. Same-host V1 may use a shared secret; mTLS is reserved for later multi-host work. +- Main application to game API: only through the WebAssembly host networking interface bound to the instance. +- Main application to notification endpoints: controlled egress with SSRF protections. + +## Concurrency and jobs + +Long-running operations are durable jobs in SQLite. A per-instance lock serializes mutually exclusive actions. Jobs use explicit phases and checkpoints so a restart can resume, retry safely or mark manual intervention required. UI requests enqueue work and return an operation identifier rather than keeping a long HTTP request open. + +The scheduler is internal for V1 and handles cron backups, retention, audit purge, optional start/stop schedules, module/catalog update checks and queued notifications. Only one scheduler leader exists because V1 runs one main-application replica. + diff --git a/docs/architecture/wasm-modules.md b/docs/architecture/wasm-modules.md new file mode 100644 index 0000000..4e9a6e7 --- /dev/null +++ b/docs/architecture/wasm-modules.md @@ -0,0 +1,77 @@ +# WebAssembly integration modules + +## Purpose + +A module is a small translator between a game server API and DoGaMa's normalized game API. + +```text +Game-specific API <-> WebAssembly adapter <-> normalized DoGaMa API +``` + +It may query status, list players, request an in-game save, announce, shut down gracefully, kick, ban or unban when the game supports those operations. It does not own container lifecycle, files, users, backups, scheduling or UI. + +## Package + +```text +palworld-rest-1.0.0.dogama-module/ + module.wasm + manifest.yaml + README.md + LICENSE +``` + +The installed artifact is content-addressed. Manifest, binary checksum, source/trust status and installation time are recorded. A package cannot contain executable helpers or dynamic libraries. + +## Runtime contract + +The V1 ABI uses a versioned WebAssembly component/WIT contract or an equivalently typed ABI. JSON may be used at a debugging boundary but is not the authority for function signatures. `specs/normalized-module-api.md` defines semantics. + +The runtime grants no ambient WASI filesystem, process, environment, raw sockets or arbitrary DNS. Time and randomness are provided only if a documented operation requires them. Host functions include: + +- bounded HTTP request to logical handle `instance_api`; +- bounded TCP request only if declared by both template and manifest; +- secret lookup by declared configuration key without exposing unrelated secrets; +- structured diagnostic emission with runtime redaction; +- cancellation/deadline checks. + +## Instance-scoped networking + +Modules never receive an arbitrary destination URL. At activation, DoGaMa binds `instance_api` to a specific instance network identity and declared integration port. Every request is checked for protocol, port, method, timeout, redirect, request/response size and concurrency. + +- No Internet or LAN destinations. +- No loopback, link-local, metadata or Unix-socket destinations. +- No redirects outside the bound origin. +- DNS rebinding cannot change the authorized resolved destination. +- The management port is preferably unexposed on the host. + +## Capabilities + +Capabilities are explicit strings defined by the normalized API. A module may implement a subset. DoGaMa shows actions only when all of these agree: + +1. template enables the integration and feature; +2. manifest declares the capability; +3. module runtime reports the same capability; +4. current user has permission; +5. instance state permits the operation. + +Unknown capabilities are rejected for the current schema version. A claimed capability without its required export blocks activation. + +## Resource limits + +Per call, enforce a deadline, instruction/fuel budget, memory ceiling, maximum host calls, maximum payload/response size and cancellation. Limit concurrent calls per module and instance. Repeated traps open a circuit breaker and degrade only the integration; generic container management remains available. + +## Independent versioning + +Templates and modules have independent semantic versions. + +- A template pins an acceptable module ID and version range. +- A manifest states supported manager module-API versions and game IDs. +- Installation of a new module does not activate it automatically for existing instances. +- Activation runs schema, checksum, ABI, capability and connection tests. +- The previous version stays available for rollback until the new version is healthy. +- A module update never silently changes instance settings or template snapshots. + +## Prohibited behavior + +Modules cannot create/delete containers, read SQLite, access host/game files, create backups, execute commands, manage users, expose routes or UI, contact other instances, or make unrestricted network calls. If a proposed integration needs those powers, the generic DoGaMa contract must be extended safely instead of bypassed. + diff --git a/docs/contributing/ai-codex-guide.md b/docs/contributing/ai-codex-guide.md new file mode 100644 index 0000000..c0321f2 --- /dev/null +++ b/docs/contributing/ai-codex-guide.md @@ -0,0 +1,56 @@ +# Guide for AI and Codex contributors + +This repository is deliberately structured so an AI can add a game integration without reverse-engineering the manager. Treat normative prose, JSON Schemas and reference examples as a single contract set. + +## Before making a change + +1. Read `README.md` product invariants and root `AGENTS.md`. +2. Identify the authoritative domain document. +3. Validate current schemas and examples before editing. +4. Inspect Palworld as the reference, but do not generalize a Palworld quirk into the core model. +5. State uncertain game behavior and cite primary game/container API documentation in contributor-facing notes. + +## Adding a game without an API + +Create a template only. Declare container image, ports, storage, fields, health, backups, mods and update behavior. Do not invent a module merely to start/stop Docker; generic orchestration already handles that. + +## Adding a game with an API + +Create: + +```text +catalog//template.yaml +modules//manifest.yaml +modules//README.md +modules//src/... implementation phase +modules//tests/... +``` + +Choose only required capabilities. Translate game errors into normalized errors. Use the logical `instance_api` host functions; never accept arbitrary destination URLs. Keep game API credentials as declared secret configuration. + +## Required verification for a contribution + +- Template validates against `specs/template.schema.json`. +- Manifest validates against `specs/module-manifest.schema.json`. +- Declared capabilities correspond to normalized functions and module exports. +- Template module version range accepts the manifest version. +- All integration ports exist in the template and remain private unless explicitly required for players. +- Backup source mount exists and strategy matches module capability. +- Secret fields are write-only and excluded from examples/fixtures. +- Minimum/recommended requirements and upstream image/API facts have primary-source evidence. +- Failure behavior works when the game API is offline, slow, malformed or unauthorized. + +## Safe response to “add this new game” + +An AI should produce a short evidence table first: upstream image, required ports, persistent player-data path, health method, configuration interface, save consistency, update mechanism, mod mechanism and API availability. Unknowns remain explicit `TODO` blockers or conservative template omissions; do not fabricate them. + +Then implement the smallest valid surface: + +- Template-only when generic Docker lifecycle is enough. +- Status-only module when only status is reliable. +- Add player moderation or online save only with verified API behavior. + +## Contract-edit rule + +If a new game cannot fit the current schema, first determine whether it exposes a genuinely generic need. Extend the schema narrowly with documentation, migration/compatibility analysis, validation, negative tests and updated examples. Never add an escape hatch such as arbitrary commands, raw Compose fragments, host paths or unrestricted network permissions. + diff --git a/docs/contributing/development.md b/docs/contributing/development.md new file mode 100644 index 0000000..eab48d9 --- /dev/null +++ b/docs/contributing/development.md @@ -0,0 +1,67 @@ +# Development conventions + +## Language and style + +- Documentation and stable identifiers are English-first; UI strings are localizable from the start. +- Use idiomatic current stable Go, explicit errors and small interfaces at consumer boundaries. +- Prefer server-rendered `html/template` views, progressive enhancement and small focused browser modules; do not introduce a large SPA framework without an accepted architectural reason. +- Format and lint frontend and Go code with repository-pinned tools. +- Avoid hidden global state. Inject clock, ID generator and external interfaces for deterministic tests. +- Use UTC internally and IANA timezones at scheduling boundaries. + +## Repository shape + +The implementation may refine this layout while preserving boundaries: + +```text +cmd/dogama/ +cmd/dogama-agent/ +internal/ non-public Go packages +web/ embedded UI source +migrations/ append-only SQLite migrations +specs/ schemas and normalized contracts +catalog/ reference templates +modules/ reference modules and fixtures +docs/ +tests/integration/ +``` + +## Contract changes + +Template schema, manifest schema, normalized module API and agent deployment plan are versioned contracts. + +- Backward-compatible additions do not change existing meanings. +- Breaking changes require a new schema/API major version and documented migration. +- Released examples are updated or retained as compatibility fixtures. +- Instances pin immutable versions; runtime behavior never depends on a mutable catalog file. + +## Testing pyramid + +- Unit: domain policies, permission evaluation, cron/timezones, state transitions, retention and redaction. +- Property/fuzz: paths, archive entries, schema inputs, agent plans and normalized module payloads. +- Integration: SQLite migrations, disposable Docker agent, real archive workflows and WASM sandbox limits. +- End-to-end: bootstrap, Palworld dry/test fixture deployment, role views, backup/restore and update rollback. + +Tests must include denial and interrupted-operation cases, not only happy paths. External game APIs use recorded or purpose-built fixtures in normal CI; live Palworld verification is a separate documented test. + +## Migrations and compatibility + +Never rewrite a released migration. Migration execution is transactional where SQLite permits and creates a pre-migration database backup for release upgrades. Store application/schema version and test upgrade from the previous release. + +## Dependencies and supply chain + +Prefer standard-library or mature focused dependencies. Pin build tooling, review transitive changes and generate an SBOM/reproducible checksums for releases. Do not dynamically download executable code at runtime except administrator-installed validated WASM packages. + +## Observability + +Structured technical logs use stable event names and redaction. Metrics are operationally small. Correlation/operation IDs connect HTTP, job and agent errors without storing request secrets. + +## Review checklist + +- Correct trust boundary and backend authorization? +- Can input influence a host path, Docker plan, URL or secret? +- Are retries/idempotency and restart recovery defined? +- Could failure delete or corrupt player data? +- Do documentation, schema and examples agree? +- Are UI capability checks backed by server checks? +- Are audit and notifications appropriately minimal? diff --git a/docs/domain/authorization.md b/docs/domain/authorization.md new file mode 100644 index 0000000..809bf35 --- /dev/null +++ b/docs/domain/authorization.md @@ -0,0 +1,92 @@ +# Roles and permissions + +Authorization combines a global role with instance membership and explicit per-user overrides. It is always enforced in backend use cases. + +## Global roles + +- `admin`: full system administration and implicit access to every instance. +- `user`: no instance access until membership is assigned. + +There is no global manager role. `manager` is an instance membership so a person may manage Palworld, use Minecraft and have no access to Valheim. + +## Instance baselines + +### User + +- View game name, description, artwork and safe catalog metadata. +- View public connection address and game port. +- View instance status, readiness, uptime and non-sensitive configured values. +- View player count, player list and metrics when capability and policy allow. +- Start and stop the assigned instance. +- Submit a catalog installation request. + +### Manager + +All user permissions plus: + +- Restart and request graceful shutdown. +- Trigger a manual backup. +- Apply an administrator-allowed update. +- Edit the instance page welcome message. +- View allowed game logs. +- Send announcements and kick/ban/unban when supported. +- Manage declared mods if granted. + +Restore is intentionally not included. `backup.restore` must be explicitly granted by an administrator. + +### Administrator + +- Manage users, global settings, channels and retention. +- Manage catalog sources, templates, modules and trust decisions. +- Create, edit, duplicate, adopt and safely delete instances. +- Assign memberships and overrides. +- Configure public address, ports, storage, resources, secrets and backup policies. +- Import at creation; restore, export and delete backups. +- Approve or refuse installation requests; approval opens a prefilled creation form and never deploys immediately. +- View audit events and perform a bounded purge. + +## Stable permission identifiers + +```text +instance.view +instance.start +instance.stop +instance.restart +instance.update +instance.configure +instance.delete +instance.welcome.edit +metrics.view +players.view +players.kick +players.ban +players.unban +announcements.send +logs.view +mods.manage +backup.create +backup.list +backup.export +backup.restore +backup.delete +request.create +``` + +Admin-only system permissions are not delegated per instance in V1. + +## Evaluation algorithm + +1. Deny unauthenticated or disabled users. +2. Allow global admin, subject to re-authentication requirements for critical actions. +3. Require an active membership for the target instance. +4. Start from the membership baseline. +5. Apply explicit deny overrides before explicit allows. +6. Require operation prerequisites: module capability, current state and global policy. +7. Record important allowed and denied security actions according to the audit policy. + +Client-provided instance IDs, roles and permission lists are never trusted. Object lookup and permission evaluation occur in one application-layer call to prevent confused-deputy errors. + +## Sensitive-action safeguards + +Restore, destructive delete, membership changes, secret rotation and security configuration require recent authentication. Data removal requires separate checkboxes and typed instance-name confirmation. A manager never gains new abilities merely because a module exposes a capability. + diff --git a/docs/domain/data-model.md b/docs/domain/data-model.md new file mode 100644 index 0000000..1a7b154 --- /dev/null +++ b/docs/domain/data-model.md @@ -0,0 +1,64 @@ +# Data model + +SQLite is authoritative for product state. Runtime Docker state is reconciled into, but never replaces, the registry. + +## Core entities + +| Entity | Purpose | Important fields | +|---|---|---| +| `users` | Local identities | id, username, password_hash, global_role, disabled_at, created_at | +| `sessions` | Revocable browser sessions | id_hash, user_id, expires_at, last_seen_at | +| `instance_memberships` | Per-instance baseline role | instance_id, user_id, role (`user`, `manager`) | +| `permission_overrides` | Explicit allow/deny beyond baseline | instance_id, user_id, permission, effect | +| `templates` | Catalog identity and origin | id, origin, trust_status, active_version | +| `template_versions` | Immutable validated snapshots | template_id, version, schema_version, canonical_yaml, digest | +| `modules` | Module identity and trust | id, source, trust_status, active_version | +| `module_versions` | Immutable installed artifacts | module_id, version, manifest, wasm_digest, path, api_range | +| `instances` | Desired and observed instance state | id, slug, display_name, template snapshot, revision, lifecycle_state, public_host | +| `instance_settings` | Typed non-secret template values | instance_id, field_id, value_json | +| `instance_secrets` | Encrypted secret values | instance_id, field_id, key_version, nonce, ciphertext | +| `instance_ports` | Published and private bindings | instance_id, port_id, host_ip, host_port, container_port, protocol | +| `instance_mounts` | Approved persistent mount mapping | instance_id, mount_id, host_path, container_path, category | +| `instance_resources` | Docker limits | instance_id, cpu_limit, memory_limit_mb, reservation_mb, pids_limit | +| `instance_module_bindings` | Pinned integration version | instance_id, module_id, module_version, config_revision, status | +| `configuration_revisions` | Small rollback history | instance_id, revision, redacted_snapshot, reason, created_by | +| `backup_policies` | Schedule and retention | instance_id, enabled, cron, timezone, retention_count, safety flags | +| `backups` | Managed archive metadata | id, instance_id, origin, status, path, size, sha256, game/template versions | +| `imports` | Temporary validation workflow | id, instance_id nullable, stage_path, detected_type, status, expires_at | +| `operations` | Durable long-running workflows | id, instance_id, type, phase, status, idempotency_key, error_code | +| `scheduled_jobs` | Next execution state | id, type, owner_id, schedule, timezone, next_run_at, enabled | +| `installation_requests` | User catalog requests | id, requested_by, template_id, message, status, reviewed_by | +| `notification_channels` | Global delivery configuration | id, type, enabled, encrypted_config, event_filter | +| `notification_deliveries` | Bounded retry queue | id, channel_id, event_type, payload_redacted, attempt, next_attempt_at | +| `audit_events` | Compact significant actions | id, occurred_at, actor_id, instance_id, action, outcome, summary_json | +| `system_settings` | Admin-configured global values | key, value_json, revision | + +## Invariants + +- IDs are opaque and stable; slugs are unique but mutable only through a controlled rename. +- Released template and module versions are immutable. Editing creates a new version or an independent local copy. +- An instance pins a template snapshot and module version; catalog changes do not mutate it silently. +- There is at most one active mutating operation per instance. +- A port tuple `(host_ip scope, host_port, protocol)` cannot be assigned twice by DoGaMa. +- Mount host paths are canonical absolute paths below configured roots. +- Secret fields never coexist in plaintext settings. +- Backup metadata becomes `available` only after archive finalization and checksum persistence. +- Imports expire and their staging directories are cleaned unless attached as a managed backup. +- Audit `summary_json` is allow-listed by event type and contains no secret values or full uploaded content. + +## Backup origins + +`manual`, `scheduled`, `pre_update`, `pre_restore`, `idle_shutdown`, `imported` and `system` are stable origin identifiers. V1 retention applies only to automatic backups eligible under the instance policy. Manual, imported and explicit safety backups require deliberate deletion or a separately documented policy. + +## Operation state + +Operations use `queued`, `running`, `succeeded`, `failed`, `cancelled` or `intervention_required`. A phase-specific checkpoint records enough information to decide safely after restart whether to resume, compensate or stop. + +## Data retention + +- Audit: administrator-configurable, default 30 days, optional maximum entry count. +- Configuration history: default 10 revisions per instance, with secrets omitted. +- Notification delivery attempts: short operational retention after terminal state. +- Temporary imports: default 24 hours after last activity. +- Technical logs: standard output/error, outside SQLite and controlled by container log rotation. + diff --git a/docs/domain/instance-lifecycle.md b/docs/domain/instance-lifecycle.md new file mode 100644 index 0000000..91732f9 --- /dev/null +++ b/docs/domain/instance-lifecycle.md @@ -0,0 +1,76 @@ +# Instance lifecycle + +## User-visible states + +```text +draft -> installing -> stopped -> starting -> online + \-> degraded +online -> stopping -> stopped +any stable state -> backup | restore | update -> stable state +any state -> error | unknown | intervention_required +``` + +`container_running` is an observation, not the `online` state. Online requires the template health probe or module readiness check to succeed within its startup timeout. + +## Creation + +1. Select a validated template version. +2. Choose new world or import existing data. +3. Validate settings, secrets, ports, resources, storage and optional module. +4. If importing, upload to staging, inspect safely and show compatibility confidence. +5. Display a canonical deployment preview. +6. Persist instance intent and immutable configuration revision. +7. Ask the agent to check ports/disk and create the registered container. +8. For import, inject only validated data into the declared destination before first start. +9. Start if requested, wait for readiness and surface a precise result. + +An approved installation request begins at step 2 with suggested values; it never skips administrator review. + +## Start and stop + +Start is idempotent. It fails clearly during conflicting operations or maintenance mode. Readiness transitions from starting to online/degraded/error based on health results. + +Manual stop requests a module graceful shutdown when supported, then waits for the container to exit and uses the agent timeout as a bounded fallback. An idle automatic stop follows the backup rules: + +- With `online_save`: request and confirm the in-game save, optionally archive while running if template consistency allows, then stop. +- Without `online_save`: stop the game first, then archive player data if configured. + +The configurable failure policy is `abort_stop`, `stop_without_backup` or `force_stop_after_timeout`; safe default is `abort_stop` for automatic shutdown and an explicit choice for manual operations. + +## Maintenance and pending changes + +Maintenance mode blocks ordinary user starts and shows an administrator message while preserving manager/admin access. Settings specify `immediate` or `restart_required`; pending restart changes are applied together through a controlled container replacement. Keep a small redacted configuration history. + +## Crash-loop protection + +Track unexpected exits. Default circuit breaker: five restarts within ten minutes disables automatic restart and moves the instance to error. Manual administrator action after diagnosis resets it. Scheduled jobs do not fight this state. + +## Update + +1. Resolve current and candidate image references/digests. +2. Check template/module compatibility, disk space and mod warnings. +3. Show changes and require confirmation. +4. Create a safety backup when policy requires it. +5. Gracefully stop. +6. Pull and replace through the agent using the same approved plan plus new image. +7. Start and verify readiness. +8. On failure, restore the previous container image/configuration; never restore player data automatically unless migration modified it and a documented compensation requires it. +9. If rollback is unsafe, leave stopped and notify. + +Automatic game-server updates are off by default. + +## Deletion + +Deletion offers independent scopes: + +1. container only; +2. container plus technical/cache files; +3. player data; +4. backups. + +Scopes 3 and 4 are off by default, require typed-name confirmation and are audited. The agent deletes only the container; the main application performs carefully validated filesystem cleanup below allowed instance roots. Partial deletion remains visible until reconciled. + +## Reconciliation + +At startup and periodically, compare desired registry state with agent-inspected registered instances. Unknown Docker containers are ignored. Missing or altered managed containers become `unknown` or `intervention_required`; DoGaMa does not silently recreate or adopt them when data safety is uncertain. + diff --git a/docs/operations/backups-import-export.md b/docs/operations/backups-import-export.md new file mode 100644 index 0000000..9d73ffe --- /dev/null +++ b/docs/operations/backups-import-export.md @@ -0,0 +1,93 @@ +# Backups, import, restore and export + +## Backup policy + +Each instance supports manual backups and one V1 cron schedule using a standard five-field expression plus an IANA timezone. The UI provides common presets and a preview of upcoming runs. `retention_count` is the number of eligible automatic backups to retain. + +V1 retention deletes oldest successful `scheduled` backups beyond the count. It does not silently delete manual, imported or safety backups. Failed/incomplete artifacts are cleaned by a separate short operational policy. + +## Consistency strategies + +The template declares backup sources and one strategy: + +- `online_save`: call the module's `save_world`, wait for success, then archive declared data while running if safe. +- `stop_then_archive`: gracefully stop, archive, then restart only if it was running before the operation. + +The module only requests the game to flush state. The main application owns traversal, archive, checksum, retention and metadata. + +## Archive format + +A DoGaMa-native backup is a `tar.zst` containing: + +```text +manifest.json +data//... +``` + +The outer database metadata stores SHA-256, size and final path. `manifest.json` stores schema version, backup ID, instance/template/game versions, origin, creation time, included mount IDs, relative paths and per-file or archive integrity information. It contains no secrets. + +Write to a unique temporary file, flush, compute checksum and atomically rename. Never mark a backup available before finalization. + +## Manual and scheduled workflow + +1. Acquire the instance operation lock. +2. Check writable destination and conservative free-space estimate. +3. Reach a consistent game state according to the template. +4. Traverse only declared sources without following escaping links. +5. Build and finalize the archive. +6. Restore the prior running state if the workflow stopped it. +7. Persist success and apply retention. +8. Audit manual requests and notify configured failures/successes. + +## Import staging + +All external data first enters: + +```text +/var/lib/dogama/imports/staging// +``` + +Validation enforces global and template limits for upload size, extracted size, file count, nesting and operation time. Reject absolute paths, `..`, Windows drive paths, NULs, device/FIFO/socket entries, hard links, and symlinks that escape staging. Extraction uses create-new semantics and never overwrites application files. + +Supported formats are declared by the template. Importers handle data only; imported content is never executed. + +## Import modes + +### At instance creation + +Choose `new_world` or `import`. For import, validate and preview before creating directories or a container. After the deployment plan is approved, create the instance layout and copy normalized data into declared destination mounts before first start. + +### Existing instance + +An authorized actor may: + +- validate only; +- restore once from staging without retaining the upload; +- add the validated upload to managed backups, origin `imported`, then optionally restore. + +The preview reports detected game/type, file count, expanded size, world/player hints when safely available, destination and confidence: `confirmed`, `probable`, `recognized_unknown_version` or `unrecognized`. + +## Restore + +1. Verify permission and recent authentication. +2. Verify archive checksum, manifest and compatibility. +3. Show overwritten destinations and compatibility confidence. +4. Create a `pre_restore` safety backup by default; disabling it is an administrator-only exceptional action. +5. Stop the instance gracefully. +6. Extract into a new validated sibling staging directory, not the live directory. +7. Swap or copy using a recoverable plan; preserve the previous data until success. +8. Apply required ownership within approved roots. +9. Start only if requested and verify readiness. +10. On failure, restore the previous data when safe; otherwise keep the server stopped and mark intervention required. + +## Export + +- Backup export downloads an existing integrity-verifiable archive. +- Instance configuration export is versioned YAML containing template identity, non-secret settings, resource limits, port intent, mount categories, mods and policies. Host-specific paths may be redacted or expressed as logical roots. +- Normal exports omit passwords, tokens, API credentials, encrypted blobs, internal agent metadata and session data. +- A later explicit encrypted disaster-recovery export may be designed separately; it is not V1. + +## Palworld import note + +The reference template recognizes dedicated-server world layouts with `Level.sav` and `Players/`. A local hosted-world import may require player identity conversion. V1 must preserve the upload, warn about this possibility and never perform undocumented silent conversion. A future game-specific data converter remains separate from the WebAssembly API adapter because modules have no filesystem access. + diff --git a/docs/operations/instance-operations.md b/docs/operations/instance-operations.md new file mode 100644 index 0000000..d089650 --- /dev/null +++ b/docs/operations/instance-operations.md @@ -0,0 +1,53 @@ +# Resources, ports, storage, mods and updates + +## Resources + +Templates declare minimum and recommended CPU, memory and storage. The creation form suggests recommended values and warns below minimum. Administrators set Docker CPU limit, memory limit/reservation and PID limit within global guardrails. DoGaMa displays current CPU/RAM and configured limits without keeping long-term time series in V1. + +Before installation, update, import or backup, estimate required disk space and compare it with configurable warning and critical free-space thresholds. + +## Ports and connection address + +Templates name container ports and protocol. Administrators choose host ports or accept a conflict-free suggestion. DoGaMa checks its registry and asks the agent for an availability result that does not reveal unrelated containers. + +A global administrator configures the public IP address or DNS name. It is displayed with each public game port. DoGaMa does not claim to configure NAT, router forwarding or firewalls. Management/API ports default to private and never appear as player connection endpoints. + +## Storage + +Templates declare mount IDs, container paths, categories and whether host location is configurable. Categories include `runtime`, `configuration`, `player_data`, `mods` and `logs`. Backups include only explicitly selected persistent categories, normally player data and essential configuration. + +Host paths are absolute canonical paths under configured roots. Display-name input cannot inject path separators. The UI shows technical data, player data and backup locations separately. Moving storage is an explicit stopped-instance migration with space checks, verification and rollback. + +## Mods + +Mod support is declarative and optional. Supported provider types may include: + +- `steam_workshop` with validated numeric item IDs; +- `remote_archive` with HTTPS, checksum and SSRF protections; +- `local_upload` through safe staging; +- a future named provider with a dedicated generic implementation. + +The template states destination mount, ordering, restart requirement, dependency behavior and update compatibility. Modules do not download or install mods. Mod changes can require a safety backup and always create a configuration revision. + +DoGaMa clearly labels unofficial mod support and never assumes a server update is compatible with installed mods. + +## Configuration application + +Each template field declares `apply: immediate` or `restart_required`. Secret fields are write-only. Validate types, ranges, patterns and conflicts on both client and server. A preview lists pending changes and whether container replacement or game restart is needed. + +Keep the last 10 redacted configuration revisions by default. Rollback revalidates the old revision against the pinned template/module versions before applying it. + +## Updates + +Image updates are digest-aware. A mutable tag alone is never treated as proof that nothing changed. The UI shows current and candidate references, template release notes if available, mod warnings and whether a backup will run. + +The full update sequence and rollback behavior are normative in `docs/domain/instance-lifecycle.md`. Managers may trigger only updates allowed by global/instance policy; administrators choose channels and may pin a digest. Automatic updates remain disabled by default. + +## Template and module updates + +- Updating a catalog template creates a new immutable version; instances remain pinned. +- A migration preview compares ports, paths, settings, image and module range. +- Local copied templates are independent and are never overwritten by their origin. +- Module packages update independently and require compatibility plus connection tests before activation. +- Rollback keeps the prior template snapshot, module binary and container plan available until the new combination is verified. + diff --git a/docs/operations/notifications-and-audit.md b/docs/operations/notifications-and-audit.md new file mode 100644 index 0000000..ab62d0c --- /dev/null +++ b/docs/operations/notifications-and-audit.md @@ -0,0 +1,62 @@ +# Notifications and audit + +## Notification channels + +Administrators configure channels entirely in the UI: + +- SMTP email; +- generic HTTPS webhook; +- Discord webhook. + +Channel secrets are encrypted and write-only. A test action sends a clearly marked test message and reports a redacted result. + +## Events and filtering + +Suggested configurable events: + +- backup, restore or import failed/completed; +- update failed/completed; +- instance entered error/degraded or crash-loop protection; +- low/critical disk space; +- installation request submitted/approved/refused; +- module disabled after repeated failures; +- security-sensitive repeated authentication failure. + +Default notifications favor failures and required action. Normal health polls and metric refreshes never notify. + +Deliveries are queued after the originating transaction, use bounded exponential retry and cannot fail the lifecycle operation. Payloads contain display names and operation IDs, not secrets, raw credentials or large logs. Generic webhook requests are signed and include a timestamp and event ID for receiver deduplication. + +Webhook URL validation blocks loopback, private/link-local/metadata destinations by default, validates every redirect and resists DNS rebinding. An explicit future private-webhook feature would need a separately reviewed allowlist. + +## Light audit trail + +The audit trail answers: who performed an important action, on what instance, when and with what outcome. It is not technical logging or monitoring. + +Audit: + +- successful login and repeated/blocked login failures; +- instance create/adopt/delete and manual start/stop/restart; +- configuration, resource, port, storage or module binding changes; +- manual backup, restore, import and export; +- update and rollback; +- user, membership and permission changes; +- kick/ban/unban and announcement actions; +- catalog/template/module trust or activation changes; +- notification and security-policy changes. + +Do not audit: + +- page views; +- metrics/player/status polling; +- normal readiness probes; +- each ordinary scheduled-job tick; +- successful recurring notification delivery unless operationally needed. + +Entries are compact and use allow-listed structured summaries. Player IDs may be hashed or minimized where full identifiers are unnecessary. Never include secret values, HTTP authorization headers, imported content or module raw responses. + +## Retention + +Audit retention defaults to 30 days and is globally administrator-configurable. An optional maximum count prevents unbounded growth. Purge runs daily and records one aggregate audit event, not an event per deleted row. Unlimited retention requires an explicit warning and displays database usage. Administrators may manually purge by date with confirmation. + +Technical application logs go to stdout/stderr and use Docker log rotation. Their level and retention are separate from SQLite audit policy. + diff --git a/docs/product/acceptance-criteria.md b/docs/product/acceptance-criteria.md new file mode 100644 index 0000000..d458230 --- /dev/null +++ b/docs/product/acceptance-criteria.md @@ -0,0 +1,76 @@ +# V1 acceptance criteria + +V1 is accepted only when every mandatory criterion below has an automated test or a documented end-to-end verification result. + +## Installation and bootstrap + +- `compose.yaml` starts one public main application and one non-public restricted agent. +- The main application has no Docker socket mount. +- First run requires creation of an administrator and refuses normal use until bootstrap is complete. +- A master encryption key and agent authentication secret are read from files, not literal environment values. +- All post-bootstrap product settings listed in the specifications are editable in the UI. + +## Catalog and instance creation + +- Invalid templates and manifests are rejected with field-specific errors. +- The Palworld examples validate against the checked-in schemas. +- Creation presents image, digest or tag, ports, mounts, resources, data origin and backup policy before confirmation. +- Port conflicts, insufficient disk space and paths outside approved roots block creation. +- An administrator can create an empty instance or create one from a validated imported save. +- The created container is registered in SQLite and carries DoGaMa labels and a non-forgeable registration binding checked by the agent. +- Unrelated Docker containers never appear in the API or UI and cannot be targeted through identifier substitution. + +## Authorization + +- Global administrators can manage system configuration and all instances. +- Non-admin access is absent unless an instance membership exists. +- User and manager baselines match `docs/domain/authorization.md`. +- Fine-grained overrides can remove a manager permission or add the separately controlled `backup.restore` permission. +- Every privileged API endpoint has denial tests; UI hiding alone is not accepted. + +## Lifecycle and operations + +- State distinguishes Docker running from game ready and exposes installing, starting, online, stopping, backup, restore, update, degraded, error and unknown conditions. +- Start, stop, restart, update and delete operations are serialized per instance and are idempotent where applicable. +- Repeated crash restarts trip a configurable circuit breaker. +- Resource limits, port mappings and approved storage mounts survive container recreation. +- Update displays old and new image references, makes a safety backup when configured, verifies health and rolls back or stops safely on failure. +- Deletion preserves player data and backups unless the administrator separately confirms their removal by typing the instance name. + +## Backups, imports and exports + +- Manual and five-field cron schedules work with a retention count per instance. +- Automatic retention deletes only eligible automatic backups; manual and imported backups are not silently removed. +- If `online_save` exists, idle shutdown requests and confirms a game save before stopping; otherwise DoGaMa stops first and archives afterward. +- Backup archives include a manifest and SHA-256 checksum and are written atomically. +- Restore verifies integrity, creates a pre-restore backup by default, stops the server and has a defined rollback or safe-stop result. +- Imports are staged and reject absolute paths, traversal, escaping links, device files, excessive file counts and extraction bombs. +- Import at instance creation does not create the server until validation succeeds. +- A normal instance export contains no secrets; a backup export is integrity-verifiable. + +## Modules + +- Only WebAssembly modules are accepted. +- The runtime enforces memory, fuel/instruction, time, response-size and concurrency limits. +- A module receives no filesystem, clock, random, environment, process or raw socket access unless exposed by a documented host function. +- Network calls can reach only the bound instance endpoint, declared protocol and declared integration port. +- Capability and function disagreement fails module activation. +- Template and module versions are independently upgradeable and a compatibility check precedes activation. + +## Security, audit and notifications + +- Secrets are encrypted at rest and never returned after write. +- Authentication is rate-limited; sessions use secure cookie settings and CSRF protection. +- Audit records important human/security actions but not page views, metrics polling or normal health probes. +- Audit retention defaults to 30 days and is configurable by an administrator, including a bounded manual purge. +- Email, generic webhook and Discord can be configured and tested in the UI. +- Notification delivery is queued, redacts secrets, retries with bounds and never blocks the originating operation. +- SSRF protections cover artwork downloads, webhook destinations and any administrator-provided URL. + +## Quality gates + +- Go tests, static analysis, frontend tests and schema/example validation pass. +- Database migrations work from an empty database and from the prior released schema. +- Backup/restore, update rollback, authorization and agent-scope integration tests pass against a disposable Docker environment. +- Documentation links resolve and examples use the same field names as the schemas. + diff --git a/docs/product/roadmap.md b/docs/product/roadmap.md new file mode 100644 index 0000000..a287984 --- /dev/null +++ b/docs/product/roadmap.md @@ -0,0 +1,37 @@ +# Roadmap + +## V1 — reliable single-host foundation + +1. Go application skeleton, embedded UI, SQLite migrations, bootstrap admin and authentication. +2. Restricted same-host Docker agent with authenticated private API and allowed-root enforcement. +3. Template validation, local catalog, deployment preview and instance registry. +4. Instance lifecycle, health/readiness, resources, ports, storage and safe deletion. +5. Per-instance authorization and installation-request workflow. +6. Backup scheduler, retention, import at creation, restore and export. +7. WebAssembly runtime, normalized API and Palworld REST reference adapter. +8. Controlled updates, mods configuration, configuration history and rollback paths. +9. Email/webhook/Discord notifications and light audit trail. +10. Security hardening, end-to-end tests, contributor documentation and release packaging. + +V1 prioritizes correctness and recoverability. Features may be visually modest, but the security boundaries and data workflows must not be prototypes. + +## V1.x — usability and ecosystem + +- Instance duplication with explicit choices for data, mods and new ports. +- Start/stop schedules and configurable idle shutdown. +- Improved local catalog update and signed community catalogs. +- Module SDKs and examples for HTTP, TCP and RCON-like protocols. +- Template/module compatibility migrations and richer validation tooling. +- Optional encrypted secret export and disaster-recovery workflow. +- Additional game integrations contributed through the documented contracts. + +## V2+ — only with demonstrated need + +- Multiple trusted agents or hosts with mTLS and explicit placement. +- Remote backup destinations. +- OIDC or external identity providers. +- Richer health and operational history without becoming a monitoring platform. +- Podman-compatible execution behind the same restricted agent contract. + +Commercial hosting, billing, Kubernetes and arbitrary container administration remain out of scope unless the product mission is deliberately revised. + diff --git a/docs/product/vision-and-scope.md b/docs/product/vision-and-scope.md new file mode 100644 index 0000000..b93a1a2 --- /dev/null +++ b/docs/product/vision-and-scope.md @@ -0,0 +1,53 @@ +# Vision and scope + +## Vision + +DoGaMa lets a household or a small trusted group install, operate, update and protect a few private game servers without becoming Docker experts. The normal path is graphical, understandable and safe: choose a game, review the deployment, configure resources and storage, then create the instance. + +The product must remain lightweight enough for a self-hosted machine already running other services. It is not a general-purpose container manager and must not expose unrelated containers. + +## Target users + +- An administrator who owns the host and controls catalog, users, storage roots, public address, policies and instances. +- A manager trusted to operate one or more assigned instances. +- A user invited to play on one or more instances and perform safe day-to-day actions. +- A contributor who adds a game using a YAML template and, only when necessary, a small WebAssembly adapter. + +## V1 outcomes + +- Install DoGaMa with a minimal two-service Compose deployment. +- Complete first-run administrator creation in the web UI. +- Browse a local catalog and deploy a Palworld instance from the reference template. +- Manage only registered DoGaMa instances through the restricted agent. +- Assign user or manager access independently for each instance. +- Start, stop, inspect health, view connection information and see available game data. +- Configure ports, resource limits, storage, server settings, mods where supported and update policy. +- Create manual or cron backups with an instance-specific retention count. +- Import data during instance creation or into an existing instance; validate, restore and export safely. +- Update an instance with a pre-update backup and controlled rollback. +- Configure email, generic webhook and Discord notifications from the UI. +- Record a deliberately small audit trail with an administrator-configurable retention, default 30 days. +- Load versioned WebAssembly adapters with explicit capabilities and instance-scoped network access. + +## Explicit non-goals for V1 + +- General Docker administration or display of unrelated containers. +- Commercial hosting, billing, customer quotas, ticketing or public marketplaces. +- Kubernetes, multi-datacenter orchestration or remote multi-node agents. +- Automatic router/NAT/firewall configuration. +- Automatic updates enabled by default. +- Arbitrary shell commands, arbitrary `docker exec`, arbitrary host mounts or arbitrary module networking. +- Native, shared-library, script or container-based integration modules. +- Long-term time-series monitoring or high-volume logs. +- Remote backup providers; the data model may allow a later extension. +- Guaranteed conversion between every local-save and dedicated-server format. + +## Product principles + +1. Safe defaults over clever automation. +2. A preview before deployment or destructive changes. +3. Capabilities drive the UI; unavailable game features are hidden or clearly disabled. +4. Server-specific complexity belongs in a template or minimal adapter, never in generic orchestration code. +5. Human-readable contracts and machine-readable validation evolve together. +6. When compatibility cannot be proven, the interface says “probable” or “unknown”, never “guaranteed”. + diff --git a/docs/security/security-and-threat-model.md b/docs/security/security-and-threat-model.md new file mode 100644 index 0000000..db4ae9f --- /dev/null +++ b/docs/security/security-and-threat-model.md @@ -0,0 +1,62 @@ +# Security and threat model + +## Assets + +- Host control through the Docker socket. +- Player worlds, configuration, mods and backups. +- User accounts, sessions and instance permissions. +- Game API, SMTP and webhook credentials. +- Catalog/template/module integrity. +- Availability of game servers and the management plane. + +## Actors and assumptions + +- Internet users and authenticated non-admin users may be malicious. +- Managers are trusted only for assigned permissions, not host administration. +- Templates, modules, archives, artwork, webhooks and game responses are untrusted. +- The host administrator controls Compose, secret files and bind roots. +- A fully compromised Docker daemon or host is outside DoGaMa's containment guarantee. + +## Principal threats and controls + +| Threat | Required controls | +|---|---| +| Main-app compromise reaches Docker | No socket mount; private authenticated restricted agent; typed operations; registered-instance binding | +| Targeting unrelated containers | No list-all API; database + agent registry + labels + plan digest agreement; opaque IDs | +| Host path escape | Canonical allowlisted roots; symlink-aware validation; no arbitrary template mounts; create-new filesystem operations | +| Malicious archive | Separate staging; traversal/link/device rejection; size/count/depth/time limits; no execution; safe swap | +| Malicious WASM module | No ambient WASI; fuel/memory/time limits; capability/ABI validation; instance-only host networking; circuit breaker | +| SSRF | Scheme/port allowlists, IP classification, redirect revalidation, DNS pinning/rebinding defense for artwork, webhooks and downloads | +| Secret disclosure | Authenticated encryption; external key; write-only API; redaction; no normal export/audit/log inclusion | +| Broken object authorization | Backend instance-scoped checks; deny overrides; object lookup under principal; endpoint denial tests | +| CSRF/session theft | Secure HttpOnly SameSite cookies, CSRF token, TLS guidance, session rotation/revocation and idle/absolute expiry | +| Password attack | Modern password hashing, rate limits, backoff, generic errors, repeated-failure audit/notification | +| Supply-chain substitution | Immutable version snapshots, checksums, optional signatures/trust labels, digest-pinned images, controlled activation | +| Destructive mistake | Preview, recent authentication, typed-name confirmation, pre-restore/update backups and recoverable workflows | +| Resource exhaustion | Upload/extraction limits, job concurrency, per-instance locks, Docker limits, disk checks, notification/module bounds | +| Replay/race | Signed nonce/timestamp agent calls, idempotency keys, optimistic revisions and durable operation phases | + +## Authentication baseline + +V1 local accounts use a current password-hashing algorithm with calibrated parameters. Bootstrap accepts the first administrator only through a one-time local setup state. Sessions rotate at login/privilege change, can be revoked, and never appear in URLs. Critical actions require recent password confirmation. + +The deployment documentation must recommend TLS through a trusted reverse proxy and restrictive permissions on `secrets/`, data and backup paths. + +## Template and module trust + +Display source as `official`, `verified community`, `local`, `locally modified` or `unverified`. Trust is informative but never bypasses validation/sandboxing. Catalog updates cannot overwrite local copies or silently update live instances. + +Artwork downloads accept bounded raster formats, verify decoded content, convert locally and reject SVG in V1. Preserve source attribution metadata without loading remote assets on every page. + +## Security headers and API limits + +Use a restrictive Content Security Policy, frame denial, MIME sniffing protection and explicit referrer policy. Bound request bodies, pagination and expensive query rates. CORS is disabled by default for cross-origin browser clients. Error responses expose stable codes and safe messages, not stack traces. + +## Backup security + +Game backups are not assumed encrypted in V1; filesystem permissions and host backup policy protect them. They contain no DoGaMa secrets. Restore never follows archive links or writes outside declared destinations. Database/system disaster recovery is documented separately from player-data backups. + +## Residual risk + +The agent still holds Docker-equivalent host power. Its restriction reduces exposed functionality and mistakes but is not a sandbox for a fully compromised agent. Keep it small, non-public, dependency-light, fuzz path/plan parsers and treat agent changes as high-risk reviews. + diff --git a/docs/ux/interfaces.md b/docs/ux/interfaces.md new file mode 100644 index 0000000..e2b0461 --- /dev/null +++ b/docs/ux/interfaces.md @@ -0,0 +1,58 @@ +# Administration and manager interfaces + +## Shared design principles + +- Responsive, keyboard-accessible and understandable without Docker vocabulary. +- Capability- and permission-aware: absent functions are hidden or explain why unavailable. +- Always distinguish container state from game readiness. +- Show previews and consequences before deployment, update, restore and deletion. +- Never display stored secret values. + +## Main dashboard + +The default table/cards show name, game, state/readiness, players, CPU, RAM, last backup and permitted quick actions. No dense historical charts are required. Users see only assigned instances; administrators see all. + +## Instance page + +Tabs or clear sections: + +- Overview: artwork, welcome text, readiness, connection address and recent actionable error. +- Players: list/count and allowed announce/kick/ban controls. +- Configuration: safe visible values, pending restart changes and configuration history. +- Resources and storage: limits, current usage, ports, mount categories and disk status. +- Mods: only when supported. +- Backups: list, create/export and authorized restore/import flows. +- Updates: current/candidate image, module/template compatibility and operation history. +- Access: admin-only memberships and overrides. + +## Catalog and requests + +Users browse game cards and submit an installation request with optional name, player estimate, schedule and mods. Administrators approve or refuse with a reason. Approval opens the instance creation wizard; it does not deploy immediately. + +Catalog administration supports validated import, local copy/customization, source/trust display, version comparison and activation. Schema errors point to fields and line/path locations. + +## Creation wizard + +1. Template and version. +2. New world or import existing save. +3. Game settings and write-only secrets. +4. Public ports and connection preview. +5. CPU, memory and storage locations. +6. Mods and optional integration module. +7. Backup schedule/retention and safety policies. +8. Final canonical preview and validation. + +Import validation precedes container creation. Warnings distinguish guaranteed facts from compatibility guesses. + +## Backup and restore UI + +Backup list shows origin, created/imported dates, size, validation, game/template version and checksum status. Cron has common presets, custom expression, timezone and next-run preview. + +Restore shows overwritten data, safety-backup behavior and server downtime. It requires recent authentication and typed confirmation where data is replaced. Results distinguish rolled back, safely stopped and intervention required. + +## Administration + +Global screens cover users, public address, approved storage roots (displayed, bootstrap-controlled where appropriate), catalog/modules, notification channels, audit retention/default 30 days, upload limits, disk thresholds and safety defaults. + +The audit viewer is compact and filterable by time, actor, instance, action and outcome. It is not a raw log console. + diff --git a/modules/palworld-rest/README.md b/modules/palworld-rest/README.md new file mode 100644 index 0000000..3e16a84 --- /dev/null +++ b/modules/palworld-rest/README.md @@ -0,0 +1,40 @@ +# Palworld REST reference adapter + +This is the source specification for DoGaMa's reference WebAssembly adapter. It is intentionally a translator only. Container lifecycle and archive handling stay in generic DoGaMa workflows. + +## Endpoint mapping + +| Normalized operation | Palworld REST operation | +|---|---| +| `test_connection`, `get_server_info` | server info | +| `get_server_status` | info plus metrics readiness | +| `get_metrics` | metrics | +| `list_players` | players | +| `save_world` | save | +| `shutdown` | shutdown with bounded wait/message | +| `send_announcement` | announce | +| `kick_player` | kick by stable player ID | +| `ban_player` | ban by stable player ID | +| `unban_player` | unban by stable player ID | + +The official API category is [Palworld REST API](https://docs.palworldgame.com/category/rest-api/). Implementation must verify exact current paths, methods and response fields against the pinned game/API version and use fixtures for that version. + +## Network and credentials + +The host binds logical handle `instance_api` to template port `rest_api`. The module never constructs a host or accepts a URL. It requests only relative `/v1/api/...` paths. Authentication values come from declared configuration and are placed in the request by module logic; diagnostics must never contain the header or password. + +## Save and shutdown semantics + +`save_world` returns success only after the API acknowledges the save operation. DoGaMa then archives the `saved` mount. `shutdown` sends the in-game shutdown request; generic lifecycle code observes container exit and asks the restricted agent for a bounded stop only when necessary. + +## Build status + +No WebAssembly binary is included in this specification baseline. The zero checksum in `manifest.yaml` is a visible placeholder. The implementation task must: + +1. define the WIT/typed bindings for normalized API v1; +2. implement and test all declared capabilities; +3. compile `module.wasm` without ambient WASI capabilities; +4. replace the placeholder with the real SHA-256; +5. verify runtime limits and offline/unauthorized/malformed-response cases; +6. package the manifest, binary, README and license. + diff --git a/modules/palworld-rest/fixtures/metrics.json b/modules/palworld-rest/fixtures/metrics.json new file mode 100644 index 0000000..bd1fd98 --- /dev/null +++ b/modules/palworld-rest/fixtures/metrics.json @@ -0,0 +1,9 @@ +{ + "serverfps": 57, + "currentplayernum": 2, + "serverframetime": 16.7671, + "maxplayernum": 16, + "uptime": 3600, + "basecampnum": 4, + "days": 12 +} diff --git a/modules/palworld-rest/manifest.yaml b/modules/palworld-rest/manifest.yaml new file mode 100644 index 0000000..d038b2c --- /dev/null +++ b/modules/palworld-rest/manifest.yaml @@ -0,0 +1,61 @@ +schema_version: 1 +id: palworld-rest +name: Palworld REST API adapter +version: 1.0.0 +description: Lightweight adapter from the private Palworld REST API to DoGaMa's normalized module API. +license: Apache-2.0 +homepage: https://docs.palworldgame.com/category/rest-api/ + +game_ids: + - palworld + +runtime: + type: wasm + abi: dogama:game-module@1.0.0 + +compatibility: + manager_api: ">=1.0.0 <2.0.0" + module_api: ">=1.0.0 <2.0.0" + +capabilities: + - server_info + - metrics + - player_list + - online_save + - graceful_shutdown + - announcement + - kick + - ban + - unban + +permissions: + network: + scope: instance_only + protocols: + - http + port_ids: + - rest_api + http_methods: + - GET + - POST + +limits: + memory_mb: 32 + timeout_ms: 10000 + max_response_bytes: 1048576 + max_concurrent_calls: 2 + +configuration: + - id: username + type: string + required: true + description: Palworld REST Basic Authentication username. + default: admin + - id: admin_password + type: secret + required: true + description: Palworld AdminPassword used for private REST Basic Authentication. + +artifacts: + wasm: module.wasm + sha256: "0000000000000000000000000000000000000000000000000000000000000000" diff --git a/specs/module-manifest.schema.json b/specs/module-manifest.schema.json new file mode 100644 index 0000000..a8f7282 --- /dev/null +++ b/specs/module-manifest.schema.json @@ -0,0 +1,99 @@ +{ + "$schema": "https://json-schema.org/draft/2020-12/schema", + "$id": "https://dogama.dev/schemas/module-manifest-v1.json", + "title": "DoGaMa WebAssembly module manifest v1", + "type": "object", + "additionalProperties": false, + "required": ["schema_version", "id", "name", "version", "description", "license", "game_ids", "runtime", "compatibility", "capabilities", "permissions", "configuration", "artifacts"], + "properties": { + "schema_version": { "const": 1 }, + "id": { "$ref": "#/$defs/id" }, + "name": { "type": "string", "minLength": 1, "maxLength": 150 }, + "version": { "$ref": "#/$defs/semver" }, + "description": { "type": "string", "minLength": 1, "maxLength": 1000 }, + "license": { "type": "string", "minLength": 1, "maxLength": 100 }, + "homepage": { "type": "string", "format": "uri", "pattern": "^https://" }, + "game_ids": { "type": "array", "minItems": 1, "uniqueItems": true, "items": { "$ref": "#/$defs/id" } }, + "runtime": { + "type": "object", + "additionalProperties": false, + "required": ["type", "abi"], + "properties": { + "type": { "const": "wasm" }, + "abi": { "const": "dogama:game-module@1.0.0" } + } + }, + "compatibility": { + "type": "object", + "additionalProperties": false, + "required": ["manager_api", "module_api"], + "properties": { + "manager_api": { "type": "string", "minLength": 1, "maxLength": 50 }, + "module_api": { "type": "string", "minLength": 1, "maxLength": 50 } + } + }, + "capabilities": { + "type": "array", + "uniqueItems": true, + "items": { "$ref": "#/$defs/capability" } + }, + "permissions": { + "type": "object", + "additionalProperties": false, + "required": ["network"], + "properties": { + "network": { + "type": "object", + "additionalProperties": false, + "required": ["scope", "protocols", "port_ids"], + "properties": { + "scope": { "const": "instance_only" }, + "protocols": { "type": "array", "minItems": 1, "uniqueItems": true, "items": { "enum": ["http", "https", "tcp"] } }, + "port_ids": { "type": "array", "minItems": 1, "uniqueItems": true, "items": { "$ref": "#/$defs/id" } }, + "http_methods": { "type": "array", "uniqueItems": true, "items": { "enum": ["GET", "POST", "PUT", "PATCH", "DELETE"] } } + } + } + } + }, + "limits": { + "type": "object", + "additionalProperties": false, + "properties": { + "memory_mb": { "type": "integer", "minimum": 1, "maximum": 256 }, + "timeout_ms": { "type": "integer", "minimum": 100, "maximum": 60000 }, + "max_response_bytes": { "type": "integer", "minimum": 1024, "maximum": 16777216 }, + "max_concurrent_calls": { "type": "integer", "minimum": 1, "maximum": 16 } + } + }, + "configuration": { + "type": "array", + "maxItems": 64, + "items": { + "type": "object", + "additionalProperties": false, + "required": ["id", "type", "required"], + "properties": { + "id": { "$ref": "#/$defs/id" }, + "type": { "enum": ["string", "integer", "boolean", "secret"] }, + "required": { "type": "boolean" }, + "description": { "type": "string", "maxLength": 500 }, + "default": {} + } + } + }, + "artifacts": { + "type": "object", + "additionalProperties": false, + "required": ["wasm", "sha256"], + "properties": { + "wasm": { "type": "string", "pattern": "^[A-Za-z0-9._-]+\\.wasm$", "maxLength": 200 }, + "sha256": { "type": "string", "pattern": "^[a-f0-9]{64}$" } + } + } + }, + "$defs": { + "id": { "type": "string", "pattern": "^[a-z0-9]+(?:[._-][a-z0-9]+)*$", "minLength": 1, "maxLength": 100 }, + "semver": { "type": "string", "pattern": "^(0|[1-9][0-9]*)\\.(0|[1-9][0-9]*)\\.(0|[1-9][0-9]*)(?:-[0-9A-Za-z.-]+)?$" }, + "capability": { "enum": ["server_info", "metrics", "player_list", "online_save", "graceful_shutdown", "announcement", "kick", "ban", "unban"] } + } +} diff --git a/specs/normalized-module-api.md b/specs/normalized-module-api.md new file mode 100644 index 0000000..14b49c2 --- /dev/null +++ b/specs/normalized-module-api.md @@ -0,0 +1,81 @@ +# Normalized module API v1 + +## Conventions + +All operations are cancellable and receive a host-enforced deadline. IDs and text returned by a game are untrusted, size-bounded UTF-8. Timestamps use RFC 3339 UTC. Unknown fields are rejected at typed ABI boundaries for V1. + +Standard errors: + +```text +unsupported +invalid_configuration +unauthorized +unreachable +timeout +rate_limited +invalid_response +conflict +game_error +internal_module_error +``` + +Errors contain a safe user message, stable code and optional retryability. Raw credentials, authorization headers and unbounded game responses are prohibited. + +## Required exports + +### `initialize(config) -> module_info` + +Validates declared configuration and returns module ID/version, API version and capabilities. It must not perform long-lived background work. + +### `test_connection() -> connection_result` + +Makes the smallest safe request needed to validate reachability/authentication and reports server/API version hints when available. + +### `get_server_status() -> server_status` + +Returns `starting`, `ready`, `degraded`, `stopping`, `offline` or `unknown`, plus a bounded safe reason. + +Every module implements these three exports. + +## Optional capability exports + +| Capability | Export | Result | +|---|---|---| +| `server_info` | `get_server_info()` | name, game version, max players, optional world/version fields | +| `metrics` | `get_metrics()` | bounded current gauges/counters, no long-term history | +| `player_list` | `list_players()` | stable game player ID, display name, optional joined time/ping | +| `online_save` | `save_world()` | acknowledgement that the game completed/persisted the save | +| `graceful_shutdown` | `shutdown(request)` | accepted/completed state and safe message | +| `announcement` | `send_announcement(message)` | accepted result | +| `kick` | `kick_player(player_id, reason?)` | action result | +| `ban` | `ban_player(player_id, reason?)` | action result | +| `unban` | `unban_player(player_id)` | action result | + +An optional export cannot exist as an enabled UI action unless manifest, runtime report, template and permission agree. + +## Host functions + +### `http_request(instance_api, request)` + +Request fields: relative path, allow-listed method, bounded headers excluding host/connection overrides, bounded body and timeout not exceeding runtime maximum. The host fixes scheme, authority and destination port, validates redirects and bounds the response. + +### `tcp_exchange(instance_api, request)` + +Available only when declared. Sends and receives bounded bytes on the bound declared TCP integration port with fixed connect/read/write deadlines. It is not a general socket handle. + +### `get_secret(key)` + +Returns only a manifest-declared secret configuration item for the current instance. The value cannot be logged through diagnostics and should be consumed immediately. + +### `emit_diagnostic(level, code, message)` + +Accepts bounded structured diagnostic data. The host redacts and rate-limits it. Modules cannot write directly to application logs. + +## Semantic requirements + +- `save_world` success means the game acknowledged completion, not merely request dispatch. +- `shutdown` never stops a Docker container directly; lifecycle code observes container exit and uses the agent for timeout fallback. +- Player actions use stable game IDs, not only display names. +- Metrics use documented normalized names/units. Unknown game-specific metrics may be omitted rather than smuggled into arbitrary maps in V1. +- All calls are side-effect-free except save, shutdown, announcement and player-action exports. + diff --git a/specs/template.schema.json b/specs/template.schema.json new file mode 100644 index 0000000..0ddd25b --- /dev/null +++ b/specs/template.schema.json @@ -0,0 +1,268 @@ +{ + "$schema": "https://json-schema.org/draft/2020-12/schema", + "$id": "https://dogama.dev/schemas/template-v1.json", + "title": "DoGaMa game-server template v1", + "type": "object", + "additionalProperties": false, + "required": ["schema_version", "id", "version", "game", "requirements", "container", "storage", "configuration", "capabilities", "backup", "healthcheck", "imports", "updates", "compatibility"], + "properties": { + "schema_version": { "const": 1 }, + "id": { "$ref": "#/$defs/id" }, + "version": { "$ref": "#/$defs/semver" }, + "source": { + "type": "object", + "additionalProperties": false, + "required": ["type"], + "properties": { + "type": { "enum": ["official", "verified_community", "local", "locally_modified", "unverified"] }, + "url": { "type": "string", "format": "uri", "pattern": "^https://" } + } + }, + "game": { + "type": "object", + "additionalProperties": false, + "required": ["id", "name", "description"], + "properties": { + "id": { "$ref": "#/$defs/id" }, + "name": { "type": "string", "minLength": 1, "maxLength": 100 }, + "description": { "type": "string", "minLength": 1, "maxLength": 2000 }, + "website": { "type": "string", "format": "uri", "pattern": "^https://" }, + "artwork": { + "type": "object", + "additionalProperties": false, + "properties": { + "poster_source_url": { "type": "string", "format": "uri", "pattern": "^https://" }, + "logo_source_url": { "type": "string", "format": "uri", "pattern": "^https://" }, + "attribution": { "type": "string", "maxLength": 500 } + } + } + } + }, + "requirements": { + "type": "object", + "additionalProperties": false, + "required": ["minimum", "recommended"], + "properties": { + "minimum": { "$ref": "#/$defs/resources" }, + "recommended": { "$ref": "#/$defs/resources" } + } + }, + "container": { + "type": "object", + "additionalProperties": false, + "required": ["image", "tag", "stop_timeout_seconds", "ports"], + "properties": { + "image": { "type": "string", "pattern": "^[a-zA-Z0-9._/-]+$", "maxLength": 300 }, + "tag": { "type": "string", "pattern": "^[a-zA-Z0-9._-]+$", "maxLength": 128 }, + "entrypoint": { "type": "array", "items": { "type": "string", "maxLength": 500 }, "maxItems": 8 }, + "arguments": { "type": "array", "items": { "type": "string", "maxLength": 500 }, "maxItems": 64 }, + "assets": { + "type": "array", + "maxItems": 16, + "items": { + "type": "object", + "additionalProperties": false, + "required": ["source", "destination", "sha256", "read_only"], + "properties": { + "source": { "type": "string", "pattern": "^(?!/)(?!.*\\.\\./).+$", "maxLength": 200 }, + "destination": { "type": "string", "pattern": "^/[^\\u0000]*$", "maxLength": 500 }, + "sha256": { "type": "string", "pattern": "^[a-f0-9]{64}$" }, + "read_only": { "const": true } + } + } + }, + "stop_timeout_seconds": { "type": "integer", "minimum": 5, "maximum": 900 }, + "ports": { + "type": "array", + "minItems": 1, + "maxItems": 32, + "items": { + "type": "object", + "additionalProperties": false, + "required": ["id", "container_port", "protocol", "purpose", "publish"], + "properties": { + "id": { "$ref": "#/$defs/id" }, + "container_port": { "type": "integer", "minimum": 1, "maximum": 65535 }, + "protocol": { "enum": ["tcp", "udp"] }, + "purpose": { "enum": ["game", "query", "integration", "other"] }, + "publish": { "type": "boolean" }, + "required": { "type": "boolean", "default": true } + } + } + } + } + }, + "storage": { + "type": "object", + "additionalProperties": false, + "required": ["mounts"], + "properties": { + "mounts": { + "type": "array", + "minItems": 1, + "maxItems": 16, + "items": { + "type": "object", + "additionalProperties": false, + "required": ["id", "container_path", "category", "backup", "user_configurable"], + "properties": { + "id": { "$ref": "#/$defs/id" }, + "container_path": { "type": "string", "pattern": "^/[^\\u0000]*$", "maxLength": 500 }, + "category": { "enum": ["runtime", "configuration", "player_data", "mods", "logs"] }, + "backup": { "type": "boolean" }, + "user_configurable": { "type": "boolean" }, + "read_only": { "type": "boolean", "default": false } + } + } + } + } + }, + "configuration": { + "type": "object", + "additionalProperties": false, + "required": ["fields"], + "properties": { + "fields": { + "type": "array", + "maxItems": 256, + "items": { "$ref": "#/$defs/configField" } + } + } + }, + "capabilities": { + "type": "array", + "uniqueItems": true, + "items": { "$ref": "#/$defs/capability" } + }, + "integration": { + "type": "object", + "additionalProperties": false, + "required": ["module_id", "version_range", "port_id"], + "properties": { + "module_id": { "$ref": "#/$defs/id" }, + "version_range": { "type": "string", "minLength": 1, "maxLength": 50 }, + "port_id": { "$ref": "#/$defs/id" }, + "required": { "type": "boolean", "default": false } + } + }, + "backup": { + "type": "object", + "additionalProperties": false, + "required": ["strategy", "source_mounts"], + "properties": { + "strategy": { "enum": ["online_save", "stop_then_archive"] }, + "source_mounts": { "type": "array", "minItems": 1, "uniqueItems": true, "items": { "$ref": "#/$defs/id" } }, + "restart_after_backup": { "type": "boolean", "default": true } + } + }, + "healthcheck": { + "type": "object", + "additionalProperties": false, + "required": ["type", "startup_timeout_seconds"], + "properties": { + "type": { "enum": ["module", "tcp", "udp", "container"] }, + "port_id": { "$ref": "#/$defs/id" }, + "startup_timeout_seconds": { "type": "integer", "minimum": 10, "maximum": 1800 }, + "interval_seconds": { "type": "integer", "minimum": 2, "maximum": 300, "default": 10 } + } + }, + "mods": { + "type": "object", + "additionalProperties": false, + "required": ["supported"], + "properties": { + "supported": { "type": "boolean" }, + "provider": { "enum": ["steam_workshop", "remote_archive", "local_upload"] }, + "destination_mount": { "$ref": "#/$defs/id" }, + "restart_required": { "type": "boolean" } + } + }, + "imports": { + "type": "object", + "additionalProperties": false, + "required": ["supported", "accepted_formats", "max_extracted_size_gb", "required_paths", "destination_mount"], + "properties": { + "supported": { "type": "boolean" }, + "accepted_formats": { "type": "array", "uniqueItems": true, "items": { "enum": ["zip", "tar", "tar.gz", "tar.zst", "directory"] } }, + "max_extracted_size_gb": { "type": "integer", "minimum": 1, "maximum": 10000 }, + "required_paths": { "type": "array", "items": { "type": "string", "pattern": "^(?!/)(?!.*\\.\\./).+$", "maxLength": 300 } }, + "destination_mount": { "$ref": "#/$defs/id" }, + "destination_relative_path": { "type": "string", "pattern": "^(?!/)(?!.*\\.\\./).*$", "maxLength": 300 }, + "requires_stopped_server": { "type": "boolean", "default": true } + } + }, + "updates": { + "type": "object", + "additionalProperties": false, + "required": ["backup_before_update", "automatic_default", "rollback_on_failure"], + "properties": { + "backup_before_update": { "type": "boolean" }, + "automatic_default": { "const": false }, + "rollback_on_failure": { "type": "boolean" }, + "health_timeout_seconds": { "type": "integer", "minimum": 10, "maximum": 1800 } + } + }, + "compatibility": { + "type": "object", + "additionalProperties": false, + "required": ["minimum_manager_version", "requires_instance_migration"], + "properties": { + "minimum_manager_version": { "$ref": "#/$defs/semver" }, + "requires_instance_migration": { "type": "boolean" } + } + } + }, + "$defs": { + "id": { "type": "string", "pattern": "^[a-z0-9]+(?:[._-][a-z0-9]+)*$", "minLength": 1, "maxLength": 100 }, + "semver": { "type": "string", "pattern": "^(0|[1-9][0-9]*)\\.(0|[1-9][0-9]*)\\.(0|[1-9][0-9]*)(?:-[0-9A-Za-z.-]+)?$" }, + "resources": { + "type": "object", + "additionalProperties": false, + "required": ["cpu_cores", "memory_mb", "storage_gb"], + "properties": { + "cpu_cores": { "type": "number", "exclusiveMinimum": 0, "maximum": 1024 }, + "memory_mb": { "type": "integer", "minimum": 128 }, + "storage_gb": { "type": "integer", "minimum": 1 } + } + }, + "capability": { "enum": ["server_info", "metrics", "player_list", "online_save", "graceful_shutdown", "announcement", "kick", "ban", "unban"] }, + "configField": { + "type": "object", + "additionalProperties": false, + "required": ["id", "label", "type", "target", "apply", "visibility", "required"], + "properties": { + "id": { "$ref": "#/$defs/id" }, + "label": { "type": "string", "minLength": 1, "maxLength": 150 }, + "description": { "type": "string", "maxLength": 1000 }, + "type": { "enum": ["string", "integer", "number", "boolean", "secret", "enum"] }, + "target": { + "type": "object", + "additionalProperties": false, + "required": ["kind", "name"], + "properties": { + "kind": { "enum": ["environment", "argument", "ini"] }, + "name": { "type": "string", "minLength": 1, "maxLength": 200 } + } + }, + "apply": { "enum": ["immediate", "restart_required"] }, + "visibility": { "enum": ["public", "members", "managers", "admin", "secret"] }, + "required": { "type": "boolean" }, + "default": {}, + "minimum": { "type": "number" }, + "maximum": { "type": "number" }, + "pattern": { "type": "string", "maxLength": 300 }, + "values": { + "type": "array", + "minItems": 1, + "items": { + "anyOf": [ + { "type": "string" }, + { "type": "number" }, + { "type": "boolean" } + ] + } + } + } + } + } +} diff --git a/tools/requirements-validation.txt b/tools/requirements-validation.txt new file mode 100644 index 0000000..2fdf413 --- /dev/null +++ b/tools/requirements-validation.txt @@ -0,0 +1,3 @@ +PyYAML==6.0.2 +jsonschema==4.25.0 + diff --git a/tools/validate_spec.py b/tools/validate_spec.py new file mode 100644 index 0000000..99115c7 --- /dev/null +++ b/tools/validate_spec.py @@ -0,0 +1,137 @@ +#!/usr/bin/env python3 +"""Validate DoGaMa specification schemas, examples and internal links.""" + +from __future__ import annotations + +import hashlib +import json +import re +import sys +from pathlib import Path + +import yaml +from jsonschema import Draft202012Validator + + +ROOT = Path(__file__).resolve().parent.parent + + +def load_json(path: Path): + with path.open("r", encoding="utf-8") as handle: + return json.load(handle) + + +def load_yaml(path: Path): + with path.open("r", encoding="utf-8") as handle: + return yaml.safe_load(handle) + + +def validate(schema_path: Path, document_path: Path) -> dict: + schema = load_json(schema_path) + Draft202012Validator.check_schema(schema) + document = load_yaml(document_path) + errors = sorted(Draft202012Validator(schema).iter_errors(document), key=lambda item: list(item.path)) + if errors: + rendered = "\n".join(f" {document_path}:{'/'.join(map(str, error.path))}: {error.message}" for error in errors) + raise ValueError(f"Schema validation failed:\n{rendered}") + return document + + +def validate_links() -> None: + pattern = re.compile(r"\[[^]]+\]\(([^)]+)\)") + failures = [] + for markdown in ROOT.rglob("*.md"): + text = markdown.read_text(encoding="utf-8") + for target in pattern.findall(text): + target = target.split("#", 1)[0].strip() + if not target or target.startswith(("http://", "https://", "mailto:")): + continue + resolved = (markdown.parent / target).resolve() + if not resolved.exists(): + failures.append(f"{markdown.relative_to(ROOT)} -> {target}") + if failures: + raise ValueError("Broken internal links:\n " + "\n ".join(failures)) + + +def validate_cross_references(template: dict, manifest: dict) -> list[str]: + warnings = [] + port_ids = {item["id"] for item in template["container"]["ports"]} + mount_ids = {item["id"] for item in template["storage"]["mounts"]} + capabilities = set(template["capabilities"]) + + integration = template.get("integration") + if integration: + assert integration["port_id"] in port_ids, "Template integration references an unknown port" + assert integration["module_id"] == manifest["id"], "Template and manifest module IDs disagree" + assert template["game"]["id"] in manifest["game_ids"], "Manifest does not support template game ID" + assert set(manifest["permissions"]["network"]["port_ids"]) <= port_ids, "Manifest references an unknown template port" + assert set(manifest["capabilities"]) == capabilities, "Template and reference manifest capabilities disagree" + + assert set(template["backup"]["source_mounts"]) <= mount_ids, "Backup references an unknown mount" + assert template["imports"]["destination_mount"] in mount_ids, "Import references an unknown mount" + mods = template.get("mods", {}) + if mods.get("supported"): + assert mods.get("destination_mount") in mount_ids, "Mods reference an unknown mount" + + for asset in template["container"].get("assets", []): + path = (ROOT / "catalog" / "palworld" / asset["source"]).resolve() + assert path.is_file(), f"Missing packaged asset: {path}" + digest = hashlib.sha256(path.read_bytes()).hexdigest() + assert digest == asset["sha256"], f"Asset checksum mismatch: {path}" + + wasm_digest = manifest["artifacts"]["sha256"] + wasm_path = ROOT / "modules" / manifest["id"] / manifest["artifacts"]["wasm"] + if wasm_digest == "0" * 64 and not wasm_path.exists(): + warnings.append("Palworld module is a source specification: module.wasm and its final checksum are intentionally pending.") + elif wasm_path.is_file(): + assert hashlib.sha256(wasm_path.read_bytes()).hexdigest() == wasm_digest, "WASM checksum mismatch" + else: + raise AssertionError("Manifest names a missing WASM artifact without the documented placeholder") + + return warnings + + +def validate_coverage() -> None: + required = { + "Docker agent": "docs/architecture/docker-agent.md", + "WebAssembly": "docs/architecture/wasm-modules.md", + "threat model": "docs/security/security-and-threat-model.md", + "SQLite": "docs/domain/data-model.md", + "backup": "docs/operations/backups-import-export.md", + "Discord": "docs/operations/notifications-and-audit.md", + "30 days": "docs/operations/notifications-and-audit.md", + "manager": "docs/domain/authorization.md", + "Palworld": "catalog/palworld/README.md", + "acceptance": "docs/product/acceptance-criteria.md", + "roadmap": "docs/product/roadmap.md", + "Codex": "docs/contributing/ai-codex-guide.md", + } + missing = [] + for needle, relative in required.items(): + if needle.casefold() not in (ROOT / relative).read_text(encoding="utf-8").casefold(): + missing.append(f"{needle!r} in {relative}") + if missing: + raise ValueError("Missing required coverage: " + ", ".join(missing)) + + +def main() -> int: + template = validate(ROOT / "specs/template.schema.json", ROOT / "catalog/palworld/template.yaml") + manifest = validate(ROOT / "specs/module-manifest.schema.json", ROOT / "modules/palworld-rest/manifest.yaml") + load_yaml(ROOT / "compose.yaml") + for fixture in ROOT.rglob("*.json"): + load_json(fixture) + validate_links() + warnings = validate_cross_references(template, manifest) + validate_coverage() + print("DoGaMa specification validation passed.") + for warning in warnings: + print(f"WARNING: {warning}") + return 0 + + +if __name__ == "__main__": + try: + raise SystemExit(main()) + except Exception as error: + print(f"ERROR: {error}", file=sys.stderr) + raise SystemExit(1)