alt-clowns

Monorepo for the ALT achievements web application.

Structure

• backend/ FastAPI application, source collectors, metrics, awards, CLI, public API
• frontend/ Next.js public UI and admin UI
• scripts/ Podman preview and container helper scripts

Status

Current stage: public MVP with public read-only pages, admin UI, award catalog seeding, separate import and recalculation flows, and Podman preview on PostgreSQL.

License

This project is licensed under the GNU Affero General Public License v3.0 or later. See LICENSE.

Development

Backend:

cd backend
python3 -m venv .venv
source .venv/bin/activate
pip3 install -r requirements-dev.txt
pytest3
python3 -m uvicorn app.main:app --reload

Frontend:

cd frontend
npm install
npm test
npm run dev

Selected focused test runs:

pytest3 backend/tests/test_importers.py backend/tests/test_awards.py backend/tests/test_public_api.py
pytest3 backend/tests/test_import_cli.py backend/tests/test_evaluate_awards_cli.py

Podman Preview

Build and run the preview stack:

./scripts/run_podman_preview.sh

To run the local preview against an external PostgreSQL instance instead of the local alt-clowns-db container, set ALT_CLOWNS_DATABASE_URL before starting the local app container.

Example:

ALT_CLOWNS_DATABASE_URL='postgresql+psycopg2://<db-user>:<db-password>@<db-host>:5432/<db-name>' \
./scripts/run_podman_preview.sh

This starts:

• alt-clowns-db on registry.altlinux.org/p11/postgresql:latest
• alt-clowns-preview on registry.altlinux.org/p11/python

The site is published on all interfaces on port 8000:

http://<host-ip>:8000/

For a production-style deployment, the rootless Podman containers can be managed by user-level systemd units for the deployment user:

• container-alt-clowns-db.service
• container-alt-clowns-preview.service

Useful remote commands:

ssh <deploy-user>@<deploy-host> 'systemctl --user status container-alt-clowns-db.service container-alt-clowns-preview.service'
ssh <deploy-user>@<deploy-host> 'systemctl --user restart container-alt-clowns-db.service container-alt-clowns-preview.service'
ssh <deploy-user>@<deploy-host> 'podman ps --format "{{.Names}} {{.Status}} {{.Ports}}"'

Runtime Notes

• PostgreSQL is the active runtime database for preview.
• The runtime container uses registry.altlinux.org/p11/python.
• Frontend is built in a separate registry.altlinux.org/p11/node stage and served as static export by FastAPI.
• Python runtime modules in the container are installed from ALT packages, not from PyPI.
• PostgreSQL runs in a separate registry.altlinux.org/p11/postgresql:latest container.
• /admin and /admin/* are protected with HTTP Basic authentication.
• The default admin username is admin.
• Set the password with ALT_CLOWNS_ADMIN_PASSWORD.
• The preview script migrates data from local alt_clowns.db into PostgreSQL only when the target database is empty.
• Imported upstream activity is persisted locally and used as the basis for recalculation.

Data Import And Recalculation

Import and award recalculation are separate operations.

Import source data:

PYTHONPATH=backend python3 -m app.cli.import_data --source bugzilla --start 2026-03-01 --end 2026-03-31 --period-type month
PYTHONPATH=backend python3 -m app.cli.import_data --source rdb --start 2026-03-01 --end 2026-03-31 --period-type month --branch sisyphus
PYTHONPATH=backend python3 -m app.cli.import_data --source rdb --start 2026-03-01 --end 2026-03-31 --period-type month --branch sisyphus --nickname gamzin --nickname geochip

Recalculate awards from already imported snapshots:

PYTHONPATH=backend python3 -m app.cli.evaluate_awards --period-type month --disable-moderation
PYTHONPATH=backend python3 -m app.cli.evaluate_awards --definition-code tracker-closer-crown --period-type month
PYTHONPATH=backend python3 -m app.cli.evaluate_awards --snapshot-id <snapshot-id>

The admin UI also supports award recalculation through a dedicated form:

• open /admin/
• choose period
• optionally enter snapshot_id for a targeted recalculation
• run Пересчитать награды

Automatic awards are currently configured to publish immediately. The moderation queue is only used for grants that still need a manual decision.

The admin UI also supports local Bugzilla metric recomputation without upstream API calls:

• open /admin/
• choose Bugzilla, year, and optionally a single month
• optionally enable Собрать квартал и год
• run Пересчитать локально

Run a full preview benchmark that clears PostgreSQL, imports Bugzilla and RDB in parallel for the selected year and month, recalculates awards, and appends timing records to .data/import-benchmarks/history.jsonl:

PYTHONPATH=backend python3 -m app.cli.historical_benchmark --year 2025 --month 2026-01

Run a full-year optimized backfill that clears PostgreSQL, imports every month of the selected year in parallel, aggregates quarter and year snapshots from canonical monthly data, and records timings for every step:

PYTHONPATH=backend python3 -m app.cli.year_backfill_benchmark --year 2025 --bugzilla-rps 3 --rdb-rps 10

Run a descending multi-year import without clearing the database between years:

ALT_CLOWNS_START_YEAR=2024 ALT_CLOWNS_END_YEAR=2023 ALT_CLOWNS_IMPORT_LIMIT=0 ./scripts/import_years_desc.sh

The same script is also available inside the preview container image:

podman exec -it \
  -e ALT_CLOWNS_DATABASE_URL='postgresql+psycopg2://<db-user>:<db-password>@<db-host>:5432/<db-name>' \
  -e ALT_CLOWNS_BUGZILLA_REQUESTS_PER_SECOND=3 \
  -e ALT_CLOWNS_RDB_REQUESTS_PER_SECOND=7 \
  -e ALT_CLOWNS_IMPORT_LIMIT=0 \
  -e ALT_CLOWNS_START_YEAR=2024 \
  -e ALT_CLOWNS_END_YEAR=2010 \
  alt-clowns-preview \
  sh -lc 'cd /app && ./scripts/import_years_desc.sh'

The descending import script:

• loads Bugzilla and RDB month snapshots in parallel for every month of each year when ALT_CLOWNS_DATABASE_URL points to PostgreSQL
• automatically switches to sequential month imports when ALT_CLOWNS_DATABASE_URL points to SQLite, to avoid database is locked
• keeps RDB on sisyphus
• aggregates quarter and year snapshots from canonical monthly data
• recalculates awards after each imported year
• defaults to 3 rps for both sources unless you override ALT_CLOWNS_BUGZILLA_REQUESTS_PER_SECOND or ALT_CLOWNS_RDB_REQUESTS_PER_SECOND
• defaults to ALT_CLOWNS_IMPORT_LIMIT=0, which means full monthly imports; set a positive value only for smoke tests
• uses ALT_CLOWNS_RDB_TIMEOUT_SECONDS for slow task_info responses; default is 60 seconds

For long-running backfills, prefer PostgreSQL. SQLite is acceptable for local development and single-process imports, but it is not a good fit for concurrent monthly source imports.

Aggregate larger periods from canonical monthly snapshots without re-querying upstream APIs:

PYTHONPATH=backend python3 -m app.cli.aggregate_period --source bugzilla --period-type quarter --year 2025 --quarter 1
PYTHONPATH=backend python3 -m app.cli.aggregate_period --source rdb --period-type year --year 2025

Recompute canonical monthly Bugzilla metrics from locally stored bugs and comments without calling upstream APIs, then optionally rebuild quarter and year snapshots from the recomputed months:

PYTHONPATH=backend python3 -m app.cli.recompute_local_metrics --source bugzilla --year 2025
PYTHONPATH=backend python3 -m app.cli.recompute_local_metrics --source bugzilla --year 2025 --month 10
PYTHONPATH=backend python3 -m app.cli.recompute_local_metrics --source bugzilla --year 2025 --aggregate-periods

Snapshot Rules

The project distinguishes two kinds of import windows:

• canonical period snapshots
• ad-hoc diagnostic snapshots

Canonical snapshots are used by default for:

• public leaderboards
• period-based automatic award recalculation

Ad-hoc snapshots are kept for diagnostics and targeted checks, but they are ignored by normal monthly, quarterly, and yearly recalculation unless you explicitly pass --snapshot-id.

Canonical behavior:

• the same source + period_type + period_start + period_end reuses one snapshot and replaces its metrics instead of appending duplicates
• overlapping or nested windows do not participate in normal periodic award recalculation unless explicitly targeted
• Bugzilla imports fetch user profiles lazily only for participants missing locally or considered stale, instead of reloading all participants every time
• Bugzilla historical imports can skip comment enrichment with --skip-comments when the current benchmark only needs award-driving metrics
• imported Bugzilla bugs and comments are stored locally, and period metrics are built from those local tables
• RDB period imports now use branch task history as the primary source for period metrics instead of walking package task histories one package at a time
• RDB task details are cached locally by task_id, so repeated imports can reuse already fetched task metadata when the upstream task timestamp has not changed
• normalized RDB task package events are stored locally, and period metrics are rebuilt from those local task records instead of in-memory upstream responses
• RDB imports keep a persistent checkpoint per period window so repeated runs can resume from already materialized tasks
• quarter and year snapshots can be rebuilt from canonical monthly snapshots, so long historical imports should prefer month-by-month loading plus local aggregation

Practical guidance:

• use full calendar windows for production imports
• month: full month
• quarter: full quarter
• year: full year
• use partial windows only for debugging or exploratory imports

Historical Import Benchmarks

• The benchmark CLI clears the active preview PostgreSQL schema before loading data.
• Timings are stored per operation in .data/import-benchmarks/history.jsonl.
• Full stdout and stderr logs for every step are stored under .data/import-benchmarks/<run-id>/.
• Bugzilla and RDB imports for the same period run in parallel as separate processes.
• The optimized yearly benchmark imports the year month-by-month, then aggregates quarter and year snapshots locally from those canonical months.
• Award recalculation runs after imports complete, as a separate phase.
• The benchmark keeps per-source request rates configurable through ALT_CLOWNS_BUGZILLA_REQUESTS_PER_SECOND and ALT_CLOWNS_RDB_REQUESTS_PER_SECOND.

Useful CLI Commands

Deduplicate award grants that were created before idempotent evaluation was introduced:

PYTHONPATH=backend python3 -m app.cli.dedupe_awards

Copy data from one application database to another:

PYTHONPATH=backend python3 -m app.cli.migrate_database --source-url sqlite:////absolute/path/to/alt_clowns.db --target-url postgresql+psycopg2://<db-user>:<db-password>@<db-host>:5432/<db-name>

Maintenance

• Keep this README in sync with user-facing workflows, CLI commands, container behavior, and import/recalculation rules as development continues.
• Award icon uploads normalize raster files to 256x256 during upload while leaving SVG files untouched. Existing raster icons can be re-optimized in place with python3 -m app.cli.optimize_award_icons.