fix(docker): route staging's runtime pip fallback through internal PyPI mirror

[mekarpeles]

Summary

docker/ol-install-missing-deps.sh (added in #12984 alongside the new pydantic-settings pin) runs a bare pip install on every container start when the package isn't already importable:

python -c "import pydantic_settings" 2>/dev/null || python -m pip install -q --user 'pydantic-settings==2.9.1'

This never referenced PIP_INDEX_URL (Archive's internal Nexus PyPI mirror — the only path allowed through the firewall). On the testing/staging server, where live bind-mounted code (merged via make-integration-branch.sh) can be ahead of whatever olbase image is cached locally, the import check genuinely fails and the fallback hits the public PyPI index directly, which the firewall blocks.

This PR grew through three commits as live testing on ol-dev1 surfaced each layer of the gap in turn:

1. Runtime fallback install now routes through the mirror: adds --index-url "${PIP_INDEX_URL:-https://pypi.org/simple/}" to the fallback install, and forwards PIP_INDEX_URL into compose.staging.yaml's web/fast_web runtime environment: (it wasn't being passed through at all before).
2. Don't depend on the operator's shell state: the fix in (1) only works if whoever runs docker compose up happened to source /opt/olsystem/bin/build_env.sh first in that exact shell session — nothing enforces it, and it's easy to forget (confirmed live: a docker compose up without sourcing it first reproduced the identical failure). Since /olsystem is already bind-mounted into the container, ol-install-missing-deps.sh now sources build_env.sh from there directly when PIP_INDEX_URL isn't already set, so it no longer depends on host shell state at all.
3. PIP_INDEX_URL also needs to be a build arg, not just a runtime env var: after (1) and (2) fixed the runtime pydantic-settings fallback, a rebuild on ol-dev1 still failed with ModuleNotFoundError: No module named 'web' — web.py installs from a git source (git+https://github.com/webpy/webpy.git@...) at build time via Dockerfile.oldev's RUN uv pip install -r requirements_test.txt, which has no runtime fallback at all. Dockerfile.oldev already declares ARG PIP_INDEX_URL/ARG UV_DEFAULT_INDEX for exactly this, but compose.staging.yaml never wired it into the build: section — only ever into the runtime environment. Added args: - PIP_INDEX_URL=${PIP_INDEX_URL:-} to both web and fast_web.

Production is not affected by any of this today — its containers only pull pre-built openlibrary/olbase:latest images (rebuilt weekly on GitHub Actions with unrestricted internet), so none of these fallback/build paths ever execute there. But the same scripts are deployed there too, so this closes latent gaps, not just staging-only ones.

Testing

1. Compose config resolves correctly for both runtime env and build args:

   $ HOSTNAME=test-host PIP_INDEX_URL=https://nexus.us.archive.org/repository/pypi-proxy/simple/ \
       docker compose -f compose.yaml -f compose.staging.yaml config
   ...
     build:
       args:
         PIP_INDEX_URL: https://nexus.us.archive.org/repository/pypi-proxy/simple/
     environment:
       PIP_INDEX_URL: https://nexus.us.archive.org/repository/pypi-proxy/simple/
   

2. Reproduced the original crash in a real python:3.14.5-slim-trixie container (matches Dockerfile.olbase's pin) by pointing the unfixed install line at an unreachable index — got the identical error from the incident.

3. Ran the actual patched fallback script against the real openlibrary/olbase:latest image (pulled fresh): confirmed the image already has pydantic_settings baked in, forcibly removed it to simulate the staging drift scenario, and confirmed the patched docker/ol-install-missing-deps.sh self-heals — both via an explicitly-set PIP_INDEX_URL and via the new /olsystem/bin/build_env.sh auto-source fallback (verified with a stand-in build_env.sh mounted at /olsystem, confirming it's only sourced when PIP_INDEX_URL isn't already set, and that an already-set value takes precedence).

4. Ran a real docker build -f docker/Dockerfile.oldev --build-arg PIP_INDEX_URL=https://pypi.org/simple/: completes successfully, and import web succeeds in the resulting image — confirms the build-arg wiring actually reaches the uv pip install step that installs web.py.

5. Live-verified on ol-dev1 through the actual incident: confirmed PIP_INDEX_URL blank causes the exact reported failure, confirmed sourcing build_env.sh and rebuilding fixes the pydantic-settings runtime path, then hit the separate web.py build-time gap, which commit 3 addresses. Awaiting a full rebuild + boot confirmation on ol-dev1 with all three commits applied.

Pre-commit passed clean on all changed files (locally-runnable hooks; mypy/generate-pot fail locally as documented — they require Docker/the infogami submodule).

Related

Closes #13177

[jimchamp]

Give it a try.