🔴 Live TV: internetarchive/openlibrary/pull/

Sanket17052006 · 2026-05-15T08:11:23Z

Closes #

This PR migrates import_preview_json from web.py to FastAPI.

Technical

Added openlibrary/fastapi/importapi.py with GET /import/preview and POST /import/preview endpoints
Marked legacy import_preview_json class in import_ui.py with @deprecated("migrated to fastapi")
Registered the new router in asgi_app.py

Testing

docker compose run --rm home python -m pytest openlibrary/tests/fastapi/test_importapi.py -v

Screenshot

Stakeholders

Copilot

Pull request overview

This PR migrates the import preview JSON handler toward FastAPI by adding a new router, registering it in the ASGI app, marking the legacy JSON handler deprecated, and adding endpoint tests.

Changes:

Adds FastAPI GET/POST handlers for import preview JSON behavior.
Registers the import API router in the main ASGI app.
Adds FastAPI tests for auth, save behavior, and invalid source handling.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.

File	Description
`openlibrary/fastapi/importapi.py`	Adds FastAPI import preview endpoints and shared request handling.
`openlibrary/asgi_app.py`	Includes the new import API router.
`openlibrary/plugins/importapi/import_ui.py`	Marks the legacy JSON preview handler as deprecated.
`openlibrary/tests/fastapi/test_importapi.py`	Adds tests for the new FastAPI import preview endpoints.

Comments suppressed due to low confidence (1)

openlibrary/fastapi/importapi.py:84

Registering POST on /import/preview intercepts the existing HTML form posts from openlibrary/templates/import_preview.html. Those posts currently render a preview or redirect after saving, but this FastAPI handler always returns JSON, so the import preview UI's submit/import flow will break; use the legacy JSON endpoint URL (such as the .json route) or keep the HTML route handled by the legacy page.

@router.post("/import/preview")

RayBB

In general it looks like it's in the right direction.

Doesn't seem to be working. Please test the endpoints before asking for review.
http://localhost:18080/import/preview.json?provider=ia&identifier=roughcutmystery00gorm

We'll have to use site.get() instead of web.ctx.site

The tests could also be DRY'd up with fixtures for mocking current user, admin user, import request.

So please go head and fix this and ensure it actually works locally for both new endpoints. Give the examples you used to test too.

Sanket17052006 · 2026-05-28T10:11:33Z

Thanks for the review! I'll fix the issues and test the endpoint thoroughly before updating.

Sanket17052006 · 2026-05-28T17:30:21Z

Hey @RayBB , I've fixed the issues locally - these are the examples I used :

curl -b cookies.txt "http://localhost:18080/import/preview.json?provider=ia&identifier=roughcutmystery00gorm"
{"success": true, "edition": {"key": "/books/OL10M", "status": "matched"}, "work": {"key": "/works/OL1W", "status": "matched"}, "preview": true}

POST with save:

curl -b cookies.txt -X POST "http://localhost:18080/import/preview.json" -d "provider=ia&identifier=roughcutmystery00gorm&save=true"
{"success": true, "edition": {"key": "/books/OL10M", "status": "matched"}, "work": {"key": "/works/OL1W", "status": "matched"}}

No auth:

curl "http://localhost:18080/import/preview.json?provider=ia&identifier=roughcutmystery00gorm"
{"detail": "Authentication required"} (401)

RayBB

Looks close but a few more things to fix up.

Please remove the infogami changes.

Also, do not set web.ctx.site. We do not use ctx in the fastapi context. We need to replace the usage of web.ctx.site with site.get() wherever it's called. Not use adding it ot the top of the function.

for more information, see https://pre-commit.ci

RayBB

I added LibrarianDep but other than that this was basically perfect. Thanks for the hard work here!

Adds docs/ai/imports/index.md — a system reference for AI agents and contributors working on the Open Library import pipeline. Covers: - Architecture overview (IA import, MARC, batch, partner sources) - Key files with descriptions and links - API endpoints including FastAPI /import/preview.json (PR #12745) - Validation gate: CompleteBook / StrongIdentifierBook two-model logic - Common failure modes including not-differentiable (issue #10756) - Open issues and roadmap items

* docs(ai/imports): import pipeline architecture reference Adds docs/ai/imports/index.md — a system reference for AI agents and contributors working on the Open Library import pipeline. Covers: - Architecture overview (IA import, MARC, batch, partner sources) - Key files with descriptions and links - API endpoints including FastAPI /import/preview.json (PR #12745) - Validation gate: CompleteBook / StrongIdentifierBook two-model logic - Common failure modes including not-differentiable (issue #10756) - Open issues and roadmap items * docs(ai/imports): expand roadmap with feed registry, trust tiers, and sequencing * docs(ai/imports): split monolithic index into focused sub-docs Break the 460-line index.md into five focused files: - index.md: architecture overview, key files, cross-links, open PRs - api.md: endpoints, bulk MARC, local_id, batch import, IA import - validation.md: validation gate, required fields, not-differentiable gap - adding-sources.md: adapter pattern, identifier registration, PR review expectations - debugging.md: known limitations, debug playbook Roadmap content removed from contributor docs — lives in agent-land (~/.cmux/impa/notes/import-roadmap.md, pm/workflows/import_workflow.md).

mekarpeles requested a review from Copilot May 15, 2026 16:35

Copilot started reviewing on behalf of mekarpeles May 15, 2026 16:36 View session

Copilot AI reviewed May 15, 2026

View reviewed changes

Comment thread openlibrary/fastapi/importapi.py Outdated

mekarpeles assigned RayBB May 18, 2026

github-project-automation Bot added this to Ray's Project May 18, 2026

github-project-automation Bot moved this to Waiting Review/Merge from Staff in Ray's Project May 18, 2026

RayBB requested changes May 28, 2026

View reviewed changes

github-project-automation Bot moved this from Waiting Review/Merge from Staff to Someone else is working on it in Ray's Project May 28, 2026

Sanket17052006 force-pushed the migrate-import-preview-json branch from e97c8ff to d4c3260 Compare May 28, 2026 17:13

github-actions Bot added the Needs: Response Issues which require feedback from lead label May 29, 2026

RayBB requested changes Jun 2, 2026

View reviewed changes

Comment thread openlibrary/fastapi/importapi.py Outdated

Comment thread openlibrary/fastapi/importapi.py Outdated

Comment thread openlibrary/fastapi/importapi.py Outdated

Sanket17052006 force-pushed the migrate-import-preview-json branch 2 times, most recently from 387ad8a to 3b501b6 Compare June 2, 2026 06:43

Sanket17052006 and others added 7 commits June 2, 2026 16:33

migrate import_preview_json to FastAPI

08d3fa5

[pre-commit.ci] auto fixes from pre-commit.com hooks

655ef43

for more information, see https://pre-commit.ci

preserve the legacy behaviour

1f5f02a

Add web.ctx.site=site.get() and improve tests

de73557

Address review

8c78b40

[pre-commit.ci] auto fixes from pre-commit.com hooks

c2486f5

for more information, see https://pre-commit.ci

Fix ruff

f9173c2

Sanket17052006 force-pushed the migrate-import-preview-json branch from 3b501b6 to f9173c2 Compare June 2, 2026 11:07

create LibrarianDep

96134c2

RayBB approved these changes Jun 5, 2026

View reviewed changes

RayBB merged commit 9ca2164 into internetarchive:master Jun 5, 2026
4 checks passed

github-project-automation Bot moved this from Someone else is working on it to Done in Ray's Project Jun 5, 2026

mekarpeles mentioned this pull request Jun 23, 2026

docs(ai/imports): import pipeline architecture reference #13004

Merged

Sanket17052006 deleted the migrate-import-preview-json branch July 1, 2026 11:16

Navigation Menu

Uh oh!

Uh oh!

migrate import_preview_json to FastAPI - #12745

migrate import_preview_json to FastAPI#12745
RayBB merged 8 commits into
internetarchive:masterfrom
Sanket17052006:migrate-import-preview-json

Sanket17052006 commented May 15, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

RayBB left a comment

Uh oh!

Sanket17052006 commented May 28, 2026

Uh oh!

Sanket17052006 commented May 28, 2026

Uh oh!

RayBB left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

RayBB left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Uh oh!

Conversation

Sanket17052006 commented May 15, 2026

Technical

Testing

Screenshot

Stakeholders

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

RayBB left a comment

Choose a reason for hiding this comment

Uh oh!

Sanket17052006 commented May 28, 2026

Uh oh!

Sanket17052006 commented May 28, 2026

Uh oh!

RayBB left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

RayBB left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants