scholarr: deliver the integrated Sol xhigh product sprint #11

New issue

Open

opened 2026-10-03 00:12:19 +02:00 by justinzeus · 0 comments

justinzeus commented

2026-10-03 00:12:19 +02:00

Owner

Goal

Build the integrated Scholarr product from baseline 19c3305 (after owner-selected expiry-priority timing; previous baseline 44e7518) (preserving initial reset commit 0e22839) on docs/9-product-sprint, following SPRINT.md as the single scope/decision/execution reference. One product branch and one reviewed product PR; every later item inherits the preceding verified commit. Approval tracking: justinzeus/scholarr#9; baseline PR: justinzeus/scholarr#10.

Constraints

Preparation/readiness is not launch approval. Do not begin implementation until Justin approves the exact resolved sprint plan and SPRINT.md defaults. The owner go is a launch gate, not a fake completed dependency.
Go + Vue + SQLite; frozen design and source-agnostic identity requirements remain locked. Honor the existing detailed data-model contracts and their 22 acceptance cases.
ONLY openai-codex/gpt-6.1-sol, xhigh in all five sprint roles. No max, fallback, production deployment, main merge, publication, real-data import, paid provider use, or quota burn without useful work.
Every automatic check uses local HTTP/identity-provider fixtures. No provider credentials/private data in writer sandboxes. The separately authorized six-request host live check belongs to #9 after implementation proof, not to these sandbox items.
Commit one verifiable item at a time, preserving earlier proven work. ./tools/check.sh is the project baseline; create its first real Go gate in item 1 and add frontend checks when the frontend exists. Never weaken it to hide a regression.
Data-model case references below are authoritative behavior, not source-text assertions. Unsupported real-import/corpus claims, manual UI approval and the household shadow run remain owner-side gates in #9, not fabricated passes.

Acceptance

A runnable Go server validates one declarative config, opens persistent SQLite, and exposes honest health/readiness; the first real feature creates ./tools/check.sh and its failing-before/passing-after config/lifecycle check.
Numbered SQLite migrations create the documented table contracts atomically, verify backup failure refuses upgrade, and pass up/down, interrupted-upgrade, foreign-key and integrity cases (data model case 13).
First-admin bootstrap can create exactly one administrator and refuses an insecure or repeated setup attempt through the actual HTTP interface.
Internal password login/logout/recovery uses maintained hashing, expiring secure sessions, CSRF and bounded attempts; hostile requests cannot obtain a session.
Generic OIDC login verifies issuer/signature/state/nonce/PKCE and rejects hostile local identity-provider fixtures without disabling internal recovery.
Trusted-header authentication accepts only configured trusted proxies and rejects a directly spoofed identity header.
Following one strong source identity for two users creates one canonical author and two independent follows (case 1).
Same-name authors with different strong IDs remain separate, with no name-only automatic merge (case 2).
Exact normalized publication identifiers deduplicate globally; similar titles alone never merge works (case 8).
A publication reached through two followed authors has one per-user reading/favorite state and two origins (case 3), without adding an out-of-design favorite UI.
Another user's publication state stays private and independent through actual authorized API requests (case 4).
Unfollow hides the final visible origin without destroying reading state, and refollow restores it (case 5).
NEW is derived from first incremental discovery for 14 days, never from baseline/manual/legacy import, using a controlled clock.
The shared provider queue permits one outstanding request per provider, persists pacing/daily spend across restart, and counts retries/manual work against the same allowance.
Injected 429/auth/ban/503 responses produce the documented immediate circuit/cooldown transitions; restart cannot bypass Retry-After or reopen a stopped lane.
Provider egress rejects forbidden hosts, private-address targets and unsafe redirects, and never contacts Google; prove with a local hostile transport.
The real OpenAlex adapter parses/paginates sanctioned responses and reports missing/invalid key health without anonymous fallback, exercised against local HTTP fixtures.
The real Crossref adapter identifies the caller and follows documented response limit headers without treating undocumented intervals as fixed RPS, exercised locally.
The real arXiv adapter uses a validated sanctioned TLS endpoint and globally spaced single-connection requests, with parser/error fixtures.
The real Unpaywall adapter performs keyed-by-DOI lookup with the required contact identity and treats retired search as unavailable, with local fixtures.
The real ORCID adapter uses its registered public OAuth lane and refuses missing credentials or unconfirmed eligibility with actionable disabled health, using local fixtures.
Baseline sync is queued, additive, cursor-persisted and limited to one active run per user; a second request cannot create duplicate work.
Incremental sync resumes after interruption without duplicate publications or lost per-source provenance, proved through actual queued processing with local providers.
Sync cancellation and partial-provider failure preserve committed progress and expose accurate resumable state rather than invented completion.
Repeated unresolved evidence updates one review card; unchanged skipped evidence stays skipped and a new evidence version reopens it (case 10).
A reader's keep-separate choice cannot suppress another user's review or modify global identity evidence (case 19).
Administrator negative-identity evidence suppresses unchanged forbidden attachments; undo removes the tombstone and permits future review without automatic reattachment (cases 11 and 18).
Strong identifiers and explicit provider crosswalks attach safely while weak/conflicting candidates go to review; use public-safe calibration-shaped fixtures and report private-corpus validation as unperformed.
Merging authors without a co-follow preserves aliases, follows and origins, with exact undo (case 6a).
Merging co-followed authors coalesces follows/origins without duplicate keys and undo restores original public IDs/timestamps (case 6b).
A conflicting row-version change makes undo fail atomically with the named conflict and zero writes (case 7a).
A non-conflicting observation timestamp refresh survives successful undo (case 7b).
Publication merge preserves source records, identifiers, authorship evidence, origins and per-user state, with exact graph undo (case 9).
Nested merges support documented LIFO undo and reject an out-of-order undo with the blocking operation identified (case 15).
Normalization is idempotent, merge results are argument-order independent, and public-ID aliases resolve stably under property checks (case 14).
Concurrent ingests of one DOI produce one publication without surfacing a raw uniqueness error (case 17, including race-detector proof).
Admin garbage collection requires backup/dry-run/reference checks/audit; sanctioned strong-ID re-ingest may recreate a new row while purged legacy-only data stays absent (case 20).
Author search returns truthful provider/identity candidates and adding a selection confirms a follow through the limited queue, never a name-only identity shortcut.
Bulk author import produces a no-write preview, requires confirmation, and persists only validated selected records with an audit operation.
The offline legacy importer validates source fingerprint/schema and explicit user mapping, is idempotent, preserves read/favorite state and legacy origins, and makes zero network calls (cases 12a-e) using synthetic supported-format inputs.
Legacy Scholar shells resolving to one strong identity collapse through an auditable undoable author merge (case 21); Scholar metadata is parsed locally only.
An interrupted offline import resumes without duplicate rows or half-applied merges (case 22).
The full integrity battery names deliberately corrupted origin/author relationships and reports zero violations after supported domain/import operations (case 16).
User export preserves supported public identity/provenance/state and cannot expose another user's private records.
Legal OA resolution records location/license/provenance and exposes a human-clickable link without fetching PDFs or arbitrary publisher content.
SQLite backup and restore into a fresh data directory preserve a verified installation and refuse corrupt/incompatible archives.
The Vue dashboard uses the real API for the current user's counts and freshness, with the frozen populated/empty/loading/degraded states.
The publication list supports the frozen filters/read actions over real user-scoped API state, not production demo records.
Publication detail exposes real source/provenance/legal-OA metadata and persists the user's reading action.
The author list uses real follows and shows truthful identity/sync state with functional navigation and unfollow actions.
Author detail shows the canonical identity, related publications and sync controls backed by the real API.
The review screen presents actual evidence and permitted decisions, with conflict/error states rather than inert buttons.
The author search/add screen implements real preview/confirmation and actionable disabled-provider states.
The bulk import screen supports preview/selection/confirmation against the real import API and displays rejected rows safely.
The sources screen reflects actual configuration, quotas/cooldowns and disabled credentials without exposing secrets.
The activity screen presents actual audited operations and conflict-checked undo results.
Settings validates real user/admin configuration changes and documents restart-required transitions.
Login/onboarding screens implement the supported authentication and first-run flows against the real server without bypassing access controls.
Not-found behavior, responsive navigation, keyboard access and light/dark themes follow the frozen design across the implemented routes.
The built Go binary embeds and serves the built Vue assets locally, without design support.js, external fonts, private fixtures or placeholder product data.
A non-root container and documented self-contained binary installation start from a clean data directory and honor validated secret environment references.
The owner-approved MIT license is included without claiming an unauthorized public release.
README documents exact installation, auth/recovery, provider-disabled defaults, credentials/contact/terms, safe acquisition enablement and backup/restore.
An offline seeded end-to-end smoke through the real HTTP API proves onboarding, preview/confirmation, baseline sync, restart/resume, incremental sync, review/undo, reading and export across two isolated users.
The final gate passes Go race/parser-fuzz smoke, frontend type/build checks and the complete offline check script; report any unmet product criterion instead of declaring a scaffold finished.

Next action

Register the prepared Scholarr project, run native nightly readiness, and draft/approve the exact Sol xhigh plan after matching runner/plugin installation. Show the saved plan and preflight evidence to Justin; launch only on explicit go. During the night, start with the first unchecked item and keep every proof and commit in this issue/PR.

## Goal Build the integrated Scholarr product from baseline 19c3305 (after owner-selected expiry-priority timing; previous baseline 44e7518) (preserving initial reset commit 0e22839) on docs/9-product-sprint, following SPRINT.md as the single scope/decision/execution reference. One product branch and one reviewed product PR; every later item inherits the preceding verified commit. Approval tracking: justinzeus/scholarr#9; baseline PR: justinzeus/scholarr#10. ## Constraints - Preparation/readiness is not launch approval. Do not begin implementation until Justin approves the exact resolved sprint plan and SPRINT.md defaults. The owner go is a launch gate, not a fake completed dependency. - Go + Vue + SQLite; frozen design and source-agnostic identity requirements remain locked. Honor the existing detailed data-model contracts and their 22 acceptance cases. - ONLY openai-codex/gpt-6.1-sol, xhigh in all five sprint roles. No max, fallback, production deployment, main merge, publication, real-data import, paid provider use, or quota burn without useful work. - Every automatic check uses local HTTP/identity-provider fixtures. No provider credentials/private data in writer sandboxes. The separately authorized six-request host live check belongs to #9 after implementation proof, not to these sandbox items. - Commit one verifiable item at a time, preserving earlier proven work. ./tools/check.sh is the project baseline; create its first real Go gate in item 1 and add frontend checks when the frontend exists. Never weaken it to hide a regression. - Data-model case references below are authoritative behavior, not source-text assertions. Unsupported real-import/corpus claims, manual UI approval and the household shadow run remain owner-side gates in #9, not fabricated passes. ## Acceptance - [ ] A runnable Go server validates one declarative config, opens persistent SQLite, and exposes honest health/readiness; the first real feature creates ./tools/check.sh and its failing-before/passing-after config/lifecycle check. - [ ] Numbered SQLite migrations create the documented table contracts atomically, verify backup failure refuses upgrade, and pass up/down, interrupted-upgrade, foreign-key and integrity cases (data model case 13). - [ ] First-admin bootstrap can create exactly one administrator and refuses an insecure or repeated setup attempt through the actual HTTP interface. - [ ] Internal password login/logout/recovery uses maintained hashing, expiring secure sessions, CSRF and bounded attempts; hostile requests cannot obtain a session. - [ ] Generic OIDC login verifies issuer/signature/state/nonce/PKCE and rejects hostile local identity-provider fixtures without disabling internal recovery. - [ ] Trusted-header authentication accepts only configured trusted proxies and rejects a directly spoofed identity header. - [ ] Following one strong source identity for two users creates one canonical author and two independent follows (case 1). - [ ] Same-name authors with different strong IDs remain separate, with no name-only automatic merge (case 2). - [ ] Exact normalized publication identifiers deduplicate globally; similar titles alone never merge works (case 8). - [ ] A publication reached through two followed authors has one per-user reading/favorite state and two origins (case 3), without adding an out-of-design favorite UI. - [ ] Another user's publication state stays private and independent through actual authorized API requests (case 4). - [ ] Unfollow hides the final visible origin without destroying reading state, and refollow restores it (case 5). - [ ] NEW is derived from first incremental discovery for 14 days, never from baseline/manual/legacy import, using a controlled clock. - [ ] The shared provider queue permits one outstanding request per provider, persists pacing/daily spend across restart, and counts retries/manual work against the same allowance. - [ ] Injected 429/auth/ban/503 responses produce the documented immediate circuit/cooldown transitions; restart cannot bypass Retry-After or reopen a stopped lane. - [ ] Provider egress rejects forbidden hosts, private-address targets and unsafe redirects, and never contacts Google; prove with a local hostile transport. - [ ] The real OpenAlex adapter parses/paginates sanctioned responses and reports missing/invalid key health without anonymous fallback, exercised against local HTTP fixtures. - [ ] The real Crossref adapter identifies the caller and follows documented response limit headers without treating undocumented intervals as fixed RPS, exercised locally. - [ ] The real arXiv adapter uses a validated sanctioned TLS endpoint and globally spaced single-connection requests, with parser/error fixtures. - [ ] The real Unpaywall adapter performs keyed-by-DOI lookup with the required contact identity and treats retired search as unavailable, with local fixtures. - [ ] The real ORCID adapter uses its registered public OAuth lane and refuses missing credentials or unconfirmed eligibility with actionable disabled health, using local fixtures. - [ ] Baseline sync is queued, additive, cursor-persisted and limited to one active run per user; a second request cannot create duplicate work. - [ ] Incremental sync resumes after interruption without duplicate publications or lost per-source provenance, proved through actual queued processing with local providers. - [ ] Sync cancellation and partial-provider failure preserve committed progress and expose accurate resumable state rather than invented completion. - [ ] Repeated unresolved evidence updates one review card; unchanged skipped evidence stays skipped and a new evidence version reopens it (case 10). - [ ] A reader's keep-separate choice cannot suppress another user's review or modify global identity evidence (case 19). - [ ] Administrator negative-identity evidence suppresses unchanged forbidden attachments; undo removes the tombstone and permits future review without automatic reattachment (cases 11 and 18). - [ ] Strong identifiers and explicit provider crosswalks attach safely while weak/conflicting candidates go to review; use public-safe calibration-shaped fixtures and report private-corpus validation as unperformed. - [ ] Merging authors without a co-follow preserves aliases, follows and origins, with exact undo (case 6a). - [ ] Merging co-followed authors coalesces follows/origins without duplicate keys and undo restores original public IDs/timestamps (case 6b). - [ ] A conflicting row-version change makes undo fail atomically with the named conflict and zero writes (case 7a). - [ ] A non-conflicting observation timestamp refresh survives successful undo (case 7b). - [ ] Publication merge preserves source records, identifiers, authorship evidence, origins and per-user state, with exact graph undo (case 9). - [ ] Nested merges support documented LIFO undo and reject an out-of-order undo with the blocking operation identified (case 15). - [ ] Normalization is idempotent, merge results are argument-order independent, and public-ID aliases resolve stably under property checks (case 14). - [ ] Concurrent ingests of one DOI produce one publication without surfacing a raw uniqueness error (case 17, including race-detector proof). - [ ] Admin garbage collection requires backup/dry-run/reference checks/audit; sanctioned strong-ID re-ingest may recreate a new row while purged legacy-only data stays absent (case 20). - [ ] Author search returns truthful provider/identity candidates and adding a selection confirms a follow through the limited queue, never a name-only identity shortcut. - [ ] Bulk author import produces a no-write preview, requires confirmation, and persists only validated selected records with an audit operation. - [ ] The offline legacy importer validates source fingerprint/schema and explicit user mapping, is idempotent, preserves read/favorite state and legacy origins, and makes zero network calls (cases 12a-e) using synthetic supported-format inputs. - [ ] Legacy Scholar shells resolving to one strong identity collapse through an auditable undoable author merge (case 21); Scholar metadata is parsed locally only. - [ ] An interrupted offline import resumes without duplicate rows or half-applied merges (case 22). - [ ] The full integrity battery names deliberately corrupted origin/author relationships and reports zero violations after supported domain/import operations (case 16). - [ ] User export preserves supported public identity/provenance/state and cannot expose another user's private records. - [ ] Legal OA resolution records location/license/provenance and exposes a human-clickable link without fetching PDFs or arbitrary publisher content. - [ ] SQLite backup and restore into a fresh data directory preserve a verified installation and refuse corrupt/incompatible archives. - [ ] The Vue dashboard uses the real API for the current user's counts and freshness, with the frozen populated/empty/loading/degraded states. - [ ] The publication list supports the frozen filters/read actions over real user-scoped API state, not production demo records. - [ ] Publication detail exposes real source/provenance/legal-OA metadata and persists the user's reading action. - [ ] The author list uses real follows and shows truthful identity/sync state with functional navigation and unfollow actions. - [ ] Author detail shows the canonical identity, related publications and sync controls backed by the real API. - [ ] The review screen presents actual evidence and permitted decisions, with conflict/error states rather than inert buttons. - [ ] The author search/add screen implements real preview/confirmation and actionable disabled-provider states. - [ ] The bulk import screen supports preview/selection/confirmation against the real import API and displays rejected rows safely. - [ ] The sources screen reflects actual configuration, quotas/cooldowns and disabled credentials without exposing secrets. - [ ] The activity screen presents actual audited operations and conflict-checked undo results. - [ ] Settings validates real user/admin configuration changes and documents restart-required transitions. - [ ] Login/onboarding screens implement the supported authentication and first-run flows against the real server without bypassing access controls. - [ ] Not-found behavior, responsive navigation, keyboard access and light/dark themes follow the frozen design across the implemented routes. - [ ] The built Go binary embeds and serves the built Vue assets locally, without design support.js, external fonts, private fixtures or placeholder product data. - [ ] A non-root container and documented self-contained binary installation start from a clean data directory and honor validated secret environment references. - [ ] The owner-approved MIT license is included without claiming an unauthorized public release. - [ ] README documents exact installation, auth/recovery, provider-disabled defaults, credentials/contact/terms, safe acquisition enablement and backup/restore. - [ ] An offline seeded end-to-end smoke through the real HTTP API proves onboarding, preview/confirmation, baseline sync, restart/resume, incremental sync, review/undo, reading and export across two isolated users. - [ ] The final gate passes Go race/parser-fuzz smoke, frontend type/build checks and the complete offline check script; report any unmet product criterion instead of declaring a scaffold finished. ## Next action Register the prepared Scholarr project, run native nightly readiness, and draft/approve the exact Sol xhigh plan after matching runner/plugin installation. Show the saved plan and preflight evidence to Justin; launch only on explicit go. During the night, start with the first unchecked item and keep every proof and commit in this issue/PR.