- Batch candidates in groups of 50 (was 200) to avoid output token
truncation that caused all-zero scores
- Stream scored results via SSE as each batch completes instead of
waiting for all batches to finish
- Include solve_id in done event so validate works when results fit
in a single page
- Rewrite prompt to judge sentence coherence ("does it make sense")
rather than word predictability ("is it expected")
- Frontend inserts results in score-sorted order as they arrive
Co-Authored-By: Claude Opus 4.6 <[email protected]>
The new Wiktionary dictionary already includes 31k multi-word nouns,
making the combinatorial adjective+noun phase redundant. Add two_word
parameter (default False) to solve_word_dictionary to opt in when needed.
Co-Authored-By: Claude Opus 4.6 <[email protected]>
When known_start was set with capitalized casing, the unknown portion was
forced to all-lowercase, breaking width measurements for multi-word names
(e.g., "hn doe" instead of "hn Doe"). Added _case_unknown_portion helper
that respects word boundaries. Also fixed effective_left/right kerning
context to use correctly cased characters from the display string.
Co-Authored-By: Claude Opus 4.6 <[email protected]>
analyze_spot_redaction is now async and calls identify_boundary_text()
to use Claude Haiku for clean left/right text extraction instead of
center-point character filtering. Falls back to center-point if LLM
is unavailable.
Co-Authored-By: Claude Opus 4.6 <[email protected]>
Enrich POST /api/doc/{doc_id}/page/{page}/spot to call
analyze_spot_redaction when OCR data is cached, returning full
analysis JSON (segments, gap, font, line, offsets) alongside the
bounding box. This makes spot redactions first-class, matching the
analysis format returned by get_page_data for auto-detected redactions.
Co-Authored-By: Claude Opus 4.6 <[email protected]>
Add a synchronous function that runs the analysis mini-pipeline on a
single known bounding box using cached OCR data. This enables spot
redaction endpoints to get full analysis (font, left/right context text,
pixel alignment offsets) without needing LLM detection.
Co-Authored-By: Claude Opus 4.6 <[email protected]>
Replaced test_solve_endpoint_dictionary and test_dictionary_crud with
test_solve_endpoint_name_mode testing the new name mode.
Co-Authored-By: Claude Opus 4.6 <[email protected]>
Replaces uppercase_only bool with casing string parameter. Adds
known_start/known_end support for filtering and gap-width measurement.
Co-Authored-By: Claude Opus 4.6 <[email protected]>
Supports casing (lowercase/uppercase/capitalized), known_start/known_end
filtering with gap-width measurement of the unknown portion only.
Co-Authored-By: Claude Opus 4.6 <[email protected]>
Generate first x last name combinations from associate lists, apply
Title Case or UPPER casing, measure rendered widths, and return matches
within tolerance. Also checks associate name variants (initials,
nicknames) with deduplication.
Co-Authored-By: Claude Opus 4.6 <[email protected]>
The full-name solver collected all results into a Vec with no limit,
causing OOM crashes for realistic targets where millions of valid
first×last combinations exist. Thread max_results through
solve_subtree, dfs_first, and the Cartesian product loop so the
search stops early once enough results are found.
Co-Authored-By: Claude Opus 4.6 <[email protected]>
Wire together the per-page pipeline: OCR -> LLM redaction detection ->
guided OpenCV search -> font detection with masking -> pixel alignment.
Includes font caching per line and progress callbacks.
Co-Authored-By: Claude Opus 4.6 <[email protected]>
Adds a new detect_font_masked() function that masks redaction boxes to
white (255) before running pixel-based font detection on the full line
crop. This replaces the fragile approach of cropping just the left/right
portion of a line, which lost signal from narrow crops and coordinate
mismatches. The Dice-based scoring naturally ignores masked (white)
regions since they contain no ink pixels.
Co-Authored-By: Claude Opus 4.6 <[email protected]>
Add find_redaction_in_region() that searches for a black rectangle
within a specific page region identified by the LLM. Uses a lower
minimum area threshold (100px vs 500px) since the LLM provides
spatial guidance, preventing merged redactions and false positives.
Co-Authored-By: Claude Opus 4.6 <[email protected]>
Introduce llm_detect.py that sends OCR lines to Claude Haiku to identify
redaction locations via broken text patterns (garbled chars, brackets,
pipes). The LLM returns boundary words which are mapped back to pixel
positions using OCR character data. Includes comprehensive tests covering
single/multiple redactions, line start/end edge cases, and error handling.
Co-Authored-By: Claude Opus 4.6 <[email protected]>
Replace OCR-based offset calculation with pixel alignment that slides
rendered text across the page image to find the exact position with
maximum Dice overlap. Also fix char filtering to use center-point
(c.x + c.w/2) instead of right-edge, recovering characters like "nt"
in "Sent" that partially overlap the redaction box.
Co-Authored-By: Claude Opus 4.6 <[email protected]>
Replace word-width MAE scoring with pixel-overlap Dice scoring in both
_full_search() and _fine_search(). Both functions now accept a line_crop
parameter and use _score_font_line_pixel() instead of _score_font_line().
Comparisons are flipped (higher is better). The size range is now derived
from the OCR line height instead of the fixed SIZE_RANGE constant, and
uses step=1 since pixel scoring has sharp peaks at the exact font size.
Co-Authored-By: Claude Opus 4.6 <[email protected]>
Add _score_font_line_pixel() which renders candidate text and compares
pixels against the actual page image using Dice coefficient overlap.
This captures glyph shape differences that the old word-width MAE
approach cannot distinguish (e.g. serif vs sans-serif).
Co-Authored-By: Claude Opus 4.6 <[email protected]>
When a user clicks a redaction, this endpoint crops the surrounding line,
runs Tesseract OCR, detects the font, and returns text segments (before/after
the redaction), gap width, font info, and character-level bounding boxes.
Co-Authored-By: Claude Opus 4.6 <[email protected]>
Replace upfront OCR + font detection + overlay rendering with lightweight
OpenCV-based redaction detection. Upload now just rasterizes and detects
black rectangles, making it significantly faster. Remove the overlay
endpoint and update the data endpoint to return redaction bboxes.
Co-Authored-By: Claude Opus 4.6 <[email protected]>
Victim names are stored as a flat set of lowercased name variants
with no person details — the V badge appears but is not clickable,
protecting victim identities. A result can show both T1/T2/T3 and V
badges simultaneously.
- 15 known victims from public court records → 43 name variants
- Enabler/perpetrator categories promoted to T2
- Purple V badge with pointer-events disabled
- Victim matching uses prefix/suffix combination like associate matching
Co-Authored-By: Claude Opus 4.6 <[email protected]>
Lazy-loads associates.json from data dir and serves it at /api/associates,
with fallback to empty dicts if file is missing.
Co-Authored-By: Claude Opus 4.6 <[email protected]>
Measures a known word's width in font context, then verifies the solver
recovers that word from its width. Proves the width table and solver
work correctly end-to-end.
Co-Authored-By: Claude Opus 4.6 <[email protected]>
Add POST /api/solve endpoint that streams results via Server-Sent Events,
supporting both enumerate (brute-force) and dictionary solve modes. Add
dictionary CRUD endpoints (POST/GET/DELETE /api/dictionary) and solve
cancellation (DELETE /api/solve/{id}). Move static files mount to end of
app.py so API routes take priority.
Co-Authored-By: Claude Opus 4.6 <[email protected]>
Add solve_dictionary() for linear-scan matching of dictionary entries
against a target pixel width, and DictionaryStore for managing named
word/name lists. Reuses SolveResult from the solver module.
Co-Authored-By: Claude Opus 4.6 <[email protected]>
Distribute branch-and-bound search across CPU cores by generating
depth-2 prefixes and farming each subtree to a worker process.
Includes on_progress callback and automatic prefix depth selection.
Co-Authored-By: Claude Opus 4.6 <[email protected]>
Replace static overlay with client-side canvas rendering. Left panel
shows clickable OCR lines with font controls (dropdown + size slider).
Right panel renders green text overlay on the document via canvas.
Fonts loaded from backend TTF files using the FontFace API.
New endpoints: GET /api/fonts, GET /api/font/{id}
Co-Authored-By: Claude Opus 4.6 <[email protected]>
Instead of detecting one font per page, detect the font for each line
independently. The previous line's font is used as a prior — if it
scores within 15% of the best candidate, it's kept for consistency.
This handles documents with mixed fonts (bold headers, regular body,
different sections).
Also: redirect / to the static UI, fix data endpoint to return
per-line font info.
Co-Authored-By: Claude Opus 4.6 <[email protected]>
- Render each word at its OCR'd position instead of whole lines,
so word spacing matches the original document exactly
- Add y-offset correction using glyph bounding box
- Add fine-grained font size search (±3 around coarse best, step 1)
- Add alignment quality test (48.9% hit rate, >30% threshold)
- Add scipy to dev dependencies
Co-Authored-By: Claude Opus 4.6 <[email protected]>
Adds POST /api/upload (runs full pipeline: rasterize, OCR, font detect,
overlay), GET endpoints for original/overlay images and OCR data JSON.
Also adds python-multipart, anyio, and pytest-anyio dependencies.
Co-Authored-By: Claude Opus 4.6 <[email protected]>
