Nishat A

Extract usable text from scanned assessment PDFs so the content can be passed to an LLM for validation. script must be capable of extracting the user's filled in content. Below is the honest, unfiltered log of trial-and-error journey. Documenting every tool that was used to whack the files, every layout issue encountered.. (wip)

Starting Point

The assessment reports are scanned PDFs rather than digitally generated documents. Standard PDF text extraction returned little or no useful content.

Do or Do not - there is no uv try

Attempt 1: PyPDF

# uv add pypdf
# read_pdf("test_file.pdf") -- pass a pdf file to read


import pypdf

def read_pdf(file_path):
    # 1. Opening the PDF file
    reader = pypdf.PdfReader(file_path)

    # 2. Geting the total number of pages
    total_pages = len(reader.pages)
    print(f"Total Pages Found: {total_pages}\n" + "-" * 30)

    # 3. Looping through every page and extract text
    for page_number in range(total_pages):
        page = reader.pages[page_number]
        text = page.extract_text()

        print(f"--- Page {page_number + 1} ---")
        print(text)
        print("\n")

Result: ❌ Failed. Detects content, but not Layout aware. Doesn't detect the 'user filled content'.

Attempt 2: pymupdf4llm

# uv add pymupdf4llm
def read_pdf(file_path):
    # This sautomatically analyzes the layout and turns it into clean markdown text
    markdown_text = pymupdf4llm.to_markdown(file_path)
    print(markdown_text)

Result: ❌ Failed. Detects content, attempts to render as Markdown, but content extraction quality is worse than PyPDF. Not Layout aware. Doesn't detect the 'user filled content'.

Attempt 3: pdfplumber

# uv add pdfplumber
import pdfplumber
def read_pdf(file_path):
    with pdfplumber.open(file_path) as pdf:
        for page in pdf.pages:
            # Extracting visual tables flawlessly?
            tables = page.extract_tables()

            # Or extracting layout-sorted text
            text = page.extract_text(layout=True)
            print(text)

Result: ❌ Failed. Detects content, detects layout and renders as spaces. Doesn't detect the 'user filled content'.

Attempt 4: docling

# uv add docling
from docling.datamodel.base_models import InputFormat
from docling.document_converter import DocumentConverter

def read_pdf(file_path):
    converter = DocumentConverter()
    # Docling handles PDFs, images, and scanned form layouts natively
    result = converter.convert(file_path)

    # Exporting to Markdown which separates form sections clearly
    print(result.document.export_to_markdown())

Result: ❌ Failed !! Warning! shouldn't be used at all.

Detected the content significantly better than the other approaches tested and was able to partially identify user-filled content. However, it misinterpreted a significant amount of the content, which would result in an unreliable and potentially inaccurate assessment.

Attempt 5: PaddleOCR - read as IMAGE

from paddleocr import PaddleOCR

def read_image(file_path):
    # Need to enable angle classifier to handle slightly tilted phone photos/scans
    ocr = PaddleOCR(use_textline_orientation=True, lang="en")

    # PaddleOCR 3.x interface
    result = ocr.predict(file_path)

    # Loop through each page's results
    for page_res in result:
        # 1. Printing formatted text output to terminal
        page_res.print()

        # 2. Saving to a json structure to inspect it
        page_res.save_to_json("output_directory")
        page_res.save_to_img(save_path="./output_directory/")

# extract data from image
read_image("test_file.png")

Result: ⚠️ Partially successful

• Detected the content.
• Identified most of user-filled content - 80%
• Layout aware.
• Generates translated image, and json - so validation is easier.

⚠️ CAVEAT

• Doesn't detect checkbox data, and
• Doesn't detect text that is 'highlighted' by user.
• [Additional finding] - the output content quality doesn't change whether it is a pdf file or png file being read.

Reality check

Welcome to the reality of document automation, where things rarely work on the first try. Mission seemed simple on paper: extract usable text from scanned assessment PDFs so we can pass the content to a Large Language Model (LLM) for automated validation. This Project Prüfung needed to capture the actual handwritten or typed user-filled content inside the forms.

Anyway, this was a good hamster wheel run .. tbc..

Disclaimer

This post describes technical concepts and lessons I explored during a learning project. Examples and datasets have been recreated or anonymised for demonstration purposes and do not represent any production system, customer data, or confidential information.

If this looks suspiciously familiar, that is probably because document retrieval problems are remarkably universal. Any resemblance to a particular organisation's problems is, conveniently, not the subject of this post.

The purpose of this post is educational: to explore the practical challenges of OCR, retrieval-augmented generation, evidence grounding, and evaluation. It is not intended to describe or disclose the internal systems, processes, or information of any former employer or organisation.