OpenSource Tesseract + Spacy NLP Pipeline

Solution for the use case Document Processing & Information Extraction Pipeline.

AI tools

Tesseract OCR, SpaCy NLP, PyPDF2, Pandas, scikit-learn, FastAPI

Language

Python

Accuracy

88%

Published

2025-07-05

Code (Python)

<pre><code class="language-python">import pytesseract
import spacy
import pandas as pd
from PIL import Image
import PyPDF2
import cv 
import numpy as np
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.ensemble import RandomForestClassifier
import joblib

class DocumentProcessor:
    def __init__(self):
        # Load NLP model
        self.nlp = spacy.load("en_core_web_sm")
        
        # Load pre-trained document classifier
        self.doc_classifier = joblib.load("models/document_classifier.pkl")
        self.tfidf_vectorizer = joblib.load("models/tfidf_vectorizer.pkl")
        
        # Configure Tesseract
        pytesseract.pytesseract.tesseract_cmd = r"/usr/bin/tesseract"
    
    def process_document(self, file_path):
        """Main document processing pipeline"""
        # Step 1: Extract text based on file type
        if file_path.lower().endswith(".pdf"):
            text_content = self.extract_pdf_text(file_path)
            images = self.pdf_to_images(file_path)
        else:
            text_content = self.extract_image_text(file_path)
            images = [Image.open(file_path)]
        
        # Step 2: Preprocess and clean text
        cleaned_text = self.preprocess_text(text_content)
        
        # Step 3: Classify document type
        doc_type = self.classify_document(cleaned_text)
        
        # Step 4: Extract entities based on document type
        entities = self.extract_entities(cleaned_text, doc_type)
        
        # Step 5: Extract tables if present
        tables = self.extract_tables(images)
        
        # Step 6: Validate extracted data
        validation_results = self.validate_extraction(entities, doc_type)
        
        return {
            "document_type": doc_type,
            "extracted_text": cleaned_text,
            "entities": entities,
            "tables": tables,
            "validation": validation_results,
            "confidence": validation_results["overall_confidence"]
        }
    
    def extract_pdf_text(self, pdf_path):
        """Extract text from PDF using PyPDF2 and OCR fallback"""
        text = ""
        try:
            # First try direct text extraction
            with open(pdf_path, "rb") as file:
                pdf_reader = PyPDF2.PdfReader(file)
                for page in pdf_reader.pages:
                    text += page.extract_text()
            
            # If no text extracted, use OCR
            if len(text.strip()) &lt; 50:
                text = self.ocr_pdf_pages(pdf_path)
                
        except Exception as e:
            # Fallback to OCR
            text = self.ocr_pdf_pages(pdf_path)
            
        return text
    
    def extract_image_text(self, image_path):
        """Extract text from image using Tesseract OCR"""
        # Load and preprocess image
        image = cv2.imread(image_path)
        gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
        
        # Apply image preprocessing for better OCR
        gray = cv2.medianBlur(gray, 3)
        
        # Use adaptive threshold
        thresh = cv2.adaptiveThreshold(
            gray, 255, cv2.ADAPTIVE_THRESH_GAUSSIAN_C, 
            cv2.THRESH_BINARY, 11, 2
        )
        
        # OCR with custom configuration
        custom_config = r"--oem 3 --psm 6"
        text = pytesseract.image_to_string(thresh, config=custom_config)
        return text

# API endpoint setup
from fastapi import FastAPI, UploadFile, File
import os

app = FastAPI(title="Document Processing API")
processor = DocumentProcessor()

@app.post("/process-document")
async def process_document_endpoint(file: UploadFile = File(...)):
    """Process uploaded document and return extracted information"""
    # Save uploaded file temporarily
    file_path = f"temp/{file.filename}"
    with open(file_path, "wb") as buffer:
        content = await file.read()
        buffer.write(content)
   
    # Process document
    try:
        result = processor.process_docum
…

Prompts

<p><strong>DOCUMENT_CLASSIFICATION_PROMPT:</strong></p><pre><code class="language-plaintext">Classify this document based on its content and structure:

Text Content: {extracted_text}
Layout Features: {layout_features}
File Metadata: {file_metadata}

Document Types to Consider:
- Invoice: Contains vendor info, amounts, line items, payment terms
- Contract: Has parties, dates, obligations, signature blocks  
- Resume: Includes personal info, experience, education, skills
- Report: Contains analysis, charts, conclusions, recommendations
- Form: Has fields, checkboxes, structured data entry

Classification Criteria:
1. Keyword patterns and terminology
2. Document structure and layout
3. Presence of specific entities (dates, amounts, names)
4. Format conventions and templates

Return classification:
{
  "document_type": "invoice|contract|resume|report|form",
  "confidence": 0.92,
  "key_indicators": ["amount_present", "vendor_address", "line_items"],
  "suggested_fields": ["vendor_name", "total_amount", "invoice_date"]
}
</code></pre><p><strong>ENTITY_EXTRACTION_PROMPT:</strong></p><pre><code class="language-plaintext">Extract structured information from this document:

Document Type: {document_type}
Text: {text_content}
Language: {detected_language}

Extract relevant entities based on document type:

For Invoices:
- Vendor information (name, address, contact)
- Customer details
- Financial data (amounts, taxes, totals)
- Dates (invoice date, due date)
- Line items with descriptions and prices

For Contracts:
- Contracting parties
- Contract dates (effective, expiration)
- Financial terms and amounts
- Key obligations and deliverables
- Signature information

Output structured JSON with confidence scores for each field.
</code></pre>