Unstructured
Open-source library plus hosted Unstructured Pipelines and API that turn messy documents — PDF, HTML, docx, images, email — into clean, chunked JSON for RAG.
Unstructured preprocesses messy documents — PDF, HTML, docx, images, email — into clean, chunked JSON for RAG. It ships as an Apache-2.0 Python library and the hosted Unstructured Pipelines platform and API, with source and destination connectors, partitioning files into typed elements, then enriching, chunking, and embedding them as an ingestion ETL layer.
Unstructured is a document preprocessing and ETL layer for LLMs. Real-world source data — PDFs, HTML, Word and PowerPoint files, spreadsheets, scanned images, and email — is messy and inconsistent, and feeding it raw into a RAG pipeline produces poor retrieval. Unstructured partitions each file into typed elements (Title, NarrativeText, Table, ListItem, and more), applies OCR and layout detection where needed, and emits clean, structured JSON that downstream chunking and embedding can rely on.
It comes in two forms. The open-source library (unstructured, Apache-2.0) runs locally and handles partitioning across dozens of file types through a single partition entry point. The hosted platform, now called Unstructured Pipelines, and the Unstructured API layer on source and destination connectors, enrichment, chunking, and embedding, so teams can build continuous ingestion pipelines from data sources into vector databases and warehouses without operating the infrastructure.
Highlights
- Many file types — one pipeline for PDF (digital and scanned), HTML, DOCX, PPTX, XLSX, CSV, images, EML/MSG, EPUB, Markdown, and more.
- OSS + Pipelines/API — Apache-2.0 library for local use, plus the managed Unstructured Pipelines platform and API with connectors and workflows.
- RAG-ready output — partition into typed elements, then chunk (e.g. by title) and serialize to JSON tuned for retrieval.
- Connectors — source and destination integrations for vector stores and data platforms like Pinecone, Weaviate, Databricks, Snowflake, and MongoDB.
Good to know
Unstructured is freemium: the core library is free and self-hostable under Apache-2.0, while the hosted Unstructured Pipelines platform and API are usage-based with a free tier and enterprise (dedicated/VPC) options. It sits at the start of a RAG stack — see how RAG works and best RAG frameworks 2026 for where ingestion fits, and pair it with a chunking strategy optimizer once documents are parsed.
Frequently asked questions
- What is Unstructured?
- Unstructured is a document preprocessing and ETL layer for LLMs. It partitions messy files — PDFs (digital and scanned), HTML, DOCX, PPTX, XLSX, images, EML/MSG email, and dozens of other formats — into clean, typed elements like Title, NarrativeText, and Table, then chunks and serializes them into JSON ready for RAG and other LLM pipelines.
- Is Unstructured free?
- It's freemium. The core library is open source under Apache-2.0 and free to self-host. The hosted Unstructured Pipelines platform and API add connectors, enrichment, embedding, and managed pipelines on a usage-based model with a free tier, plus paid usage and enterprise (dedicated/VPC) options.
- How do I use Unstructured?
- Install the open-source `unstructured` Python package and call `partition` to convert a file into elements locally, or use the hosted Unstructured Pipelines platform or API to wire source connectors, processing workflows, and destination connectors (vector stores, warehouses) without managing infrastructure yourself.
Filed under
rag · document-processing · etl · preprocessing · ocr
Related
- How RAG Actually Works: Ingestion, Chunking, Retrieval & RerankingA clear, practical walkthrough of the retrieval-augmented generation pipeline — what each stage does, where it fails, and how the pieces fit together.
- Multimodal RAG over PDFs, Scans & Charts: Two Approaches That Actually WorkRAG over visual documents — PDFs, scans, charts — where text-only extraction loses tables and layout. Parse-then-text vs embed-the-page-image, with trade-offs.
- Best RAG Frameworks in 2026The RAG frameworks worth building on in 2026 — LlamaIndex, LangChain, LangGraph, DSPy, Dify and Mastra — plus the retrieval layer that decides answer quality.
- Chunking Strategy OptimizerFind the chunking strategy and size that maximizes retrieval quality for a specific corpus, by sweeping configurations against a fixed eval set instead of guessing. Use when RAG answers miss obvious content, when standing up a new corpus, or when picking chunk size/overlap.
- Multimodal Document ExtractorExtract structured data from documents and images with a vision-language model — define the target schema, prompt the VLM to fill it from the page (invoices, forms, receipts, statements, IDs), and verify critical fields against the source. Use when you need reliable structured output from messy, varied, or scanned documents that defeat template-based OCR.
- The Best Document Parsers for RAG in 2026: LlamaParse vs Docling vs Unstructured vs ReductoLlamaParse, Docling, Unstructured, Reducto, Marker and Mistral OCR compared on deployment, license, OCR, tables and output, with a verdict for each.