Data & ML — AI Agents, Skills & Tools
Agents, skills, guides, tools, and commands for data & ml — 78 curated resources for building with AI coding agents.
Analysis Reviewer
Use this agent to review a finished analysis for methodological errors before it ships — checking grain and double counting, join fan-out, rows silently dropped by filters and inner joins, sampling and truncation, null handling, time zone and date boundaries, numbers in the prose that disagree with the code's output, charts that mislead, and causal language resting on correlational evidence. Examples — 'review this notebook before I send the deck', 'the query and the summary disagree somewhere, find it', 'does this analysis actually support the conclusion it draws?'.
Data Engineer
Use this agent to build and maintain data pipelines — ingestion, ELT/ETL, warehouse modeling, orchestration, and data-quality tests. Examples — building an idempotent ingestion job, modeling a fact/dimension table in dbt, writing a safe backfill for a changed schema.
Data Scientist
Use this agent for data analysis — exploration, statistics, SQL, and clear findings. Examples — analyzing a dataset, writing an analytical SQL query, summarizing experiment results.
ML Engineer
Use this agent for production ML — pipelines, training, serving, evaluation, and MLOps. Examples — building a training pipeline, deploying a model, setting up evaluation.
Postgres Migration Engineer
Use this agent to plan and execute a zero-downtime Postgres schema migration — decomposing a breaking change into expand-contract steps, writing batched backfills, building indexes CONCURRENTLY, validating constraints online, and keeping every step reversible with the project's migration tooling. Examples — "add a NOT NULL column to a 200M-row table without downtime", "rename a column safely across a rolling deploy", "split this risky migration into reversible expand/contract steps".
Prompt Engineer
Use this agent to design and iterate the prompts behind an LLM-powered product feature — instructions, few-shot examples, tool schemas, and the evals that prove a change actually helped. Examples — "this classification prompt is flaky, make it reliable", "design the system prompt and function schema for our support agent", "our extraction prompt regressed after I tweaked it, set up evals so this stops happening".
Vector Search Engineer
Use this agent to design, build, and tune the vector-database layer of a search or RAG system — schema and index design (HNSW/IVF + quantization), metadata/payload filtering, hybrid (dense + sparse) search, and ingestion/upsert pipelines — sized to a real latency, recall, and cost budget. Examples — "set up pgvector for our docs with HNSW and filtered search", "our Qdrant queries are slow and recall dropped after quantization", "add metadata filtering so search only returns the current tenant's documents".
SQL Pro
Use this agent for SQL itself — correct joins and window functions, indexing, EXPLAIN plans, schema design, and safe migrations on Postgres/MySQL. Examples — making a slow query fast, designing a normalized schema, writing a reversible migration.
Analysis Memo Writer
Turn findings and figures you supply into a stakeholder writeup with a fixed spine: headline finding, what the number is and what it is not, method in three lines, a mandatory caveats and limitations section, what would change the conclusion, and one recommended next step. Never invents a figure and never upgrades a correlation into a cause. Use when the analysis is done and has to survive being read by someone who was not in the query.
Chart Chooser
Pick the chart for a stated question and data shape and defend the choice: the recommendation, the reasoning, the alternatives rejected and why each fails this question, encoding rules for axis baseline, sorting, color, and labels, and generated code in the plotting library you name. Use when you know what you want the chart to say and need the one form that says it, not a gallery.
Dataset First Look
Turn a pasted CSV header with sample rows, a pasted table, or an attached data file into a fixed plain-language profile: shape, column inventory with inferred types, null and blank patterns, cardinality, suspicious columns, duplicate-key risk, outliers worth a look, and the questions to settle before analyzing. Use when a dataset just landed and you need to know what you are holding before you write a query.
Spreadsheet Formula Auditor
Audit pasted spreadsheet formulas or a described model for the errors that survive review: hardcoded values buried inside formulas, ranges that drift or truncate, one cell in a row that does not match its neighbors, circular references, sign errors, IFERROR masking a real failure, and volatile functions, returned as a fix list ordered by how much money the error moves. Use before a model that carries a real decision leaves your hands.
SQL Explainer
Walk through a pasted SQL query in execution order in plain language: what each step does, the grain of the result, where a join can fan rows out, which filters silently drop rows, and a ranked list of what could be wrong. Comprehension only, not performance tuning. Use when you inherited a query, are reviewing one before trusting its numbers, or have to explain it to someone who does not read SQL.
Multimodal Document Extractor
Extract structured data from documents and images with a vision-language model — define the target schema, prompt the VLM to fill it from the page (invoices, forms, receipts, statements, IDs), and verify critical fields against the source. Use when you need reliable structured output from messy, varied, or scanned documents that defeat template-based OCR.
SQL Optimizer
Diagnose a slow SQL query from its execution plan and propose a verified optimization — finding the real bottleneck (sequential scan, missing or unused index, bad join order, app-side N+1) and measuring the fix before and after. Use when a query is slow and you need a fix backed by EXPLAIN ANALYZE, not a guess.
Embedding Index Tuner
Tune a vector index — HNSW graph parameters and quantization — to hit a recall target at the lowest latency and memory, by sweeping settings against a fixed query set instead of trusting defaults. Use when vector search is slow or memory-hungry, when recall dropped after enabling quantization, or when standing up an index and you need defensible parameters.
Postgres Index Strategist
Recommend the right Postgres index for a query or workload — choosing B-Tree vs. GIN vs. BRIN vs. partial/covering/expression, checking for redundant or unused indexes, and verifying the choice against the query plan. Use when a query needs an index, when deciding an index type for jsonb/array/full-text/time-series data, or when auditing an over-indexed table.
Safe Data Backfill Planner
Plan and implement a resumable production data backfill with bounded batches, checkpoints, rate limits, correctness verification, observability, and a pause or rollback path. Use when populating a new column, repairing historical rows, re-keying records, migrating derived data, or updating millions of records without overwhelming the database or replicas.
How to Check an AI Data Analysis Before You Trust It
Eight ways an AI-generated analysis goes wrong, the specific check that catches each one, and a copyable checklist to run before a number ships.
Claude Code for Data Analysts: Notebooks, SQL, and CSV Work Without a Data Engineer
Run analysis work in Claude Code: an analysis repo with CLAUDE.md, a CSV profiling pass, a pandas loop, a read-only warehouse query, and permission rules.
Anthropic's Data Plugin for Claude: Every Skill Explained
Every skill in Anthropic's open-source data plugin for Claude Cowork and Claude Code, the warehouse connectors it expects, the install commands, and its gaps.
Claude for Data Analysis: The Complete 2026 Guide
The four surfaces where Claude touches data, what the code execution sandbox really is, how an AI analysis fails silently, and the checks that catch it.
Claude for Excel: What It Does, Where It Fails, How to Use It
Install it from AppSource, learn which Excel builds are supported, use cell-level citations, and apply the review discipline before a workbook leaves your desk.
Claude Skills for Data Analysts: 5 to Upload Today
Five portable skills that make Claude behave like a careful analyst: first look, chart choice, memo writing, SQL explanation, and formula auditing.
Text-to-SQL with Claude: A Safe Read-Only Postgres Setup
Set up text-to-SQL with Claude on Postgres safely: a read-only role, an MCP server in read-only mode, schema and metric context, and a two-way check.
Which Claude Plan (and Model) Should a Data Analyst Pay For?
A decision guide by situation: occasional CSV work, daily Excel modeling, Claude Code against a warehouse, or a team sharing a Project. No prices.
The Best AI Tools for Data Analysts in 2026
The AI tools data analysts actually use in 2026: assistants that run code, spreadsheet add-ins, notebooks, text-to-SQL, and agents in the terminal.
The Best Document Parsers for RAG in 2026: LlamaParse vs Docling vs Unstructured vs Reducto
LlamaParse, Docling, Unstructured, Reducto, Marker and Mistral OCR compared on deployment, license, OCR, tables and output, with a verdict for each.
The Best Text-to-SQL Tools in 2026
Text-to-SQL tools compared on how they ground the model in your schema, what accuracy really means, read-only safety, deployment, licensing, and pricing model.
ChatGPT vs Claude for Data Analysis
ChatGPT vs Claude for data analysis, compared on sandbox behavior, file limits, Excel add-ins, chart output, citations, warehouse access, and verification.
Choosing Embeddings in 2026: OpenAI vs Cohere vs Voyage vs Open-Source
A decision guide for picking an embedding model for retrieval — accuracy, dimensions, cost, multilingual and domain fit, self-hosting, and lock-in.
How Embeddings Work: Vectors, Similarity, and Choosing a Model
What an embedding actually is, how similarity is measured, how the models are trained, and the practical rules for using embeddings well in search and RAG.
Best Vector Database in 2026: pgvector vs Pinecone vs Qdrant vs Weaviate vs Milvus vs Chroma vs LanceDB
A decision guide to vector databases — embedded, server, or managed; whether you already run Postgres; and which fits your scale, filtering, and RAG needs.
Postgres Connection Pooling: Sizing, Timeouts, and Serverless
Size Postgres connection pools across application instances, configure lifecycle timeouts, diagnose saturation, and choose PgBouncer or a managed pooler.
Indexing Postgres at Scale: B-Tree vs GIN vs BRIN and the Hidden Cost of Over-Indexing
A practical guide to choosing Postgres index types — B-Tree, GIN, BRIN, partial, and covering — and why every index you add taxes every write.
Vector Search at Scale: ANN Indexes, Quantization & Sharding
How to run vector search over millions to billions of vectors without blowing latency, memory, or cost — index families, quantization, filtering, and sharding.
Zero-Downtime Postgres Migrations: The Expand-Contract Playbook for 2026
How to change a live Postgres schema without downtime or broken deploys — the expand-contract pattern, safe column changes, batched backfills, and CONCURRENTLY.
8 Best Claude Skills for Database Work
Compare Claude skills for safe migrations, data backfills, indexes, query plans, pooling, deadlocks, vector search, and SQL tuning.
Using Vision-Language Models for OCR, Documents, and Video Understanding
How to use vision-language models for OCR, documents, and video: how they differ from traditional OCR, their failure modes, and getting reliable output.
Chroma
An open-source, Python-first vector database that runs in-process — the fastest path from pip install to a working retrieval prototype.
Claude For Excel
Anthropic's Excel add-in that explains, edits, debugs, and builds spreadsheet models in place, with cell-level citations, on paid Claude plans.
Databricks Genie
Databricks' conversational analytics layer: Genie Agents answer natural-language questions over Unity Catalog data using curated instructions and SQL.
Deepnote
A collaborative data notebook whose AI agent edits and runs your blocks, with data apps, a semantic layer, and integrations across warehouses and BI tools.
Docling
Open-source Python library that parses PDFs, DOCX, PPTX, HTML, and images into structured Markdown and JSON with layout, tables, and reading order for RAG.
Hex
A collaborative notebook and data-app platform whose AI agents write SQL and Python, answer questions in Threads, and run on curated workspace context.
Julius
A chat-first AI data analyst: upload a spreadsheet or connect a warehouse, ask in plain English, and get charts, code, and shareable exports back.
LanceDB
An open-source embedded vector database built on the Lance columnar format — serverless, multimodal, and designed to scale on local disk or object storage.
Marker
Open-source pipeline that converts PDFs, images, and Office docs into clean Markdown, JSON, or HTML fast, with optional LLM assist for tables and equations.
Mem0
A memory layer for AI agents and apps — persistent, personalized long-term memory across sessions.
Milvus
An open-source vector database built for billion-scale similarity search, with a distributed architecture and a wide menu of index types.
Pandasai
A Python library that adds a chat method to your dataframes: it generates and runs pandas code to answer questions, with an optional Docker sandbox.
pgroll
An open-source CLI for zero-downtime, reversible Postgres schema migrations using the expand-contract pattern behind versioned schema views.
pgvector
An open-source Postgres extension that adds a vector type and HNSW/IVFFlat indexes for similarity search inside your existing database.
Pinecone
A fully managed, serverless vector database for similarity search and RAG — no nodes to run, indexes to tune, or infrastructure to operate.
Qdrant
An open-source vector database written in Rust, built for low-latency similarity search at scale.
Qwen3-VL
Alibaba Qwen's open-weights vision-language model family (2B–235B, Apache-2.0): image and document understanding, OCR, visual reasoning, and video.
Reducto
High-accuracy document ingestion API — parsing, agentic OCR, table and figure extraction, and splitting that turns messy PDFs into LLM-ready data for RAG.
Thoughtspot Spotter
ThoughtSpot's agentic analyst: it resolves questions into search tokens against a governed semantic model rather than raw SQL, then acts on the answer.
Unstructured
Open-source library plus hosted Unstructured Pipelines and API that turn messy documents — PDF, HTML, docx, images, email — into clean, chunked JSON for RAG.
Vanna
An MIT-licensed Python framework for text-to-SQL: a user-aware agent that learns from successful queries and streams tables, charts, and summaries back.
Voyage AI
Embedding and reranking models tuned for retrieval, now part of MongoDB.
Weaviate
An open-source vector database with built-in hybrid search, pluggable vectorizer modules, and GraphQL/REST/gRPC APIs.
Define Metric
Write or refine a metric definition — name, plain-language meaning, grain, filters, source tables and columns, edge cases, and owner — into analysis/metrics/<slug>.md, after searching the repo for a definition that already exists.
First Look
Read the first rows and the true row count of a CSV or spreadsheet, profile it against a fixed checklist, and write the result to analysis/profiles/<name>.md so the dataset's condition is on record before anyone queries it.
DB Migrate
Generate and apply a database migration the safe way — using the project's migration tool, with expand-contract discipline for breaking changes, lock-free DDL, and a reversible up/down.
Scaffold a pgvector Schema & HNSW Index
Scaffold a production-ready pgvector schema and HNSW index for a corpus — matching the project's migration tooling, distance metric, and embedding dimensions.
Seed Data
Generate realistic, referentially-consistent seed data and a re-runnable seed script from your actual schema — types and constraints respected, plausible values, FK-dependency insert order, idempotent, never aimed at production.
Profile Postgres Queries
Profile a Postgres workload to find the queries actually costing you — rank by total time with pg_stat_statements, EXPLAIN the worst offenders, and recommend the highest-leverage fix.
AI Data Analyst
An AI data analyst is a tool that takes a data question, writes and runs the code or SQL to answer it, and returns a chart or summary you still have to check.
Attention Mechanism
Attention lets a model weigh how relevant every other token is to each token, building a context-aware representation as a weighted blend of their values.
Conversational Analytics
Conversational analytics is asking questions of governed business data in plain language and getting a chart or number back, without opening a dashboard.
Diffusion Model
A diffusion model generates an image by starting from random noise and removing it step by step, guided by a prompt, until a coherent picture remains.
Knowledge Cutoff
A knowledge cutoff is the date a model's training data ends, so it has no built-in knowledge of any event, release, or fact that came after it.
Semantic Layer
A semantic layer defines business metrics, dimensions, and joins once, so every query and every AI agent computes the same number the same way.
Test-Time Compute
Test-time compute is spending more computation at inference — longer reasoning, sampling, or search — to improve answers without retraining the model.
Text-to-SQL
Text-to-SQL is turning a plain-language question into a SQL query a database can run, using a model grounded in your schema, documentation, and past queries.
Tokenization
Tokenization splits text into tokens — the sub-word units a model reads and writes — and maps each to an integer ID the model processes.
Transformer
The neural-network architecture (Vaswani et al., 2017) that uses self-attention to process sequences in parallel — the basis of nearly all modern LLMs.