Python · ML · Tabular Data · Verified 2025
Skrub Library — Practitioner Guide
Skrub handles the preprocessing step that eats most of your modeling time — mixed column types, high-cardinality strings, datetimes, fuzzy table joins, multi-table pipelines. Here’s what it actually does, what changed recently, and where it still needs your judgment.
pip install skrub. Optional: pip install skrub[transformers] for TextEncoder (HuggingFace-backed).Quick start — from raw dataframe to cross-validated score
The fastest path from a messy dataframe to a working model baseline. tabular_pipeline() wraps a TableVectorizer (which handles column detection and encoding) and a HistGradientBoostingRegressor or HistGradientBoostingClassifier (which handles nulls natively, so no imputer needed).
tabular_learner() is deprecated as of recent releases — use tabular_pipeline() instead. The old function still works but will be removed. Any code or tutorial you find that references tabular_learner needs updating.
from skrub import tabular_pipeline from skrub.datasets import fetch_employee_salaries from sklearn.model_selection import cross_val_score dataset = fetch_employee_salaries() df, y = dataset.X, dataset.y # One line baseline: auto-encodes job titles, division names, # date columns, and numeric features without any manual setup model = tabular_pipeline('regressor') scores = cross_val_score(model, df, y, cv=5) print(f"R²: {scores.mean():.3f} ± {scores.std():.3f}") # R²: 0.912 ± 0.013
To explore the dataframe before fitting, use TableReport. In a Jupyter notebook, skrub.patch_display() replaces the default pandas/polars repr with a TableReport automatically, so every dataframe cell renders interactively:
import skrub from skrub import TableReport # Option A: explicit report report = TableReport(df) report # renders in Jupyter: distributions, correlations, missing values # Option B: patch all dataframes in the notebook skrub.patch_display() df # now renders as TableReport automatically # Statistical dependencies between columns from skrub import column_associations column_associations(df) # pairwise statistical dependence matrix
Core components
TableReport
Interactive HTML summary of any dataframe — distributions, correlations, missing value patterns, high-cardinality flags, sorted column detection. Supports pandas, polars, and numpy arrays. Use patch_display() to make it the default in Jupyter.
TableVectorizer
Detects column types and routes each to the right encoder: StringEncoder for high cardinality (≥40 unique values), OneHotEncoder for low cardinality, DatetimeEncoder for datetimes, passthrough for numerics. Produces a consistent float32 output matrix.
tabular_pipeline()
One-line scikit-learn Pipeline combining a TableVectorizer and a HistGradientBoosting estimator. Replaces the deprecated tabular_learner(). Pass 'regressor', 'classifier', or any sklearn estimator as the argument.
Cleaner
Lightweight pre-processor that normalizes column dtypes, handles null strings, and casts to consistent numeric types. Used internally by TableVectorizer but available standalone. The old SimpleCleaner was renamed to Cleaner.
Joiner & AggJoiner
Joiner merges tables on fuzzy string keys using approximate matching. AggJoiner aggregates a detail-level table (e.g., transactions) and joins the result onto the main table. Both are sklearn-compatible transformers.
InterpolationJoiner
Joins on non-exact spatial or temporal keys using a learned interpolation model. Useful for enriching datasets with nearby weather readings, geospatial lookups, or any numeric key where exact matching isn’t possible.
Encoders for strings — which one to use
The choice of string encoder significantly affects both performance and memory footprint. Skrub provides four options with different tradeoffs, and TableVectorizer picks automatically — but understanding when to override the default is worth the few seconds it takes.
| Encoder | How it works | Best for |
|---|---|---|
| StringEncoder | TF-IDF on character n-grams, reduced with truncated SVD. Normalized output. Default for high-cardinality columns since 0.6.0 (replaced GapEncoder). | Most string columns. Fast, memory-efficient, solid baseline. Use this unless you have a specific reason to switch. |
| TextEncoder | Dense embeddings from a HuggingFace sentence-transformer model (local or Hub). Requires pip install skrub[transformers]. | Free-text columns with diverse, sentence-like entries — product descriptions, customer feedback, notes. Semantically-aware but slower. |
| GapEncoder | Soft topic model on character n-grams — each component corresponds to an interpretable pattern. Older default, still available. | When feature interpretability matters more than speed. Slower and more memory-intensive than StringEncoder. |
| MinHashEncoder | Fast approximate n-gram similarity via MinHash. Very low memory footprint. | Very large datasets where speed and memory dominate. Slightly lower quality than StringEncoder. |
from skrub import TableVectorizer, TextEncoder, SquashingScaler from skrub import DatetimeEncoder # Swap in TextEncoder for free-text columns, # SquashingScaler (clips outliers, more robust than StandardScaler), # and circular encoding for datetime seasonality table_vec = TableVectorizer( high_cardinality=TextEncoder(), # LLM embeddings for text numeric=SquashingScaler(), # robust numeric scaling datetime=DatetimeEncoder( periodic_encoding="circular" # sin/cos for seasonality ), )
SquashingScaler clips extreme values and scales, making it more robust than StandardScaler when numeric columns have outliers. It’s the default numeric encoder in tabular_pipeline() when using linear models.
DataOps — multi-table pipelines with hyperparameter tuning
DataOps is the significant architectural addition in recent skrub releases. Where tabular_pipeline() handles a single table, DataOps handles the realistic case: multiple related tables, transformations applied to different subsets of columns, choices between different pipeline configurations, and the need to export a trained pipeline for deployment — all within a framework that tracks every operation for reproducibility.
The core idea is that you declare operations on placeholder variables using skrub.var(). These operations are recorded but not immediately executed. Once you’ve described the full pipeline, you export it as a SkrubLearner — a standalone, picklable object that accepts a dictionary of inputs (your tables) and applies the recorded transformations in the correct order.
import skrub from skrub import TableVectorizer from sklearn.linear_model import Ridge # 1. Declare placeholder variables — operations are recorded, not yet run orders = skrub.var("orders", orders_df) # use preview data for type inference ratings = skrub.var("ratings", ratings_df) # 2. Describe transformations using normal Python agg_ratings = ( ratings .groupby("item_id")["score"] .agg(skrub.choose_from(["mean", "median"], name="agg_func")) # tunable! .reset_index() ) # 3. Join and vectorize merged = orders.merge(agg_ratings, on="item_id", how="left") X = merged.skb.apply(TableVectorizer()).skb.mark_as_X() y = merged["revenue"].skb.mark_as_y() # 4. Add model as a choice (enable tuning between estimators) pred = X.skb.apply( skrub.choose_from( [Ridge(alpha=1.0), Ridge(alpha=0.1)], name="model" ) ) # 5. Tune over all choices at once search = pred.skb.make_randomized_search(fitted=True) search.results_ # DataFrame: mean_test_score, agg_func, model, ... # 6. Export the best pipeline as a deployable SkrubLearner learner = pred.skb.make_learner(fitted=True) learner.predict({"orders": new_orders_df, "ratings": new_ratings_df})
The choices defined with skrub.choose_from() aren’t limited to scikit-learn hyperparameters. You can make any argument to any operation a tunable choice — aggregation functions, which columns to include, whether to apply a specific transformer — and the search explores all combinations in a single call.
SkrubLearner is a picklable, sklearn-compatible object that accepts a dictionary of tables rather than a single design matrix. This means the entire multi-table pipeline — joins, aggregations, encoding, model — is bundled into one artifact. You can call .fit(env_dict), .predict(env_dict), and .transform(env_dict) where env_dict maps table names to dataframes. It can also be fitted at export time by passing fitted=True to make_learner().
Selectors — precise column targeting
Skrub’s selectors module lets you define column groups based on properties rather than hardcoded names. Combined with DataOps, this handles the common pattern of applying different transformers to different column subsets without duplicating logic.
from skrub import selectors as s import skrub # Build selectors using set-like operations high_card = s.string() - s.cardinality_below(40) # high-cardinality strings has_nulls = s.has_nulls() # columns with any missing values leftover = s.all() - high_card - has_nulls # everything else df_var = skrub.var("df", df) # Apply different transformers based on column properties encoded_strings = df_var.skb.select(cols=high_card).skb.apply( skrub.StringEncoder(n_components=30) ) rest = df_var.skb.select(cols=leftover).skb.apply( skrub.TableVectorizer() ) result = rest.skb.concat([encoded_strings], axis=1) # Available selectors # s.string(), s.numeric(), s.has_nulls(), s.all() # s.cardinality_below(n), s.has_dtype(dtype), s.glob("prefix_*")
Fuzzy joining — when keys don’t match exactly
Joiner handles the realistic case where two tables should be merged on a string key but the strings don’t match exactly — department names with different abbreviations, city names with typos, product codes with varying formatting. It uses approximate string matching rather than exact equality, and is a full sklearn transformer so it fits on training data and transforms consistently at inference.
from skrub import Joiner # main_table has "dept_name"; aux_table has "department" # The values are similar but not identical ("IT Dept" vs "IT Department") joiner = Joiner( aux_table=budget_df, main_key="dept_name", aux_key="department", ) enriched = joiner.fit_transform(main_df) # budget_df columns now appended to main_df rows
from skrub import AggJoiner # transactions_df has one row per transaction. # AggJoiner aggregates to one row per customer, then joins onto customers_df. agg_joiner = AggJoiner( aux_table=transactions_df, main_key="customer_id", aux_key="customer_id", operations=["sum", "mean", "max"], ) customers_enriched = agg_joiner.fit_transform(customers_df) # Adds columns: amount_sum, amount_mean, amount_max, etc.
Skrub’s fuzzy matching uses its own approximate string similarity — no external dependency like fuzzywuzzy is needed. The Joiner has a threshold parameter to control how loose the matching is. Start with the default and inspect the skrub_Joiner_score column added to the output — low scores indicate uncertain matches that may need manual review.
Skrub vs other tools
| Tool | What it does well | Where it doesn’t help |
|---|---|---|
| pandas / polars | General-purpose dataframe manipulation, large ecosystem, performance at scale | No ML preprocessing automation, no fuzzy joins, no sklearn compatibility layer |
| scikit-learn only | Strong model variety, mature Pipeline API | ColumnTransformer is verbose for mixed-type tables; no fuzzy join, no multi-table support, no auto type detection |
| skrub | Auto-encodes mixed column types, fuzzy table joins, multi-table DataOps with tunable choices, SkrubLearner for deployment | Large tables may need sampling for TableReport; fuzzy join quality depends on key similarity; DataOps adds learning curve over sklearn pipelines |
Skrub is additive rather than competitive — it wraps scikit-learn rather than replacing it. The primary audience is teams who spend meaningful time writing column-type detection, high-cardinality encoding, and table-joining boilerplate for each new project.
FAQ – Skrub Library
tabular_learner vs tabular_pipeline — which should I use?
tabular_pipeline() — tabular_learner() is deprecated and will be removed in a future release. The function signatures and behavior are the same; it’s a rename. Update any existing code by replacing tabular_learner with tabular_pipeline.When should I use DataOps instead of tabular_pipeline?
tabular_pipeline() when you have a single flat table and want a quick baseline. Move to DataOps when you need any of: multiple input tables, column-specific transformers beyond TableVectorizer’s defaults, hyperparameter choices across the entire pipeline (not just sklearn params), or a deployable SkrubLearner that accepts a dictionary of tables rather than a single matrix.Does skrub require fuzzywuzzy or any external string matching library?
fuzzywuzzy, rapidfuzz, or other external library is needed as a dependency. The only optional dependency is transformers (via pip install skrub[transformers]) if you want to use TextEncoder with HuggingFace models.What changed between GapEncoder and StringEncoder as the default?
StringEncoder became the default high-cardinality encoder in skrub 0.6.0, replacing GapEncoder. StringEncoder is faster, uses less memory, and produces comparable or better downstream accuracy for most tasks. GapEncoder is still available and remains useful when you need interpretable features — its components correspond to character n-gram patterns that can be read and understood. For most production pipelines where you care about performance, stick with StringEncoder.Does skrub work with polars, or only pandas?
Can I use skrub with PyTorch or deep learning models?
Resources
Accuracy note: API names, default encoder changes, and deprecations are sourced from the skrub release history and official documentation at skrub-data.org. Python 3.10+ and scikit-learn 1.4.2+ requirements are from the release notes. UIG Data Lab is independent and not affiliated with the skrub project or its contributors.
