Overview
The problem
Key features
- Automatic detection: labels every column with its privacy role, subtype, GDPR special-category flag and a confidence score. Macro-F1 is 0.774 with column names hidden, against the baseline's 0.401.
- Free-text PII finder: finds names, phone numbers, addresses and national IDs in chat messages with 0.999 recall, above the 0.95 the health scenario requires.
- Three-tier AI detector: rules and checksums first, then machine learning (kNN and logistic regression), then a local 27B-parameter LLM for the columns it is still unsure about.
- Scenario-aware protection: picks one privacy technique per column (keyed HMAC pseudonyms, generalisation, suppression, redaction) to fit the sharing scenario.
- Automatic tuning: searches up to 50 candidate plans to meet the k-anonymity, l-diversity and t-closeness targets while keeping the data useful.
- Release gate: reloads and re-checks every output file, and publishes only a release that passes.
- 7-step CI-style web UI: Input, Detect, Review, Select, Search, Transform and Evaluate, each with result tables and a human review step.
Technologies used
- Python, pandas, NumPy, scikit-learn: the pipeline stages, the ML router and parallel model fitting with joblib.
- AutoTrust-JEV-27B (4-bit MLX): a local LLM on Apple Silicon (about 18 GB of memory) that scores labelling decisions, so no data is sent to a cloud API.
- React 19, TypeScript, Vite: the web client.
- Tailwind CSS 4 and Ant Design 6: UI styling and components.
- Chrome on-device AI (Gemini Nano): optional in-browser help with uncertain columns.
- pets-eval: the organisers' toolkit for privacy and utility scorecards.
- Prettier, ESLint, Husky: code quality and pre-commit checks.
Challenges and learnings
Making detection 2x faster with identical results
Shipping a UI before the backend existed
Keeping the scores honest
Outcome
| Metric | Organisers' baseline | Our result | Improvement |
|---|---|---|---|
| Column detection, names hidden (macro-F1) | 0.401 | 0.774 | nearly 2x |
| Column detection, with names (macro-F1) | 0.646 | 0.854 | +32% |
| Free-text personal data (F2) | 0.184 | 0.998 | 5.4x |
| Detector run time, 9 datasets | 7.2s | 3.5s cold, 0.4s warm | 2x / 18x |