bardsai/eu-pii-anonimization-multilang-v3

EU PII Anonimization Multilingual Detector v3

A multilingual PII detector for the 24 official EU languages, built to work on the text real systems see: web pages and raw scrapes, mailing lists, invoices, PDF tables and OCR output. It is the successor of v2 and v1.

Try the live demo: it runs the INT8 model entirely in your browser, so your text never leaves your device.

• Model: ModernBERT token classifier, fine-tuned from jhu-clsp/mmBERT-base, 307M parameters, up to 8192-token context.
• Labels: 37 entity types in BIO format (listed below). New in v3: DATE_TIME (dates and times that are not a date of birth) and URL (any web address).

Results

PIIMB leaderboard

PIIMB scores character-level, label-agnostic masking F2 on six test sets, with the standard transformers pipeline (aggregation_strategy="simple"). Numbers from the unmodified piimb evaluate (0.3.0):

Task	v1	v2	v3	Best other model at release
ai4privacy-en	0.831	0.916	0.991	0.978
ai4privacy-multi (23 languages)	0.843	0.935	0.992	0.976
gretel	0.890	0.930	0.991	0.983
mapa-eur-lex (21 languages)	0.254	0.267	0.633	0.573
nemotron-pii	0.715	0.743	0.959	0.972
privy	0.575	0.848	0.993	0.987
Average	0.685	0.773	0.927	0.836

Every language improved over v2: on ai4privacy-multi the weakest language scores 0.989 (v2: 0.911), on MAPA EUR-LEX 0.558 (v2: 0.183).

Real text, strict scoring

A gold span counts as found only if every letter and digit in it is predicted with the right type, so missing part of a word is a miss. A predicted span counts as correct only if it lies inside a gold span of the same type and does not cut a word. Types are grouped into person, email, phone, address, ID, secret and URL. Scores use the word-level decoder in pii_detector.py.

Test set (real text, hand-checked gold)	Spans	v1	v2	v3
Mailing lists, Impressum/contact pages, raw scrapes, PDF text	332	0.891	0.982	0.992
Fresh web pages (contact pages from 10 sites)	75	0.921	0.968	0.976
FineWeb-2 chunks (multilingual, held out)	48	0.835	0.948	0.933
All real text	455	0.885	0.973	0.980

On all 455 real spans v3 has strict precision 0.975 and recall 0.985. The real test sets are small, so differences under about 0.01 are noise.

Quick start

Use the word-level decoder in pii_detector.py, shipped in this repo, rather than a plain per-token argmax, which emits sub-word fragments.

from huggingface_hub import hf_hub_download
import importlib.util

repo = "bardsai/eu-pii-anonimization-multilang-v3"
path = hf_hub_download(repo, "pii_detector.py")
spec = importlib.util.spec_from_file_location("pii_detector", path)
pii_detector = importlib.util.module_from_spec(spec); spec.loader.exec_module(pii_detector)

det = pii_detector.PIIDetector(repo)          # GPU if available, else CPU
text = "Kontakt: Jan Kowalski, tel. 600 123 456, jan.kowalski@example.pl, ul. Długa 5, 00-238 Warszawa"
for span in det.predict(text):
    print(span)
# {'start': 9, 'end': 21, 'label': 'PERSON_NAME', 'text': 'Jan Kowalski'} ...

print(det.anonymize(text))
# Kontakt: [PERSON_NAME], tel. [PHONE_NUMBER], [EMAIL_ADDRESS], [POSTAL_ADDRESS]

det.anonymize(text, labels={"PERSON_NAME", "EMAIL_ADDRESS"}) redacts only the listed types. If you do not want generic dates or web addresses masked, leave DATE_TIME and URL out of labels. Requires transformers (a version with ModernBERT support) and torch.

ONNX / CPU

ONNX exports are in onnx/. The decoder can run them with onnxruntime (the tokenizer still comes from transformers):

det = pii_detector.PIIDetector(repo, onnx="onnx/model_quantized.onnx")   # or "onnx/model.onnx"

Weights	Size	Strict F1, all real text (455 spans)
PyTorch / onnx/model.onnx (FP32)	1.2 GB	0.980
onnx/model_quantized.onnx (dynamic INT8)	0.3 GB	0.980

The FP32 export gives the same logits as PyTorch, including on padded batches (it is exported with eager attention). The INT8 model in the browser demo (transformers.js, WASM) scores 0.978.

How the decoder works

1. The text is split into 512-token windows with a 128-token overlap.
2. Each word gets one label from the mean of its sub-token probabilities, with B- and I- probability pooled per type. Words are split on whitespace and on , ; | " < > ( ) [ ] { } * ^, and edge punctuation is trimmed.
3. Neighbouring words of the same type are merged unless the next word looks like the start of a new entity.
4. Format checks drop implausible spans: a link must look like a URL, a phone number needs at least 5 digits, and ID types need at least 2 digits.
5. Every regex-matched email address is returned as EMAIL_ADDRESS.

Labels

PERSON_NAME, PERSON_ALIAS, PERSON_IDENTIFIER, PERSON_ATTRIBUTE, PERSON_ROLE_OR_TITLE, DATE_OF_BIRTH, DATE_TIME, EMAIL_ADDRESS, PHONE_NUMBER, CONTACT_HANDLE, POSTAL_ADDRESS, LOCATION, GEO_LOCATION, IP_ADDRESS, DEVICE_IDENTIFIER, IDENTIFYING_LINK, URL, ACCOUNT_IDENTIFIER, AUTH_SECRET, BANK_ACCOUNT_IDENTIFIER, PAYMENT_CARD, PAYMENT_CARD_SECURITY, FINANCIAL_AMOUNT, DOCUMENT_IDENTIFIER, DOCUMENT_REFERENCE, VEHICLE_IDENTIFIER, ORGANIZATION_NAME, ORGANIZATION_IDENTIFIER, PROPER_NAME, HEALTH_DATA, BIOMETRIC_DATA, ETHNIC_ORIGIN, POLITICAL_OPINION, RELIGION_OR_BELIEF, SEXUAL_ORIENTATION, TRADE_UNION_MEMBERSHIP, CRIMINAL_OFFENCE_DATA.

Company registration and VAT numbers (NIP, REGON, KvK, USt-IdNr, SIREN and similar) are ORGANIZATION_IDENTIFIER, not personal IDs. IDENTIFYING_LINK is a link that points to a person (profile pages, personal sites); URL is any other web address.

What changed from v2

• Two new types, DATE_TIME and URL, and every label of the source datasets is now mapped (v2 dropped dates, times, URLs and some attributes, which were then taught as "not PII").
• More training data, train splits only: Gretel PII masking EN v1, MAPA EUR-LEX (21 languages), the large privy train split and more OpenPII and Nemotron-PII rows. Rows whose text appears in the PIIMB test set were removed.
• Partial-annotation loss per source, with EU case-law citations in MAPA taught as "not PII".
• Calibration: the classifier bias of the O label is lowered by 1.0 (built into the weights), which raises recall. This setting was chosen on our real-text test sets and on the PIIMB sentences.
• Init: fine-tuned from the v2 weights for 2 epochs on the new mix.

Training data

All training data is openly licensed or generated:

• ai4privacy/pii-masking-openpii-1m (train split)
• nvidia/Nemotron-PII (CC-BY-4.0, train split)
• gretelai/gretel-pii-masking-en-v1 (Apache-2.0, train split)
• dglover1/mapa-eur-lex (CC-BY-4.0, train split)
• piimb/privy (MIT, train-large split; v2 also used a sample of test-small)
• HuggingFaceFW/fineweb-2 and FineWeb (ODC-By) text, labelled by Qwen3-30B-A3B-Instruct (Apache-2.0), with surrogate values
• Synthetic documents from bards.ai generators and an Apache-2.0 open-weight LLM

The domains used in the real-text test sets were excluded from the training text.

Limitations

• DATE_TIME masks generic dates and times; if you only need personal data, filter it out (see Quick start).
• nemotron-pii labels generic job words ("author", "consultant"), durations ("60 seconds") and blank form fields as PII. v3 masks these less often, which is most of its gap on that task.
• Remaining error patterns on real text: multi-line addresses with an extra campus or building line, labels glued to values ("T 09 331 01 01"), names glued to the next word, and venue or organisation names tagged as locations.
• Performance varies by language, domain and input quality. Heavy OCR noise and unusual layouts lower recall.
• This is a detection model, not a compliance program. Use it to support a redaction workflow with review where the stakes require it.

Files

• model.safetensors, config.json: weights and label mapping
• tokenizer.json, tokenizer_config.json: tokenizer
• onnx/model.onnx: FP32 ONNX export (inputs input_ids, attention_mask; output logits)
• onnx/model_quantized.onnx: dynamic INT8 quantization of the ONNX export, for CPU
• pii_detector.py: the decoder used for all reported scores (PyTorch or ONNX)

About bards.ai

We build product ML for teams shipping AI to real users. bards.ai