Access to Intenter

Intenter is released for research, evaluation, internal validation and education. Tell us who you are and we will grant access.

By requesting access you agree to the Elda Community License 1.0: no commercial use, no redistribution of the weights or derivatives, and attribution as "Built with Elda". Commercial licensing is available on request.

Elda-Intenter — real-time speech-act + entity extraction in a single pass

Korean-first, multilingual (ko · ja · en). 307M parameters · 8,192-token window · single forward pass.

⚠ Latency is not stated for this build. The previous release measured p50 27.8 ms on a 12-layer / 512-token backbone; this one is 22 layers with a 8,192 window, so that number does not carry over and we have not re-measured it.

One forward pass returns four aligned channels: what the speaker is doing (speech act), what they are talking about (entities), and the attribute/predicate structure binding them. Not a general-purpose NER model — the perception layer of a live conversational system.

"간판의 품격이라는 음식점이 있다며, 그 음식점은 어디에 있냐고"

  intent    NARRATE · DESCRIBE · QUESTION      (three spans, one pass)
  entity    간판의 품격 → ORGANIZATION
            음식점      → ORGANIZATION
  signals   is_pure_negation=0 · is_claim=0 · polarity=0 · depends_on_prev=1

What is different here — byte-level spans, so an unseen name is still a span

This build runs on a byte-level BPE tokenizer (Gemma-2 vocabulary, 256k). That is a decoder-style property in an encoder, and it is the point of the model, not a detail.

28 shop names that are not in any index, tokenized:
  previous build (SentencePiece unigram)   20 / 28 become [UNK]
  this build (byte-level BPE)               0 / 28 become [UNK]

A name that becomes [UNK] cannot be a span, and a missing span is worse than a wrong type: the layer downstream has no string to look up. With byte fallback every surface is representable — rare syllables, emoji, (@_!)--/ punctuation runs — so the span survives even when the name is new.

"🍜 너울사인 간판 재질 알려줘"   → 너울사인 / ORGANIZATION · REQUEST
"ㅸㅿ뷁공방 어디야"              → ㅸㅿ뷁공방 / ORGANIZATION · QUESTION

Both are shop names that do not exist; the second is deliberately unpronounceable. Neither produces an [UNK] token on this build.

★ Representable is not the same as emitted. On a held-out probe of synthetic shop names built from syllables the previous tokenizer could not encode, anchor reach went 0% → 95% in the question frame — but the same names inside an emoji-heavy frame still abstained 70–75% of the time. Tokenization removes the floor; the span still has to be taught. That gap is this model's open work.

Results

All sets below are excluded from training by construction.

MASSIVE (dev, re-annotated) — conversational speech, same five peers

Voice-assistant utterances, our actual traffic shape. Same axes, same particle rule; peers emit KLUE's six types, so only the shared axes are comparable and type is omitted.

	abstention ↓	span recall	boundary	precision	anchor reach
ko ours (469 sent.)	2.7	97.3	96.5	96.6	81.8
ko — 5 peers	33.7 – 67.9	32.1 – 66.3	69.9 – 86.4	89.4 – 94.7	6.6 – 72.7
ja ours (749 sent.)	2.7	97.3	97.7	95.0	91.3
ja — 5 peers	30.1 – 76.2	23.8 – 69.9	50.8 – 84.0	72.3 – 93.9	18.8 – 48.0
en ours (796 sent.)	2.9	97.1	92.6	82.1	95.2
en — 5 peers	38.6 – 69.4	30.6 – 61.4	12.5 – 63.3	90.7 – 94.8	28.5 – 64.0

We lead every axis on ko and ja, and all but precision on en. Entity F1 under our own scorer: ja 79.7 · ko 85.5 · en 74.5 (one trailing function word allowed).

Not comparable to published MASSIVE scores: that benchmark is intent classification plus slot filling with 55 slot types. We map 21 onto 11 of our entity types; peers are KLUE-NER models run outside their training domain. Zero training overlap on dev is verified by the evaluator.

KLUE-NER (dev · 600 sentences / 1,687 gold), entity-level F1

	entity F1	boundary F1
exact match	84.9	87.2
allowing one trailing Korean particle	86.2	88.5

Korean particles attach to the noun. The second row accepts one trailing particle when the start offset matches. ⚠ On this build the two rows are only 1.3 pp apart, because the model now emits bare nouns — see Korean particles under Limitations.

Measured with our own scorer on the same inputs, a KLUE-NER specialist fine-tuned on the full KLUE train set scores 83.4 (soddokayo/klue-roberta-large-klue-ner). This model sees 500 KLUE sentences plus 14k news sentences of it and is otherwise a general conversational model.

build	KLUE F1 (one particle)	note
previous release (12-layer, 512)	63.9	SentencePiece unigram
this release (22-layer, 8,192)	86.2	byte-level BPE + news sentences

The jump is two changes at once — backbone and training data — so it is not a clean ablation of either.

KLUE-NER — the same five peers, on their own training domain

Every peer is a dedicated KLUE-NER model fine-tuned on all 21,008 KLUE-NER training sentences, emitting exactly KLUE's six types. We see 12,500 of them, emit 17 types projected onto those six, and run four channels plus six auxiliary heads in the same pass.

⚠ The table below is stale. It was measured on the previous backbone generation and has not been re-run since the backbone changed — in that interval our KLUE entity F1 moved 63.9 → 86.2, so these per-axis numbers understate this build. We are re-measuring with the same five peers and will replace the table rather than delete it. Treat the F1 figures above as current and this table as a floor.

axis	ours	KF-DeBERTa	KLUE-BERT	RoBERTa-large	KoELECTRA	RoBERTa-base
type accuracy	88.2	98.2	97.7	98.1	95.1	92.2
abstention (lower better)	14.0	2.0	2.4	13.6	18.3	41.1
span recall	86.0	98.0	97.6	86.4	81.7	58.9
boundary	97.2	97.3	96.2	96.2	91.9	62.9
precision	89.6	97.2	97.4	98.0	96.0	96.7
anchor reach	79.2	94.8	94.6	69.8	54.8	13.0

We lose on every axis; boundary is a tie. Third or fourth of six. Note the peer spread — anchor reach runs 13.0 to 94.8, so a single-peer table could have been picked to favour us.

Axes. abstention — nothing emitted at a gold position. span recall — something emitted there. boundary — returned span contains the gold string. precision — emitted spans landing on gold, over spans carrying a KLUE-mappable type (1,427/1,592); over all our spans it is 78.9, since 165 carry types KLUE does not annotate (WORK, ARTIFACTS, TERM, EVENT, DURATION). anchor reach — span returned under a type that opens a downstream lookup (LOCATION / ORGANIZATION; 447 of 1,687 gold here).

Our training mix is 4.0 % news. KLUE-NER is news text; our material is wiki, conversational Korean and synthetic templates. Whether these peers hold up on conversational Korean is untested here.

Reported by others, on their own runs — not our ruler, not verified by us: KLUE-RoBERTa-large 90.8 · XLM-R-large 85.9 · KR-BERT-base 77.2 · mBERT-base 73.2 (KLUE paper). Our re-scoring of a KLUE-NER RoBERTa-large on this split gave 83.4, not 90.8.

Frozen internal gates

Re-run on every build, evaluated per surface, not summed.


Role nouns — detection / typing (405 slots)	401 / 399
Referent expressions, held out	23 · 23 · 22
Conversational-speech detection / safety floor	92.4 % · 12 / 12
Possessive structure — owner kept (40 surfaces)	31 / 40
Discourse signals — false positives on short utterances	1 / 48
Discourse signals — polarity grid	23 / 24
Formal-register interrogatives	13 / 14
Proper-name span probe, third-party frames (80 slots)	78 / 80
Name-span probe, third-party (698 slots)	685 / 698
Span-convention compliance (126 slots)	126 / 126

The two third-party probe rows are produced by a separate team on their own infrastructure.

Why an encoder

	Elda-Intenter	typical small-LLM extraction
Parameters	307M	500M – 4B
Latency (single, incl. heads)	not measured on this build	hundreds of ms
Latency (batch of 16)	118.5 ms	seconds
Serving precision	fp32 (2 workers, 6.1 GB GPU)	varies
Output	fixed contract, char offsets	free text to be parsed

Measured on the production serving path (RTX 3060, fp32, heads attached). Span extraction is classification over token pairs, and a bidirectional encoder sees the whole utterance at once.

Output contract

Four Global Pointer channels over one shared mmBERT-base backbone (22 layers, 8,192-token window, byte-level BPE), plus binary signal heads and one span-attribute head. All channels are character spans over the original string.

channel	what it carries
`intents`	speech act over the utterance or clause — 18 types incl. AGREE / DISAGREE / QUESTION / COMMAND / NARRATE / DESCRIBE / REQUEST
`entities`	17 entity types × subtypes, plus `about_speaker` per span
`attributes`	modifiers bound to an entity
`predicates`	what is asserted of it
`signals`	four binary discourse signals, plus two three-valued judgements, each with probabilities

about_speaker is true / false / null (null = abstains). signals also carries place_q ("did the speaker point at one specific place or business?") and needs_map ("must something outside the model be consulted to answer?"), each "yes" / "no" / null with a probability. No threshold is applied to any of them — label and probability are both emitted and the cut is the consumer's. See ABOUT_SPEAKER_OUTPUT_CONTRACT.md and SIGNALS_JUDGE_OUTPUT_CONTRACT.md.

Spans may nest (「제 친구」 and 「친구」). Key by span offsets, not by surface.

Usage

No from_pretrained one-liner — custom four-channel span architecture.

import json, sys
from pathlib import Path

d = Path("path/to/this/repo")
sys.path.insert(0, str(d))
import modeling_btrack_4ch_w4 as M

m = M.load(str(d), "cuda")        # base + 5 gap-fill heads + about_speaker + W4 + 2 judge heads
print(json.dumps(M.infer(m, "판교 카카오 본사 어디야"), ensure_ascii=False, indent=1))

# model.safetensors carries the same weights in the same dtype, if you prefer it:
# from safetensors.torch import load_file
# m.load_state_dict(load_file(d / "model.safetensors"))

backbone_config.json is included — the repository is self-contained. md5sum -c MD5SUMS verifies a download in full; model.pt is bit-identical to the file answering live traffic.

Serve in fp32 (weights stored bf16). INT8 collapses this model — measured.

Limitations

Korean first. ja and en are supported and measured; Korean gets the curated material.
Not a general NER model. Type inventory and span scope come from a conversational product.
Korean particles — this build trims them. Earlier releases kept the particle inside the span (어부가); this one emits the bare noun (어부). We measured the shift rather than intending it: the gap between exact match and one-particle-allowed scoring collapsed from 8.6 pp to 1.3 pp, and on a held-out probe 79.5 % of the spans that changed were particle-bearing spans replaced by their bare form (97 matched pairs, zero cases of a noun being cut short). The training mix now includes KLUE-NER, whose gold excludes particles, and the model followed that convention. Whether to pull it back is an open decision for us — reverting costs KLUE score.
News text is no longer the weak spot — 14k KLUE news sentences are in the mix, and KLUE F1 went 63.9 → 86.2. That gain came with the losses listed under Known regressions below.
is_pure_negation has false positives no threshold recovers (8 fp / 8 fn on 3,193 rows at τ=0.7). A soft signal, not a filter.
Boundaries, not senses. Entity linking is a separate layer, not in this repository.

Known regressions (vs. the current production model)

This build changed the backbone, so some axes moved. We publish the numbers rather than the summary.

gate	production	previous release	this build
risk-utterance floor, serving form (14 lines)	14 / 14	13 / 14	★ 14 / 14
referential expressions, held out (23 slots)	23 / 23	22 / 23	★ 23 / 23
connective-clause abstention (50 cells)	0 / 50	0 / 50	0 / 50
role nouns, detected (405 slots)	401	405	404
map-intent judge heads, consumer sheet	114 / 114	112 / 113	112 / 113
entity-set drift probe, net (boundary moves excluded)	baseline	−50	−128

The safety floor and the referential axis are back to production parity — the first time in this backbone generation. What we gave up is the drift probe: of 541 changed spans, 366 are boundary moves and 175 are real losses, concentrated in one-syllable object nouns (배 · 종 · 성문 · 빵 · 물). That axis has not closed in six builds and is our open work.

The judge loss is one shape: "{district} {trade} signage material" reads as find me a shop rather than tell me about signage, confidently, on four consecutive builds.

★ Latency is not in this card. 22 layers and a 8,192 window are not the 12/512 shape the previous p50 27.8 ms was measured on, and we have not re-measured. Assume nothing from that number.

This repository

Build 45dc3282. Refreshed roughly monthly. ⚠ This build is not yet the production model — it is published for evaluation while the serving team measures latency and decides promotion. The previous backbone generation is what live traffic sees today. Two output contracts ship alongside the weights (ABOUT_SPEAKER_OUTPUT_CONTRACT.md, SIGNALS_JUDGE_OUTPUT_CONTRACT.md).

Attribution — third-party training data

Parts of the training corpus are adapted from publicly licensed datasets. Their licenses apply to those parts and are reproduced here.

KLUE — KLUE: Korean Language Understanding Evaluation, Park et al., 2021. Licensed under CC BY-SA 4.0. Source: https://github.com/KLUE-benchmark/KLUE · paper: arXiv:2105.09680.

Changes we made (this is an adaptation, not a copy). 2,500 sentences from the KLUE-NER training split were re-annotated under our own span convention and type inventory: KLUE's six entity types were mapped onto our seventeen, sentences longer than four eojeol were dropped, single-character entities were dropped, and sentences overlapping our held-out probes were removed. Intent, attribute and predicate channels carry no KLUE labels and are masked out during training. The dev split is used for evaluation only and appears nowhere in training — the evaluator verifies this and refuses to run otherwise.

MASSIVE — MASSIVE: A 1M-Example Multilingual Natural Language Understanding Dataset, FitzGerald et al., Amazon Science. Licensed under CC BY 4.0. Source: https://github.com/alexa/massive.

Changes we made. Korean and Japanese rows were re-annotated under our span convention; 21 of MASSIVE's 55 slot types were mapped onto 11 of our entity types. MASSIVE's intent labels are not used — that channel is masked out per source. Dev rows used for evaluation are excluded from training by construction.

Everything else in the corpus is our own work. Training data and evaluation sets are not distributed with this repository.

License & access

Released under the Elda Community License 1.0 (see LICENSE).

✅ Research, evaluation, internal validation, education — free of charge
✳ Attribution: "Built with Elda"
⛔ Commercial use and redistribution require a separate agreement

Access is gated: tell us who you are and access is granted automatically. Training data and evaluation sets are not distributed.

Citation

@software{intenter2026,
  title  = {Elda-Intenter: real-time multilingual speech-act and entity extraction},
  author = {Elda AI},
  year   = {2026},
  url    = {https://huggingface.co/Elda-AI/intenter}
}

Downloads last month: -

Safetensors

Model size

0.3B params

Tensor type

BF16

Datasets used to train Elda-AI/intenter

Paper for Elda-AI/intenter

KLUE: Korean Language Understanding Evaluation

Paper • 2105.09680 • Published May 20, 2021 • 1