Frequency dictionary

HU Freq (OpenSubtitles) is this project’s frequency dictionary: 50,000 Hungarian lemmas ranked by how often they occur in film and TV subtitles — i.e., ranked by spoken Hungarian, which is what an immersion learner actually meets.

With it installed, every Yomitan popup shows the word’s frequency rank — instantly answering the eternal mining question: “is this word worth a card?” (Rough guide: rank < 10k almost always yes; > 30k, only if it keeps showing up in your content.)

Install

1. Download hu-freq-opensubtitles.zip from our GitHub releases.

2. Yomitan settings → Dictionaries → Import (or paste the release URL into Import from URLs). Don’t unzip it.

3. Look up any word — the rank appears next to the entry. In Anki cards, the {frequencies} marker now carries it too.

Also in the release: hu_lemmas_50k.csv (lemma, rank, count) — the raw list behind the Magyar Starter deck.

How it’s built

Fully open — the pipeline lives in scripts/frequency/:

1. Token counts from the OpenSubtitles 2018 Hungarian corpus (hermitdave/FrequencyWords, CC-BY-SA 4.0)
2. Lemmatization with simplemma, counts aggregated per lemma (so vagyok, voltál, lennénk all feed van)
3. Validation against the hunspell hu_HU dictionary to drop subtitle typos and foreign junk
4. Top 50,000 lemmas → Yomitan rank-based frequency format

Other frequency resources

• Wiktionary’s Hungarian frequency list (1–10,000) — a second opinion from a different corpus
• MNSZ query interface — the Hungarian National Corpus; set the personal/informal subcorpus for spoken-like frequencies
• awesome-hungarian-nlp — the master list of Hungarian NLP tools and corpora, for anyone hacking on pipelines like ours

Honest limitations (v0.1)

• Lemmatization is dictionary-based and misses some longer suffix chains — a few inflected forms survive as separate “lemmas” in the deeper ranks. The top ~10k, where mining decisions actually happen, is largely clean.
• Subtitle language over-represents dialogue vocabulary (fine for learners — that’s a feature).

Spotted a wrong entry? Open an issue — corrections regenerate the dictionary in one script run.