jmdict-wrapper

JMDict-Simplified for Node.js

@scriptin's jmdict-simplified project provides a sane JSON version of the famous JMDict open-source Japanese dictionary project.

This current project, jmdict-simplified-node (the one you're reading about), helps Node.js applications load JMDict-Simplified's JSON into a LevelDB database to facilitate fast searches for both text (which often contain kanji) and readings (no kanji), on both prefixes and full-text. It does this by simply creating indexes on all substrings of all text and readings.

This means that after a one-time setup, your apps can start instantly and search this dictionary with lightning speed (all thanks to LevelDB of course). Note that you don't need to know or care anything about LevelDB to use this library—it handles all the details for you.

This project also contains TypeScript interfaces describing the JMDict-Simplified project, allowing your TypeScript projects to effortlessly navigate this data.

Installation and setup

I expect you have a Node.js project already. In it,

1. install jmdict-simplified-node: npm i jmdict-simplified-node
2. download a recent release of the JMDict-Simplified JSON
3. import jmdict-simplified-node into your project: in TypeScript, this would be import {setup as setupJmdict} from 'jmdict-simplified-node'
4. setup: const jmdictPromise = setupJmdict('my-jmdict-simplified-db', 'jmdict-eng-3.1.0.json');

Tutorial

For full details about the API, see the next section, but in a nutshell, here's how you can start using this library after you complete the two installation steps above (install this npm package and download JMDict-Simplified):

import {readingBeginning, setup as setupJmdict} from 'jmdict-simplified-node';

if (module === require.main) {
  (async function main() {
    const jmdictPromise =
        setupJmdict('my-jmdict-simplified-db', 'jmdict-eng-3.1.0.json');
    const {db} = await jmdictPromise;
    // This `db` is used by all functions in this API, so hang on to it.
    // You only need to run `setup`/`setupJmdict` ONCE in your app, to get this `db`.
    const results = await readingBeginning(db, 'あおい', 3);
    console.dir(results, {depth: null});
  })();
}

Drop the above in demo.ts, and run npx ts-node demo.ts. The first time you run this, it'll take a minute or two to cache all the entries before printing out three JMDict entries whose readings start with あおい. If you rerun it, it'll run in an instant.

For full details on what these library functions are doing, and what other functions this library provides, read on.

API

setup(dbpath: string, filename = '', verbose = false, omitPartial = false): Promise<SetupType>

Always call this first before using any other function in this API: this function returns an object you need to call all other functions. You only need to call this function once in your entire application.

Given

• the dbpath, the path you want your LevelDB database to be stored,
• optionally the filename of the JMDict-Simplified JSON,
• optionally a verbose flag,
• optionally an omitPartial flag (the default false will allow full-text searches, use true here for a much smaller database if you don't plan on using readingAnywhere or kanjiAnywhere)

this function will return a promise of the following data:

export type SetupType = {
  db: Db,
  dictDate: string,
  version: string,
};

The first of these, db, is required by all lookup functions in this API, so hang on to this. The two strings are informational.

If a proper LevelDB database is not found in dbpath, this function will look at filename and parse the JSON in it. It takes ~90 seconds to take a 234 MB JSON file and create a 140 MB LevelDB database on a 2015-vintage Mac laptop.

Protip: if you plan on always having the LevelDB database for your app, you can just run this setup once (in your app's post-install stage maybe?) and never call this with a filename.

Protip: in my apps, I just hang on to the promise returned by this function and, in each place that needs to call anything else in this API, I await this promise. That way I don't have to ever worry about a function trying to do a lookup before the data is available.

readingBeginning(db: Db, prefix: string, limit?: number): Promise<Word[]>

Find all readings starting with a given prefix. Needs a Db-typed object, which was one of the things setup gave you. limit defaults to -1 (no limit) but isn't super-useful since this project doens't yet support paginated search. Get in touch if you need this.

Returns a promisified array of Words. A Word is an entry in JMDict, and contains things like

• an id to uniquely identify it in the dictionary,
• kanji, or the text being defined (might or might not actually include something you can call kanji: ＣＤ and 日本 are two examples),
• kana, the reading (that is, the pronunciation) of this kanji text,
• sense, i.e., the various dictionary senses this word can have.

Look at interfaces.ts for the details. It very carefully follows the soft-schema of the upstream jmdict-simplified project.

readingAnywhere, kanjiBeginning, kanjiAnywhere

These three have the same signature as readingBeginning above:

readingAnywhere(db: Db, text: string, limit?: number): Promise<Word[]>
kanjiBeginning(db: Db, prefix: string, limit?: number): Promise<Word[]>
kanjiAnywhere(db: Db, text: string, limit?: number): Promise<Word[]>

They search the reading or kanji (text) fields, either via a prefix (to match the beginning) or by free text to match anywhere.

Recall that readingAnywhere and kanjiAnywhere will yield no results of setup was called with the omitPartial argument set to true.

getTags(db: Db): Promise<Simplified['tags']>

JMDict uses a large number of acronyms that it calls "tags", e.g.,

• "MA" for "martial arts term",
• "aux-v" for "auxiliary verb",
• "fem" for "female term or language".

These acronyms will be found in the hits yielded by the four lookup functions above.

This function will return an object mapping these abbreviations to their full meaning.

getField(db: Db, key: keyof BetterOmit<Simplified, 'words'>): Promise<string>

There are a small handful of extra pieces of information that the original JSON includes, things like

• dictDate, the date the original JMDict XML file was created,
• dictRevisions, a list of revisions in the original JMDict XML file, etc.

This function lets you access these.

idsToWords(db: Db, idxs: string[]): Promise<Word[]>

This helper function will expand a list of JMDict word IDs to the full definition. This might be helpful if you only transmit words' IDs, for example.