@xxxaz/stream-api-json

Stream-API JSON

• JavaScriptオブジェクトとJSON文字列間の逐次的な変換をStream APIを用いて実装しています。
  • Fetch APIのResponse.bodyにエンコードを行った上で.pipeTo可能です。
  • チャンク化されたレスポンスで特に効果を発揮するでしょう
• 各種モダンブラウザ及びNode.js(18以上)で共通して扱うことが可能です。
• 他パッケージに依存していません。
• 巨大なJSON文字列を限定されたメモリ環境下で扱うユースケースは想定されていません
  • JSONのパース処理において途中経過のデータは全てメモリ上に保持しています

---

• This library implements the sequential processing conversion between JavaScript objects and JSON strings with Stream API.
  • It can be encoded and piped to Response.body of Fetch API.
  • It is particularly effective with chunked responses.
• It can be used in various modern browsers and Node.js (version 18 and above).
• It does not depend on any other packages.
• This library is not intended for use cases involving handling huge JSON strings in limited memory environments.
  • All intermediate data during the JSON parsing process is stored in memory.

Usage

Server

import { createServer } from 'http';
import { StringifyingJsonArray, StringifyingJsonString, toNodeReadable } from '@xxxaz/stream-api-json';

async function * outputStream() {
    yield "one";
    yield "two";
    yield "three";
    yield new StringifyingJsonString(fibonacci());
}

async function * fibonacci() {
    let prev = 0;
    let cur = 1;
    while (cur < 1000) {
        yield String(cur);
        const sw = cur;
        cur += prev;
        prev = sw;
    }
}

createServer(async (req, res) => {
    const source = new StringifyingJsonArray(outputStream());
    const stream = await toNodeReadable(source);
    res.writeHead(200, {
        'Content-Type': 'application/json',
        'Transfer-Encoding': 'chunked'
    })
    stream.pipe(res);
})
.listen(8080);

Client

import { JsonStreamingParser, ParsingJsonArray, ParsingJsonString } from '@xxxaz/stream-api-json';

async function fetchStream(url: string) {
    const response = await fetch(url);
    const readableStream = response.body?.pipeThrough(new TextDecoderStream());
    const root = await JsonStreamingParser
        .readFrom(readableStream)
        .root();
    const element = document.querySelector('#parsing');
    if(!(root instanceof ParsingJsonArray)) throw new Error('response is not Array');
    for await (const row of root) {
        if(!(row instanceof ParsingJsonString)) throw new Error('row is not String');
        const p = document.createElement('p');
        p.innerText = await row.all();
        element.textContent = JSON.stringify(root.current, null, 4);
    }
}