HuggingFace Data Loader (JS)

ETag-based updates, IndexedDB cache (browser), streaming download (Node).

Import

import { HuggingFaceDataLoader } from '@xaikorea0/js-client/utils/hf-loader';
const loader = new HuggingFaceDataLoader();

Browser (IndexedDB + ETag)

const updated = await loader.checkForUpdatesBrowser();
const data = await loader.getLocalDataBrowser();

• Stores { etag, data } in IndexedDB via idb.
• Call periodically (e.g., monthly) to refresh.

Node.js (stream + FS cache)

const data = await loader.downloadDataNode();
console.log(data.laws?.length);

Tips

• Reuse cached data to minimize bandwidth.
• Pair with queryBatch for many questions after data load.

JS API · JS Quick Start · Docs Home