Working with Large JSON Files in Node.js Without Running Out of Memory

JSON.parse(fs.readFileSync(...)) works fine until the file is a few hundred megabytes, at which point Node.js throws "JavaScript heap out of memory". This guide covers why that happens and how to process large JSON files — export dumps, log archives, dataset files — without loading the entire thing into memory at once.

Why the Naive Approach Fails

const fs = require('fs');

// This reads the ENTIRE file into a string, then parses the ENTIRE string
// into an object — both held in memory simultaneously.
const data = JSON.parse(fs.readFileSync('orders-export.json', 'utf8'));

// For a 2GB file, this can easily exceed Node's default heap size
// and crash with: FATAL ERROR: Reached heap limit Allocation failed
// - JavaScript heap out of memory

The problem is that both readFileSync and JSON.parse are synchronous, whole-file operations — there is no way to parse "part of" a JSON document with the built-in parser.

Solution 1: Stream-Process with stream-json

The stream-json package parses JSON incrementally as bytes arrive, emitting events for each array element without holding the whole array in memory:

npm install stream-json

const fs = require('fs');
const { parser } = require('stream-json');
const { streamArray } = require('stream-json/streamers/StreamArray');

let count = 0;
let totalRevenue = 0;

fs.createReadStream('orders-export.json')
  .pipe(parser())
  .pipe(streamArray())
  .on('data', ({ value: order }) => {
    // Each "order" object is processed one at a time
    count++;
    totalRevenue += order.total;
  })
  .on('end', () => {
    console.log(`Processed ${count} orders, total revenue: ${totalRevenue}`);
  })
  .on('error', (err) => {
    console.error('Stream error:', err);
  });

// Memory usage stays flat regardless of file size, because only
// one order object is held in memory at a time.

Solution 2: NDJSON / JSON Lines for New Data

If you control how the data is written, avoid a single giant JSON array entirely. Newline-delimited JSON (NDJSON) writes one JSON object per line, which can be read and processed line-by-line with a simple readline stream:

// orders.ndjson
{"id":1,"total":499}
{"id":2,"total":1299}
{"id":3,"total":899}

const readline = require('readline');
const fs = require('fs');

const rl = readline.createInterface({
  input: fs.createReadStream('orders.ndjson'),
  crlfDelay: Infinity
});

let totalRevenue = 0;

rl.on('line', (line) => {
  const order = JSON.parse(line);
  totalRevenue += order.total;
});

rl.on('close', () => {
  console.log('Total revenue:', totalRevenue);
});

NDJSON is what most log pipelines and big-data export tools (BigQuery, Elasticsearch bulk export) use internally, precisely because it is trivial to stream.

Solution 3: Increase Node's Memory Limit (Short-Term Fix)

If streaming is not immediately feasible, you can raise Node's heap limit as a stopgap — but this does not scale indefinitely and is not a substitute for streaming:

node --max-old-space-size=4096 process-orders.js
# Raises heap limit to 4GB; still bounded by available system RAM.

When to Choose Which Approach

• File under ~50MB — JSON.parse(readFileSync()) is fine, no need to complicate things.
• File is a large array of records you did not create — use stream-json to process it incrementally.
• You control the write path (logs, exports) — switch to NDJSON so consumers never need to load the whole file.
• You need random access, not sequential processing — consider loading the data into a database or SQLite file instead of parsing JSON repeatedly.

Frequently Asked Questions