When I run npm start with a Polkadot archive node locally, it results in:
node:buffer:593
slice: (buf, start, end) => buf.utf8Slice(start, end),
^
Error: Cannot create a string longer than 0x1fffffe8 characters
at Object.slice (node:buffer:593:37)
at Buffer.toString (node:buffer:811:14)
at Object.readFileSync (node:fs:497:41)
at main (/home/vados/fork-off-substrate/index.js:123:31)
at processTicksAndRejections (node:internal/process/task_queues:96:5) {
code: 'ERR_STRING_TOO_LONG'
}
NodeJS only allows for strings to be 512 MiB, currently the state of Polkadot seems to be larger that that.
Example file.
There are some async parsing libs, but they are slow and I dont know how to write the state back to disk with them either...
When I run
npm startwith a Polkadot archive node locally, it results in:NodeJS only allows for strings to be 512 MiB, currently the state of Polkadot seems to be larger that that.
Example file.
There are some async parsing libs, but they are slow and I dont know how to write the state back to disk with them either...