perf: improved data serialization with stream

[wendraw]

Description

As discussed before #9 (comment) . dump data returns a huge object, when we serialize the object with JSON.stringify, Node.js throws OOM or stack overflow.

so, add msgpackr to serialize huge object with stream and buffer to resolve this problem.

Linked Issues

Additional context

[antfu]

Thanks! That's interesting, but I am afraid that it will introduce difficulties for other viewers to load that data (for example, instead I might write a script to consume that data instead of using the viewer).

I wonder if there is a better streaming solution to JSON format instead?

[wendraw]

Thanks! That's interesting, but I am afraid that it will introduce difficulties for other viewers to load that data (for example, instead I might write a script to consume that data instead of using the viewer).

I wonder if there is a better streaming solution to JSON format instead?

ok, I got your meaning.

I replaced msgpackr with WriteStream, then manual recursive handling of huge objects. and then OOM problem was resolved.

And the test/fixtures/vite can work very well! it has 931971 lines.

BUT, the command node --import regex-doctor/register ./node_modules/nuxi/bin/nuxi.mjs build doesn't work with the stream. the result is that the output.json content is empty.

So, I create a new script build:doctor-script to exec build command, and then it can work. BUT, it only tracks small data. it only has 4784 lines.

In summary, the performance problem seems to be solved by WriteStream.

I don't know why the WriteStream doesn't work in the node command. If you have a direct reason, please tell me.

[wendraw]

🚀with the new code, I had to handle the problem.

The command node --import regex-doctor/register ./node_modules/nuxi/bin/nuxi.mjs build doesn't work with the stream.