Reading parts of JSON (several times)

[sam20908]

I have one giant JSON that contains some data for different "components" of my app, and I would like to separate out the variables that stores the data into each component (not one giant struct). Is it possible?

Also, what's the most performant way to do it? I know generic JSON can help me do this, but I know my keys and hope that I can somehow take advantage of small dynamic allocations

[stephenberry]

Hi! I would not use generic JSON for this problem, because that will load the entire structure into memory and be slow.

You can use glz::skip to indicate values that you do not want to read in. This will acknowledge the keys existence and simply skip the value. But, if may be annoying to add a bunch of skips. You can also set error_on_unknown_keys to false, which will then skip any keys that you have not added to your struct. This might be your best option, because then you can simply register the keys that you want to decode.

struct S {
  int i{};
};

template <>
struct glz::meta<S> {
  static constexpr auto value = object("key_to_skip", skip{}, "x", &S::i);
};

So, in short, look into using error_on_unknown_keys = false. If this doesn't work for you, consider skip. If neither of these are good solutions, I'd love to get a bit more information on your problem, but you may be best off using simdjson. Note that these recommended solutions skip values that you do not care about, so it avoids allocating any memory and makes parsing faster.

[sam20908]

That makes sense. I wonder what you mean by "skip" values, because does that mean it avoids parsing the skipped keys and values entirely (something like a linear scan of the string every time the string is parsed but it just goes over the skipped parts)? I might be misunderstanding so please help me out 👍

[stephenberry]

@sam20908,
It means that the value associated with the key is not read into C++ memory. It will simply parse over the value. Now, the value could be an entire JSON object or array, and it will just parse over it and then read in the next key that doesn't have a skip. I hope that makes sense.