Code

[persist] Structured file format#31080

Code

Merged

bkirwi merged 11 commits intoMaterializeInc:mainMaterializeInc/materialize:mainfrom bkirwi:structured-on-disk-2bkirwi/materialize:structured-on-disk-2Copy head branch name to clipboardJan 27, 2025

+374-513

Commits

Commits on Jan 21, 2025

• Allow encoding batches with only structured data

  bkirwi

  committed
  1274067

  Copy full SHA for 1274067

• Allow decoding structured-only data

  bkirwi

  committed
  7a12e7d

  Copy full SHA for 7a12e7d

• Add a new batch format variant

  bkirwi

  committed
  d8352f6

  Copy full SHA for d8352f6

• Enable structured-only writes in CI

  bkirwi

  committed
  66bcbaf

  Copy full SHA for 66bcbaf

• Decode structured-only data from proto

  bkirwi

  committed
  2bf7edc

  Copy full SHA for 2bf7edc

• Fill in codec data at read time where needed

  bkirwi

  committed
  668fbc3

  Copy full SHA for 668fbc3

• Validate method now always has decoded types available

  bkirwi

  committed
  74c5052

  Copy full SHA for 74c5052

• Normalize away the encoded part

  Show description for 27438aa
  bkirwi

  committed
  27438aa

  Copy full SHA for 27438aa

• Make decoding codec data optional in FetchedPart

  bkirwi

  committed
  6078362

  Copy full SHA for 6078362

• Refactor FetchedPart to avoid as much decoding as possible

  Show description for e60157d
  bkirwi

  committed
  e60157d

  Copy full SHA for e60157d

Commits on Jan 24, 2025

• Comments suggested by review

  bkirwi

  committed
  41d0aea

  Copy full SHA for 41d0aea