v0.17.0-beta.7

What's Changed

New Features 🎉

• feat: pushdown limit & offset into file reader if there is no filter by @westonpace in #2747

Bug Fixes 🐛

• fix: fix two cases in the v2 decoder where the decode order didn't match the scheduling priority by @westonpace in #2754
• fix: fix several situations where we were incorrectly inferring the storage version by @westonpace in #2756

Performance Improvements 🚀

• perf: various fixes to improve shuffling performance at high scales by @westonpace in #2710

Full Changelog: v0.17.0-beta.6...v0.17.0-beta.7

v0.17.0-beta.6

What's Changed

New Features 🎉

• feat: make the I/O buffer size configurable by @westonpace in #2736

Documentation 📚

• docs: fix version table so it renders by @wjones127 in #2745
• docs: document newly added compaction options in optimize.py by @westonpace in #2746
• docs: fix transaction conflict table by @wjones127 in #2744

Full Changelog: v0.17.0-beta.5...v0.17.0-beta.6

v0.17.0-beta.5

What's Changed

New Features 🎉

• feat: add fixed size binary encoding to lance by @raunaks13 in #2707

Bug Fixes 🐛

• fix: raise exception if user passes unsorted indices to take_rows() of lance file api by @raunaks13 in #2729
• fix: flaky test caused by PQ distortion by @BubbleCal in #2732
• fix: don't panic when v2 scans end early (reader is dropped) by @westonpace in #2690
• fix: fix a crash that could sometimes happen reading largish string/binary data in a list by @westonpace in #2731

Documentation 📚

• docs: clean up quickstart notebook and add tags example usage by @dsgibbons in #2735

Performance Improvements 🚀

• perf: cache default session contexts by @dsgibbons in #2709

New Contributors

• @dentiny made their first contribution in #2740

Full Changelog: v0.17.0-beta.4...v0.17.0-beta.5

v0.17.0-beta.4

What's Changed

New Features 🎉

• feat: add max_bytes_per_file and batch_size to CompactionOptions by @westonpace in #2728

Performance Improvements 🚀

• perf: coalesce ids before executing take by @westonpace in #2680

Full Changelog: v0.17.0-beta.2...v0.17.0-beta.4

v0.17.0-beta.3

What's Changed

Bug Fixes 🐛

• fix: drop indices if all fragment ids are missing by @wjones127 in #2720
• fix: ignore pages with zero rows by @westonpace in #2724
• fix: various changes to fix backpressure by @westonpace in #2721

Full Changelog: v0.17.0-beta.1...v0.17.0-beta.3

v0.17.0-beta.2

What's Changed

Bug Fixes 🐛

• fix: drop indices if all fragment ids are missing by @wjones127 in #2720
• fix: ignore pages with zero rows by @westonpace in #2724
• fix: various changes to fix backpressure by @westonpace in #2721

Full Changelog: v0.17.0-beta.1...v0.17.0-beta.2

v0.17.0-beta.1

What's Changed

Breaking Changes 🛠

• feat!: index_statistics returns the concrete index type by @BubbleCal in #2716

Full Changelog: v0.16.1...v0.17.0-beta.1

v0.16.1

What's Changed

New Features 🎉

• feat: make scalar index training configurable by @dsgibbons in #2686
• feat: migrate Bitmap & LabelList index file to v2 format by @BubbleCal in #2677
• feat: make default batch size configurable via environment variable by @westonpace in #2693
• feat: move-stable row ids in compaction by @wjones127 in #2544
• feat: load multiple ivf-pq sorted files and commit index into dataset by @raunaks13 in #2681

Bug Fixes 🐛

• fix: out of bounds error in bitpacking by @albertlockett in #2692
• fix: add fix for slicing single array into many pages but don't have enough rows to do so by @westonpace in #2702
• fix: optimize cosine IVF_PQ index by @chebbyChefNEQ in #2698
• fix: search results are not correct after appending index by @BubbleCal in #2701
• fix: v2 writer used the wrong value to calculate how many rows were available in array by @westonpace in #2704
• perf: bitpacking exclude sign bit if not needed by @albertlockett in #2696
• fix: correct index type of inverted index by @BubbleCal in #2715

Performance Improvements 🚀

• perf: various fixes to improve vector transform performance at high scales by @westonpace in #2703

Full Changelog: v0.16.0...v0.16.1

v0.16.1-beta.4

What's Changed

New Features 🎉

• feat: load multiple ivf-pq sorted files and commit index into dataset by @raunaks13 in #2681

Bug Fixes 🐛

• perf: bitpacking exclude sign bit if not needed by @albertlockett in #2696
• fix: correct index type of inverted index by @BubbleCal in #2715

Performance Improvements 🚀

• perf: various fixes to improve vector transform performance at high scales by @westonpace in #2703

Full Changelog: v0.16.1-beta.3...v0.16.1-beta.4

v0.16.1-beta.3

What's Changed

New Features 🎉

• feat: move-stable row ids in compaction by @wjones127 in #2544

Bug Fixes 🐛

• fix: optimize cosine IVF_PQ index by @chebbyChefNEQ in #2698
• fix: search results are not correct after appending index by @BubbleCal in #2701
• fix: v2 writer used the wrong value to calculate how many rows were available in array by @westonpace in #2704

Full Changelog: v0.16.1-beta.2...v0.16.1-beta.3