Tags: vortex-data/vortex
Tags
Avoid re-building vortex-duckdb twice (#8035) ## Summary Currently, if you build something like `duckdb-bench` cargo will rebuild `vortex-duckdb` once before it'll be satisfied that it doesn't need to rebuild it. The root cause is that cargo's fingerprinting can't accurately track files that are created **during** the build. This change is fine because we still track our own changes, the only external change will be something like a duckdb version upgrade which will change `build.rs` and re-trigger the build as expected. Signed-off-by: Adam Gutglick <adam@spiraldb.com>
bench: bit-packed compare-constant baseline (#8012) ## Summary - Adds a divan benchmark `bitpack_compare` in `vortex-fastlanes` that compares an `Operator::Eq` / `Operator::Lt` against an out-of-range constant on a `BitPackedData` array vs. an explicit "decompress, then Arrow compare" baseline that materialises the unpacked `PrimitiveArray` first. - The constant is chosen as `1 << BW`, just past the packable range, so a future compare-constant kernel can recognise it and short-circuit. Today both arms decompress; this PR establishes a baseline for that follow-up to land against. - Grid sized for fast runs: `len ∈ {1024, 65536}`, `bit_width ∈ {4, 16}`, Eq + Lt. The follow-up optimization (out-of-range fast path on `BitPacked`, plus the `bitpack_constant` analytical encoder) is in #PR2-PLACEHOLDER, stacked on this branch. Splitting the bench out lets the speedup PR show concrete numbers against this measured baseline. ## Test plan - [x] `cargo check -p vortex-fastlanes --benches` - [ ] `cargo bench -p vortex-fastlanes --bench bitpack_compare` records the slow baseline numbers prior to the follow-up landing 🤖 Generated with [Claude Code](https://claude.com/claude-code) Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
Skip validity dispatch for non-nullable arrays (#7748) Two small optimizations: - Skip the `is_invalid` vtable dispatch when `dtype` is non-nullable - Demote the dtype-equality post-check to debug assertion. It's an encoding-correctness invariant, not runtime input validation Signed-off-by: Baris Palaska <barispalaska@gmail.com>
Split up the monster `ci.yml` workflow (#7472) ## Summary `ci.yml` is reaching 1000 lines of YAML, which no one wants. This PR splits into 5 files: 1. ci.yml - now only contains linters and test runs 2. codspeed.yml - just codspeed benchmarks 3. cuda.yaml - all the cuda incantations 4. publish-dry-runs.yaml - all the dark magic that we have to try and make sure we can actually publish 5. rust-instrumented.yaml - rust-specific jobs that require nightly and some extra instrumentations (coverage, miri, etc) The real magic here is that there are no changes to the job names, so all rules still apply! This also makes github retries much easier, as github only allows retries after the whole workflow finishes. --------- Signed-off-by: Adam Gutglick <adam@spiraldb.com>
Clean up `AGENTS.md` (#7366) ## Summary Fixes the instruction to run `./scripts/public-api.sh` instead of `cargo xtask` just for compat (first commit), and also formats everything. ## Testing N/A --------- Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
Python benchmarks script and re-organize dev dependencies (#7192) ## Summary 1. Add a script to run the python benchmarks which are basically a secret right now. 2. Re-organize dev-dependencies, now that UV does support transitive dev dependencies --------- Signed-off-by: Adam Gutglick <adam@spiraldb.com> Co-authored-by: Dan King <dan@spiraldb.com>
fix(python): cap substrait-python before 0.29.0 (#7161) ## Summary I made a mistake in #7153, for details about the mistake, see this comment: #7153 (comment) This PR correctly fixes the `substrait-python` dependency problem. ## Testing - In a clean Python 3.12 virtual environment, `pip install vortex-data==0.64.0` resolved to `substrait==0.29.0`, `substrait-protobuf==0.85.0`, and `substrait-extensions==0.85.0`, and `import vortex` failed. - In a clean Python 3.12 virtual environment, installing `vortex-data` from this PR branch resolved to `substrait==0.28.0`, `substrait-protobuf==0.79.0`, and `substrait-extensions==0.79.0`, and `import vortex` succeeded. Signed-off-by: Xu Che <chrisxuche@gmail.com>
chore(deps): lock file maintenance (#6972) This PR contains the following updates: | Update | Change | |---|---| | lockFileMaintenance | All locks refreshed | 🔧 This Pull Request updates lock files to use the latest dependency versions. --- ### Configuration 📅 **Schedule**: Branch creation - "before 4am on monday" (UTC), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Enabled. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 👻 **Immortal**: This PR will be recreated if closed unmerged. Get [config help](https://redirect.github.com/renovatebot/renovate/discussions) if that's undesired. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR was generated by [Mend Renovate](https://mend.io/renovate/). View the [repository job log](https://developer.mend.io/github/vortex-data/vortex). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiI0My42Ni40IiwidXBkYXRlZEluVmVyIjoiNDMuNjYuNCIsInRhcmdldEJyYW5jaCI6ImRldmVsb3AiLCJsYWJlbHMiOlsiY2hhbmdlbG9nL2Nob3JlIl19--> Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
PreviousNext
