Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bump the all-dependencies group with 8 updates #112

Merged
merged 1 commit into from
Jan 23, 2025

Conversation

dependabot[bot]
Copy link
Contributor

@dependabot dependabot bot commented on behalf of github Jan 23, 2025

Bumps the all-dependencies group with 8 updates:

Package From To
pyspark 3.5.3 3.5.4
dynaconf 3.2.6 3.2.7
delta-spark 3.2.1 3.3.0
soda-core-spark-df 3.4.2 3.4.4
ruff 0.8.1 0.9.2
mypy 1.13.0 1.14.1
pytest 8.3.3 8.3.4
pre-commit 4.0.1 4.1.0

Updates pyspark from 3.5.3 to 3.5.4

Commits
  • a6f220d Preparing Spark release v3.5.4-rc3
  • b0a7d4d [SPARK-50587][INFRA][3.5] Remove unsupported curl option `--retry-all-error...
  • f7c48fe Revert "[SPARK-50430][CORE] Use the standard Properties.clone instead of manu...
  • 0fbe292 Revert "[SPARK-50430][CORE][FOLLOW-UP] Keep the logic of manual putting key a...
  • 8168ea8 [SPARK-50430][CORE][FOLLOW-UP] Keep the logic of manual putting key and value...
  • a764524 Preparing development version 3.5.5-SNAPSHOT
  • 91af6f9 Preparing Spark release v3.5.4-rc2
  • 92e650c [SPARK-50545][CORE][SQL][3.5] AccessControlException should be thrown even ...
  • e97580a [SPARK-50087][SQL][3.5] Robust handling of boolean expressions in CASE WHEN f...
  • bb953f9 [SPARK-50510][CONNECT][3.5] Fix sporadic ReattachableExecuteSuite failure
  • Additional commits viewable in compare view

Updates dynaconf from 3.2.6 to 3.2.7

Release notes

Sourced from dynaconf's releases.

3.2.7

Async Release for Integration with Ansible Automation Platform

What's Changed

Bug Fixes

  • lazy validator's default value would evaluate early (#1198). By Pedro Brochado.
  • Fixed an error that would raise when using get_history() with lazy values (#1184) (#1185). By Pedro Brochado.
  • Fixed Redis loader when ENV prefix is None.
  • Populate object method now takes internal attribute to filter out internal variables.
  • On CLI json.dumps defaults to repr for types that cannot be serialized.
  • Added an identifier to validator calls of set method
  • Fix django app discovery using DJANGO_SETTINGS_MODULE variable

Features

  • Added @insert token to call list.insert
  • Allow env loader to load from multiple prefixes
  • Allow multiple composable current environments
  • Track more data on load_file method
  • Added --json to dynaconf list CLI

Full Changelog: dynaconf/dynaconf@3.2.6...3.2.7

Changelog

Sourced from dynaconf's changelog.

3.2.7 - 2025-01-21

Bug Fixes

  • lazy validator's default value would evaluate early (#1198). By Pedro Brochado.
  • Fixed an error that would raise when using get_history() with lazy values (#1184) (#1185). By Pedro Brochado.
  • Fixed Redis loader when ENV prefix is None.
  • Populate object method now takes internal attribute to filter out internal variables.
  • On CLI json.dumps defaults to repr for types that cannot be serialized.
  • Added an identifier to validator calls of set method
  • Fix django app discovery using DJANGO_SETTINGS_MODULE variable

Features

  • Added @insert token to call list.insert
  • Allow env loader to load from multiple prefixes
  • Allow multiple composable current environments
  • Track more data on load_file method
  • Added --json to dynaconf list CLI
Commits
  • 9e37b7d Release version 3.2.7
  • 188ca5e AWX Integration fixes (#1204)
  • 43f5393 fix: lazy validator's default value would evalute early (#1198)
  • 545ad29 fix(inspect): Fixed an error that would raise when using get_history() with l...
  • dce176d Bump to version 3.2.7-dev0
  • See full diff in compare view

Updates delta-spark from 3.2.1 to 3.3.0

Release notes

Sourced from delta-spark's releases.

Delta Lake 3.3.0

We are excited to announce the release of Delta Lake 3.3.0! This release includes several exciting new features.

Highlights

Details by each component.

Delta Spark

Delta Spark 3.3.0 is built on Apache Spark™ 3.5.3. Similarly to Apache Spark, we have released Maven artifacts for both Scala 2.12 and Scala 2.13.

The key features of this release are:

  • Support for Identity Column: Delta Lake identity columns are a type of generated column that automatically assigns unique values to each record inserted into a table. Users do not need to explicitly provide values for these columns during data insertion. They offer a straightforward and efficient mechanism to generate unique keys for table rows, combining ease of use with high performance. See the documentation for more information.
  • Support VACUUM LITE to deliver faster VACUUM for periodically run VACUUM commands. When running VACUUM in LITE mode, instead of finding all files in the table directory, VACUUM LITE uses the Delta transaction log to identify and remove files no longer referenced by any table versions within the retention duration.
  • Support for Row Tracking Backfill:Row Tracking feature can now be used on existing Delta Lake tables to track row-level lineage in Delta Spark, previously it was only possible for new tables. Users can now use ALTER TABLE table_name SET TBLPROPERTIES (delta.enableRowTracking = true) syntax to alter an existing table to enable Row Tracking. When enabled, users can identify rows across multiple versions of the table and can access this tracking information using the two metadata fields _metadata.row_id and _metadata.row_commit_version. Refer to the documentation on Row Tracking for more information and examples.
  • Delta Lake now generates version checksums for each table commit, providing stronger consistency guarantees and improved debugging capabilities. It tracks detailed table metrics including file counts, table size, data distribution histograms, etc. This enables automatic detection of potential state inconsistencies and helps maintain table integrity in distributed environments. The state validation is performed on every checkpoint. The Checksum is also used to bypass the initial Spark query that retrieves the Protocol and Metadata actions, resulting in a decreased snapshot initialization latency.
  • Liquid clustering updates:
    • Support OPTIMIZE FULL to fully recluster a Liquid table. This command optimizes all records in a table that uses liquid clustering, including data that might have previously been clustered.
    • Support enabling liquid clustering on an existing unpartitioned Delta table using ALTER TABLE ). Previously, liquid clustering could only be enabled upon table creation.
    • Support creating clustered table from an external location
  • The In-Commit Timestamp table feature is no longer in preview When enabled, this feature persists monotonically increasing timestamps within Delta commits, ensuring they are not affected by file operations. With this, time travel queries yield consistent results, even if the table directory is relocated. This feature was available as a preview feature in Delta 3.2 and is now generally available in Delta 3.3. See the documentation for more information.

Other notable changes include:

  • Protocol upgrade/downgrade improvements
    • Support dropping table features for columnMapping, vacuumProtocolCheck, and checkConstraints.
    • Improve table protocol transitions to simplify the CUJ when altering the table protocol.
    • Support protocol version downgrades when the existing table features exist in the lower protocol version.
    • Update protocol upgrades behavior such that when enabling a legacy feature via a table property (e.g. setting delta.enableChangeDataFeed=true) the protocol is upgraded to (1,7) and only the legacy feature is enabled. Previously the minimum protocol version would be selected and all preceding legacy features enabled.
    • Support enabling a table feature on a table using the Python DeltaTable API with deltaTable.addFeatureSupport(...).
  • Type-widening improvements
    • Support automatic type widening in Delta Sink when type widening is enabled on the table and schema evolution is enabled on the sink.
    • Support type widening on nested fields when other nested fields in the same struct are referenced by check constraints or generated column expressions.
    • Fix type-widening operation validation for map, array or struct columns used in generated column expressions or check constraints.
    • Fix to directly read the file schema from the parquet footers when identifying the files to be rewritten when dropping the type widening table feature.
    • Fix using type widening on a table containing a char/varchar column.
  • Liquid clustering improvements
    • Fix liquid clustering to automatically fall back to Z-order clustering when clustering on a single column. Previously, any attempts to optimize the table would fail.

... (truncated)

Commits
  • f405c3f Setting version to 3.3.0
  • d5edfab [DOC][3.3] Doc changes for InCommitTimestamps (#3979)
  • 2e186b4 [3.3][Docs] Update docs for 3.3 release (#3992)
  • 9b15a8f [Spark][3.3] Make Identity Column High Water Mark updates consistent (#3990)
  • fb3fd94 Update Delta uniform documentation to include ALTER enabling - 3.3 (#3963)
  • 6177288 [3.3][Kernel] Disable setting columnMapping/icebergCompatV2 for 3.3.0 (#3980)
  • 2c210de [3.3] Add Documentation for Vacuum LITE (#3987)
  • 8c18520 [Spark][3.3] Fix auto-conflict handling logic in Optimize to handle DVs (#3982)
  • 9c967b9 [Kernel] [3.3] Remove Coordinated Commits from public API (#3938) (#3973)
  • f29aea0 [PROTOCOL][Version Checksum][3.3] Remove references to Java-specific Int.MaxV...
  • Additional commits viewable in compare view

Updates soda-core-spark-df from 3.4.2 to 3.4.4

Updates ruff from 0.8.1 to 0.9.2

Release notes

Sourced from ruff's releases.

0.9.2

Release Notes

Preview features

  • [airflow] Fix typo "security_managr" to "security_manager" (AIR303) (#15463)
  • [airflow] extend and fix AIR302 rules (#15525)
  • [fastapi] Handle parameters with Depends correctly (FAST003) (#15364)
  • [flake8-pytest-style] Implement pytest.warns diagnostics (PT029, PT030, PT031) (#15444)
  • [flake8-pytest-style] Test function parameters with default arguments (PT028) (#15449)
  • [flake8-type-checking] Avoid false positives for | in TC008 (#15201)

Rule changes

  • [flake8-todos] Allow VSCode GitHub PR extension style links in missing-todo-link (TD003) (#15519)
  • [pyflakes] Show syntax error message for F722 (#15523)

Formatter

  • Fix curly bracket spacing around f-string expressions containing curly braces (#15471)
  • Fix joining of f-strings with different quotes when using quote style Preserve (#15524)

Server

  • Avoid indexing the same workspace multiple times (#15495)
  • Display context for ruff.configuration errors (#15452)

Configuration

  • Remove flatten to improve deserialization error messages (#15414)

Bug fixes

  • Parse triple-quoted string annotations as if parenthesized (#15387)
  • [fastapi] Update Annotated fixes (FAST002) (#15462)
  • [flake8-bandit] Check for builtins instead of builtin (S102, PTH123) (#15443)
  • [flake8-pathlib] Fix --select for os-path-dirname (PTH120) (#15446)
  • [ruff] Fix false positive on global keyword (RUF052) (#15235)

Contributors

... (truncated)

Changelog

Sourced from ruff's changelog.

0.9.2

Preview features

  • [airflow] Fix typo "security_managr" to "security_manager" (AIR303) (#15463)
  • [airflow] extend and fix AIR302 rules (#15525)
  • [fastapi] Handle parameters with Depends correctly (FAST003) (#15364)
  • [flake8-pytest-style] Implement pytest.warns diagnostics (PT029, PT030, PT031) (#15444)
  • [flake8-pytest-style] Test function parameters with default arguments (PT028) (#15449)
  • [flake8-type-checking] Avoid false positives for | in TC008 (#15201)

Rule changes

  • [flake8-todos] Allow VSCode GitHub PR extension style links in missing-todo-link (TD003) (#15519)
  • [pyflakes] Show syntax error message for F722 (#15523)

Formatter

  • Fix curly bracket spacing around f-string expressions containing curly braces (#15471)
  • Fix joining of f-strings with different quotes when using quote style Preserve (#15524)

Server

  • Avoid indexing the same workspace multiple times (#15495)
  • Display context for ruff.configuration errors (#15452)

Configuration

  • Remove flatten to improve deserialization error messages (#15414)

Bug fixes

  • Parse triple-quoted string annotations as if parenthesized (#15387)
  • [fastapi] Update Annotated fixes (FAST002) (#15462)
  • [flake8-bandit] Check for builtins instead of builtin (S102, PTH123) (#15443)
  • [flake8-pathlib] Fix --select for os-path-dirname (PTH120) (#15446)
  • [ruff] Fix false positive on global keyword (RUF052) (#15235)

0.9.1

Preview features

  • [pycodestyle] Run too-many-newlines-at-end-of-file on each cell in notebooks (W391) (#15308)
  • [ruff] Omit diagnostic for shadowed private function parameters in used-dummy-variable (RUF052) (#15376)

Rule changes

  • [flake8-bugbear] Improve assert-raises-exception message (B017) (#15389)

Formatter

... (truncated)

Commits
  • 0a39348 Include build binaries
  • 027f800 Comment out non-npm-publish jobs
  • 425870d Upload npm publish logs when failed
  • c20255a Bump version to 0.9.2 (#15529)
  • 4203658 Fix joining of f-strings with different quotes when using quote style `Preser...
  • fc9dd63 [airflow] extend and fix AIR302 rules (#15525)
  • 79e52c7 [pyflakes] Show syntax error message for F722 (#15523)
  • cf4ab7c Parse triple quoted string annotations as if parenthesized (#15387)
  • d2656e8 [flake8-todos] Allow VSCode GitHub PR extension style links in `missing-tod...
  • c53ee60 Typeshed-sync workflow: add appropriate labels, link directly to failing run ...
  • Additional commits viewable in compare view

Updates mypy from 1.13.0 to 1.14.1

Changelog

Sourced from mypy's changelog.

Mypy Release Notes

Next release

Performance improvements

Mypy may be 5-30% faster. This improvement comes largely from tuning the performance of the garbage collector.

Contributed by Jukka Lehtosalo (PR 18306).

Mypyc accelerated mypy wheels for aarch64

Mypy can compile itself to C extension modules using mypyc. This makes mypy 3-5x faster than if mypy is interpreted with pure Python. We now build and upload mypyc accelerated mypy wheels for manylinux_aarch64 to PyPI, making it easy for users on such platforms to realise this speedup.

Contributed by Christian Bundy and Marc Mueller (PR mypy_mypyc-wheels#76, PR mypy_mypyc-wheels#89).

--strict-bytes

By default, mypy treats an annotation of bytes as permitting bytearray and memoryview. PEP 688 specified the removal of this special case. Use this flag to disable this behavior. --strict-bytes will be enabled by default in mypy 2.0.

Contributed by Ali Hamdan (PR 18137) and Shantanu Jain (PR 13952).

Improvements to reachability analysis and partial type handling in loops

This change results in mypy better modelling control flow within loops and hence detecting several issues it previously did not detect. In some cases, this change may require use of an additional explicit annotation of a variable.

Contributed by Christoph Tyralla (PR 18180, PR).

(Speaking of partial types, another reminder that mypy plans on enabling --local-partial-types by default in mypy 2.0).

Better discovery of configuration files

Mypy will now walk up the filesystem (up until a repository or file system root) to discover configuration files. See the mypy configuration file documentation for more details.

... (truncated)

Commits

Updates pytest from 8.3.3 to 8.3.4

Release notes

Sourced from pytest's releases.

8.3.4

pytest 8.3.4 (2024-12-01)

Bug fixes

  • #12592: Fixed KeyError{.interpreted-text role="class"} crash when using --import-mode=importlib in a directory layout where a directory contains a child directory with the same name.

  • #12818: Assertion rewriting now preserves the source ranges of the original instructions, making it play well with tools that deal with the AST, like executing.

  • #12849: ANSI escape codes for colored output now handled correctly in pytest.fail{.interpreted-text role="func"} with [pytrace=False]{.title-ref}.

  • #9353: pytest.approx{.interpreted-text role="func"} now uses strict equality when given booleans.

Improved documentation

  • #10558: Fix ambiguous docstring of pytest.Config.getoption{.interpreted-text role="func"}.

  • #10829: Improve documentation on the current handling of the --basetemp option and its lack of retention functionality (temporary directory location and retention{.interpreted-text role="ref"}).

  • #12866: Improved cross-references concerning the recwarn{.interpreted-text role="fixture"} fixture.

  • #12966: Clarify filterwarnings{.interpreted-text role="ref"} docs on filter precedence/order when using multiple @pytest.mark.filterwarnings <pytest.mark.filterwarnings ref>{.interpreted-text role="ref"} marks.

Contributor-facing changes

  • #12497: Fixed two failing pdb-related tests on Python 3.13.
Commits
  • 53f8b4e Update pypa/gh-action-pypi-publish to v1.12.2
  • 98dff36 Prepare release version 8.3.4
  • 1b474e2 approx: use exact comparison for bool (#13013)
  • b541721 docs: Fix wrong statement about sys.modules with importlib import mode (#1298...
  • 16cb87b pytest.fail: fix ANSI escape codes for colored output (#12959) (#12990)
  • be6bc81 Issue #12966 Clarify filterwarnings docs on precedence when using multiple ma...
  • 7aeb72b Improve docs on basetemp and retention (#12912) (#12928)
  • c875841 Merge pull request #12917 from pytest-dev/patchback/backports/8.3.x/ded1f44e5...
  • 6502816 Merge pull request #12913 from jakkdl/dontfailonbadpath
  • 52135b0 Merge pull request #12885 from The-Compiler/pdb-py311 (#12887)
  • Additional commits viewable in compare view

Updates pre-commit from 4.0.1 to 4.1.0

Release notes

Sourced from pre-commit's releases.

pre-commit v4.1.0

Features

Fixes

Changelog

Sourced from pre-commit's changelog.

4.1.0 - 2025-01-20

Features

Fixes

Commits
  • b152e92 v4.1.0
  • c3125a4 Merge pull request #3389 from lorenzwalthert/dev-always-unset-renv
  • c2c061c fix: ensure env patch is applied for vanilla emulation
  • cd429db Merge pull request #3382 from pre-commit/pre-commit-ci-update-config
  • 9b9f8e2 [pre-commit.ci] pre-commit autoupdate
  • 86300a4 Merge pull request #3376 from pre-commit/r-gone
  • 77edad8 install r on ubuntu runners
  • 18b3939 Merge pull request #3375 from pre-commit/dotnet-tests-ubuntu-latest
  • 31cb945 Merge pull request #3374 from pre-commit/docker-image-tests-ubuntu-22-not-pre...
  • 28c3d81 update .net tests to use .net 8
  • Additional commits viewable in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

  • @dependabot rebase will rebase this PR
  • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
  • @dependabot merge will merge this PR after your CI passes on it
  • @dependabot squash and merge will squash and merge this PR after your CI passes on it
  • @dependabot cancel merge will cancel a previously requested merge and block automerging
  • @dependabot reopen will reopen this PR if it is closed
  • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
  • @dependabot show <dependency name> ignore conditions will show all of the ignore conditions of the specified dependency
  • @dependabot ignore <dependency name> major version will close this group update PR and stop Dependabot creating any more for the specific dependency's major version (unless you unignore this specific dependency's major version or upgrade to it yourself)
  • @dependabot ignore <dependency name> minor version will close this group update PR and stop Dependabot creating any more for the specific dependency's minor version (unless you unignore this specific dependency's minor version or upgrade to it yourself)
  • @dependabot ignore <dependency name> will close this group update PR and stop Dependabot creating any more for the specific dependency (unless you unignore this specific dependency or upgrade to it yourself)
  • @dependabot unignore <dependency name> will remove all of the ignore conditions of the specified dependency
  • @dependabot unignore <dependency name> <ignore condition> will remove the ignore condition of the specified dependency and ignore conditions

Bumps the all-dependencies group with 8 updates:

| Package | From | To |
| --- | --- | --- |
| [pyspark](https://github.com/apache/spark) | `3.5.3` | `3.5.4` |
| [dynaconf](https://github.com/dynaconf/dynaconf) | `3.2.6` | `3.2.7` |
| [delta-spark](https://github.com/delta-io/delta) | `3.2.1` | `3.3.0` |
| soda-core-spark-df | `3.4.2` | `3.4.4` |
| [ruff](https://github.com/astral-sh/ruff) | `0.8.1` | `0.9.2` |
| [mypy](https://github.com/python/mypy) | `1.13.0` | `1.14.1` |
| [pytest](https://github.com/pytest-dev/pytest) | `8.3.3` | `8.3.4` |
| [pre-commit](https://github.com/pre-commit/pre-commit) | `4.0.1` | `4.1.0` |


Updates `pyspark` from 3.5.3 to 3.5.4
- [Commits](apache/spark@v3.5.3...v3.5.4)

Updates `dynaconf` from 3.2.6 to 3.2.7
- [Release notes](https://github.com/dynaconf/dynaconf/releases)
- [Changelog](https://github.com/dynaconf/dynaconf/blob/3.2.7/CHANGELOG.md)
- [Commits](dynaconf/dynaconf@3.2.6...3.2.7)

Updates `delta-spark` from 3.2.1 to 3.3.0
- [Release notes](https://github.com/delta-io/delta/releases)
- [Commits](delta-io/delta@v3.2.1...v3.3.0)

Updates `soda-core-spark-df` from 3.4.2 to 3.4.4

Updates `ruff` from 0.8.1 to 0.9.2
- [Release notes](https://github.com/astral-sh/ruff/releases)
- [Changelog](https://github.com/astral-sh/ruff/blob/main/CHANGELOG.md)
- [Commits](astral-sh/ruff@0.8.1...0.9.2)

Updates `mypy` from 1.13.0 to 1.14.1
- [Changelog](https://github.com/python/mypy/blob/master/CHANGELOG.md)
- [Commits](python/mypy@v1.13.0...v1.14.1)

Updates `pytest` from 8.3.3 to 8.3.4
- [Release notes](https://github.com/pytest-dev/pytest/releases)
- [Changelog](https://github.com/pytest-dev/pytest/blob/main/CHANGELOG.rst)
- [Commits](pytest-dev/pytest@8.3.3...8.3.4)

Updates `pre-commit` from 4.0.1 to 4.1.0
- [Release notes](https://github.com/pre-commit/pre-commit/releases)
- [Changelog](https://github.com/pre-commit/pre-commit/blob/main/CHANGELOG.md)
- [Commits](pre-commit/pre-commit@v4.0.1...v4.1.0)

---
updated-dependencies:
- dependency-name: pyspark
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: all-dependencies
- dependency-name: dynaconf
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: all-dependencies
- dependency-name: delta-spark
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: all-dependencies
- dependency-name: soda-core-spark-df
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: all-dependencies
- dependency-name: ruff
  dependency-type: direct:development
  update-type: version-update:semver-minor
  dependency-group: all-dependencies
- dependency-name: mypy
  dependency-type: direct:development
  update-type: version-update:semver-minor
  dependency-group: all-dependencies
- dependency-name: pytest
  dependency-type: direct:development
  update-type: version-update:semver-patch
  dependency-group: all-dependencies
- dependency-name: pre-commit
  dependency-type: direct:development
  update-type: version-update:semver-minor
  dependency-group: all-dependencies
...

Signed-off-by: dependabot[bot] <[email protected]>
@dependabot dependabot bot added dependencies Pull requests that update a dependency file python Pull requests that update Python code labels Jan 23, 2025
@guidok91 guidok91 merged commit e40bd28 into master Jan 23, 2025
2 checks passed
@guidok91 guidok91 deleted the dependabot/pip/all-dependencies-785f43b54a branch January 23, 2025 15:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
dependencies Pull requests that update a dependency file python Pull requests that update Python code
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant