Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow MapieRegressor to use any split strategy #386

Merged
merged 15 commits into from
Dec 20, 2023

Conversation

thibaultcordier
Copy link
Collaborator

@thibaultcordier thibaultcordier commented Dec 13, 2023

Description

Split-CP is only available with "split" keyword or ShuffleSplit class of scikit-learn.

This PR proposes to use more compatible split BaseCrossValidator/BaseShuffleSplit.

In practice, we need to use the predictions aggregation for cross-CPs and not for split-CPs. All you need to do is find out the number of splits using the get_n_splits method to know whether it's a split or a cross.

Fixes #385

Type of change

  • Breaking change (fix or feature that would cause existing functionality to not work as expected)

How Has This Been Tested?

Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce. Please also list any relevant details for your test configuration

  • Add test for the function check_no_agg_cv
  • Add new split method with MapieRegressor
  • Check that the other tests still work

Checklist

  • I have read the contributing guidelines
  • I have updated the HISTORY.rst and AUTHORS.rst files
  • Linting passes successfully : make lint
  • Typing passes successfully : make type-check
  • Unit tests pass successfully : make tests
  • Coverage is 100% : make coverage
  • Documentation builds successfully : make doc

@thibaultcordier
Copy link
Collaborator Author

thibaultcordier commented Dec 13, 2023

This PR also tidies things up a bit: the tests specific to CQR have been moved to the dedicated file (from test_utils.py to test_quantile_regression.py).

@thibaultcordier thibaultcordier marked this pull request as ready for review December 13, 2023 12:17
@codecov-commenter
Copy link

codecov-commenter commented Dec 13, 2023

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (614293e) 100.00% compared to head (1fa8b7c) 100.00%.
Report is 34 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##            master      #386    +/-   ##
==========================================
  Coverage   100.00%   100.00%            
==========================================
  Files           39        39            
  Lines         4616      4633    +17     
  Branches       487       762   +275     
==========================================
+ Hits          4616      4633    +17     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

mapie/utils.py Outdated Show resolved Hide resolved
Copy link
Collaborator

@LacombeLouis LacombeLouis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good PR! Thank you @thibaultcordier 😀
I am a little worried about adding another fit_attributes for custom algorithms. What do you think?

mapie/estimator/estimator.py Show resolved Hide resolved
mapie/regression/time_series_regression.py Outdated Show resolved Hide resolved
mapie/estimator/estimator.py Outdated Show resolved Hide resolved
mapie/utils.py Outdated Show resolved Hide resolved
Copy link
Collaborator

@vincentblot28 vincentblot28 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Everything is fine, let's merge !

@thibaultcordier thibaultcordier merged commit 3a93f31 into master Dec 20, 2023
8 checks passed
@Valentin-Laurent Valentin-Laurent deleted the 385-regressor-any-split-strategy branch November 12, 2024 13:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Allow MapieRegressor to use any split strategy
4 participants