Skip to content

Commit

Permalink
DOC: change words conformity and conf to conformalization/conformalize
Browse files Browse the repository at this point in the history
  • Loading branch information
Valentin-Laurent committed Jan 21, 2025
1 parent 01b50bf commit 2df292b
Show file tree
Hide file tree
Showing 6 changed files with 35 additions and 31 deletions.
6 changes: 3 additions & 3 deletions README.rst
Original file line number Diff line number Diff line change
Expand Up @@ -115,16 +115,16 @@ As **MAPIE** is compatible with the standard scikit-learn API, you can see that
from mapie_v1.regression import SplitConformalRegressor

X, y = make_regression(n_samples=500, n_features=1, noise=20, random_state=59)
X_train_conf, X_test, y_train_conf, y_test = train_test_split(X, y, test_size=0.5)
X_train, X_conf, y_train, y_conf = train_test_split(X_train_conf, y_train_conf, test_size=0.5)
X_train_conformalize, X_test, y_train_conformalize, y_test = train_test_split(X, y, test_size=0.5)
X_train, X_conformalize, y_train, y_conformalize = train_test_split(X_train_conformalize, y_train_conformalize, test_size=0.5)

regressor = LinearRegression()
mapie_regressor = SplitConformalRegressor(
regressor,
confidence_level=[0.95, 0.68],
)
mapie_regressor.fit(X_train, y_train)
mapie_regressor.conformalize(X_conf, y_conf)
mapie_regressor.conformalize(X_conformalize, y_conformalize)

y_pred = mapie_regressor.predict(X_test)
y_pred_intervals = mapie_regressor.predict_set(X_test)
Expand Down
6 changes: 3 additions & 3 deletions doc/quick_start.rst
Original file line number Diff line number Diff line change
Expand Up @@ -45,8 +45,8 @@ Here, we generate one-dimensional noisy data that we fit with a linear model.

regressor = LinearRegression()
X, y = make_regression(n_samples=500, n_features=1, noise=20, random_state=59)
X_train_conf, X_test, y_train_conf, y_test = train_test_split(X, y, test_size=0.5)
X_train, X_conf, y_train, y_conf = train_test_split(X_train_conf, y_train_conf,
X_train_conformalize, X_test, y_train_conformalize, y_test = train_test_split(X, y, test_size=0.5)
X_train, X_conformalize, y_train, y_conformalize = train_test_split(X_train_conformalize, y_train_conformalize,
test_size=0.5)

# We follow a sequential ``fit``, ``conformalize``, and ``predict`` process.
Expand All @@ -60,7 +60,7 @@ Here, we generate one-dimensional noisy data that we fit with a linear model.
confidence_level=[0.95, 0.68],
)
mapie_regressor.fit(X_train, y_train)
mapie_regressor.conformalize(X_conf, y_conf)
mapie_regressor.conformalize(X_conformalize, y_conformalize)

y_pred = mapie_regressor.predict(X_test)
y_pred_intervals = mapie_regressor.predict_set(X_test)
Expand Down
2 changes: 1 addition & 1 deletion doc/split_cross_conformal.rst
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
################################################################
The calibration (or "conformity") set
The calibration (or "conformalization") set
################################################################

**MAPIE** is based on two types of techniques:
Expand Down
4 changes: 2 additions & 2 deletions doc/v1_migration_guide.rst
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,7 @@ In v1.0: MAPIE separates between training and calibration. We decided to name th
- Additional fitting parameters, like ``sample_weight``, should be included in ``fit_params``, keeping this method focused on training alone.

- ``.conformalize()`` method:
- This new method performs conformalization after fitting, using separate conformity data ``(X_conf, y_conf)``.
- This new method performs conformalization after fitting, using separate conformalization data ``(X_conformalize, y_conformalize)``.
- ``predict_params`` can be passed here, allowing independent control over conformalization and prediction stages.

Step 4: Making predictions (``predict`` and ``predict_set`` methods)
Expand Down Expand Up @@ -159,7 +159,7 @@ Example 1: Split Conformal Prediction

Description
############
Split conformal prediction is a widely used method for generating prediction intervals, it splits the data into training, conformity, and test sets. The model is trained on the training set, calibrated on the conformity set, and then used to make predictions on the test set. In `MAPIE v1`, the `SplitConformalRegressor` replaces the older `MapieRegressor` with a more modular design and simplified API.
Split conformal prediction is a widely used method for generating prediction intervals, it splits the data into training, conformalization, and test sets. The model is trained on the training set, calibrated on the conformity set, and then used to make predictions on the test set. In `MAPIE v1`, the `SplitConformalRegressor` replaces the older `MapieRegressor` with a more modular design and simplified API.

MAPIE v0.9 Code
###############
Expand Down
8 changes: 4 additions & 4 deletions mapie_v1/classification.py
Original file line number Diff line number Diff line change
Expand Up @@ -35,8 +35,8 @@ def fit(

def conformalize(
self,
X_conf: ArrayLike,
y_conf: ArrayLike,
X_conformalize: ArrayLike,
y_conformalize: ArrayLike,
predict_params: Optional[dict] = None,
) -> Self:
return self
Expand Down Expand Up @@ -93,8 +93,8 @@ def fit(

def conformalize(
self,
X_conf: ArrayLike,
y_conf: ArrayLike,
X_conformalize: ArrayLike,
y_conformalize: ArrayLike,
predict_params: Optional[dict] = None
) -> Self:
return self
Expand Down
40 changes: 22 additions & 18 deletions mapie_v1/regression.py
Original file line number Diff line number Diff line change
Expand Up @@ -65,7 +65,7 @@ class SplitConformalRegressor:
Notes
-----
This implementation currently uses a ShuffleSplit cross-validation scheme
for splitting the conformity set. Future implementations may allow the use
for splitting the conformalization set. Future implementations may allow the use
of groups.
Examples
Expand All @@ -77,12 +77,14 @@ class SplitConformalRegressor:
>>> X, y = make_regression(n_samples=500, n_features=2, noise=1.0)
>>> X_train, X_conf_test, y_train, y_conf_test = train_test_split(X, y)
>>> X_conf, X_test, y_conf, y_test = train_test_split(X_conf_test, y_conf_test)
>>> X_conformalize, X_test, y_conformalize, y_test = train_test_split(
... X_conf_test, y_conf_test
... )
>>> mapie_regressor = SplitConformalRegressor(
... estimator=Ridge(),
... confidence_level=0.95
... ).fit(X_train, y_train).conformalize(X_conf, y_conf)
... ).fit(X_train, y_train).conformalize(X_conformalize, y_conformalize)
>>> prediction_points = mapie_regressor.predict(X_test)
>>> prediction_intervals = mapie_regressor.predict_set(X_test)
Expand Down Expand Up @@ -155,8 +157,8 @@ def fit(

def conformalize(
self,
X_conf: ArrayLike,
y_conf: ArrayLike,
X_conformalize: ArrayLike,
y_conformalize: ArrayLike,
predict_params: Optional[dict] = None,
) -> Self:
"""
Expand All @@ -165,10 +167,10 @@ def conformalize(
Parameters
----------
X_conf : ArrayLike
X_conformalize : ArrayLike
Features for the conformity set.
y_conf : ArrayLike
y_conformalize : ArrayLike
Target values for the conformity set.
predict_params : Optional[dict], default=None
Expand All @@ -181,8 +183,8 @@ def conformalize(
The conformalized SplitConformalRegressor instance.
"""
predict_params = {} if predict_params is None else predict_params
self._mapie_regressor.fit(X_conf,
y_conf,
self._mapie_regressor.fit(X_conformalize,
y_conformalize,
predict_params=predict_params)

return self
Expand Down Expand Up @@ -907,7 +909,7 @@ class ConformalizedQuantileRegressor:
Trains the base quantile regression estimator on the provided data.
Not applicable if `prefit=True`.
conformalize(X_conf, y_conf, predict_params=None) -> Self
conformalize(X_conformalize, y_conformalize, predict_params=None) -> Self
Calibrates the model on provided data, adjusting the prediction
intervals to achieve the specified confidence levels.
Expand Down Expand Up @@ -938,12 +940,14 @@ class ConformalizedQuantileRegressor:
>>> X, y = make_regression(n_samples=500, n_features=2, noise=1.0)
>>> X_train, X_conf_test, y_train, y_conf_test = train_test_split(X, y)
>>> X_conf, X_test, y_conf, y_test = train_test_split(X_conf_test, y_conf_test)
>>> X_conformalize, X_test, y_conformalize, y_test = train_test_split(
... X_conf_test, y_conf_test
... )
>>> mapie_regressor = ConformalizedQuantileRegressor(
... estimator=QuantileRegressor(),
... confidence_level=0.95,
... ).fit(X_train, y_train).conformalize(X_conf, y_conf)
... ).fit(X_train, y_train).conformalize(X_conformalize, y_conformalize)
>>> prediction_points = mapie_regressor.predict(X_test)
>>> prediction_intervals = mapie_regressor.predict_set(X_test)
Expand Down Expand Up @@ -1026,8 +1030,8 @@ def fit(

def conformalize(
self,
X_conf: ArrayLike,
y_conf: ArrayLike,
X_conformalize: ArrayLike,
y_conformalize: ArrayLike,
predict_params: Optional[dict] = None,
) -> Self:
"""
Expand All @@ -1038,10 +1042,10 @@ def conformalize(
Parameters
----------
X_conf : ArrayLike
X_conformalize : ArrayLike
Features for the calibration (conformity) data.
y_conf : ArrayLike
y_conformalize : ArrayLike
Target values for the calibration (conformity) data.
predict_params : Optional[dict], default=None
Expand All @@ -1057,8 +1061,8 @@ def conformalize(
self.predict_params = predict_params if predict_params else {}

self._mapie_quantile_regressor.conformalize(
X_conf,
y_conf,
X_conformalize,
y_conformalize,
**self.predict_params
)

Expand Down

0 comments on commit 2df292b

Please sign in to comment.