API Reference¶

The `RelaxedLassoLars` class¶

class relaxed_lasso.RelaxedLassoLars(alpha=1.0, theta=1.0, fit_intercept=True, verbose=False, normalize=True, precompute='auto', max_iter=500, eps=2.220446049250313e-16, copy_X=True, fit_path=True, jitter=None, random_state=None)¶

Bases: sklearn.base.MultiOutputMixin, sklearn.base.RegressorMixin, sklearn.linear_model._base.LinearModel

Relaxed Lasso model fit with Least Angle Regression.

See reference paper: Meinshausen N. (2006): Relaxed Lasso

alpha : float, default=1.0: Constant that multiplies the penalty term. Defaults to 1.0. Used for variables selection. alpha = 0 is equivalent to an ordinary least square, solved by LinearRegression. For numerical reasons, using alpha = 0 with the LassoLars object is not advised and you should prefer the LinearRegression object.
theta: float, default=1.0: Constant that relaxes the regularization parameter alpha. Value is between 0 and 1 theta = 1 is equivalent to LassoLars with regularization alpha theta = 0 is equivalent to an ordinary least square, solved by LinearRegression, applied to a subset of variables that was selected by LassoLars with regularization parameter alpha
fit_intercept : boolean, default=True: Whether to calculate the intercept for this model. If set to false, no intercept will be used in calculations (e.g. data is expected to be already centered).
verbose : boolean or integer, optional, default=False: Sets the verbosity amount
normalize : boolean, optional, default=True: This parameter is ignored when fit_intercept is set to False. If True, the regressors X will be normalized before regression by subtracting the mean and dividing by the l2-norm. If you wish to standardize, please use sklearn.preprocessing.StandardScaler before calling fit on an estimator with normalize=False.
precompute : bool, ‘auto’ or array-like, default=’auto’: Whether to use a precomputed Gram matrix to speed up calculations. If set to 'auto' let us decide. The Gram matrix can also be passed as argument.
eps : float, optional: The machine-precision regularization in the computation of the Cholesky diagonal factors. Increase this for very ill-conditioned systems. Unlike the tol parameter in some iterative optimization-based algorithms, this parameter does not control the tolerance of the optimization.
copy_X : boolean, optional, default=True: If True, X will be copied; else, it may be overwritten.
fit_path : boolean, default=True: If True the full path is stored in the coef_path_ attribute. If you compute the solution for a large problem or many targets, setting fit_path to False will lead to a speedup, especially with a small alpha.
jitter : float, default=None: Upper bound on a uniform noise parameter to be added to the y values, to satisfy the model’s assumption of one-at-a-time computations. Might help with stability.
random_state : int, RandomState instance or None (default): Determines random number generation for jittering. Pass an int for reproducible output across multiple function calls. Ignored if jitter is None.

alphas_ : array, shape (n_alphas_var,) | list of n_targets such arrays: Maximum of covariances (in absolute value) at each iteration. n_alphas is either max_iter, n_features, or the number of nodes in the path with correlation greater than alpha, whichever is smaller. Corresponds to alpha_var, i.e. alphas used for variables selection
active_ : list | list of n_targets such lists: Indices of active variables at the end of the path.
coef_path_ : array, shape (n_features, n_alphas_reg, n_alphas_var): list of n_targets such arrays

The varying values of the coefficients along the path. It is not present if the fit_path parameter is False.
coef_ : array, shape (n_features,) or (n_targets, n_features): Parameter vector (w in the formulation formula).
intercept_ : float | array, shape (n_targets,): Independent term in decision function.
n_iter_ : array-like or int: The number of iterations taken by lars_path to find the grid of alphas for each target.

>>> from relaxed_lasso import RelaxedLassoLars
>>> relasso = RelaxedLassoLars(alpha=0.01, theta=0.5)
>>> relasso.fit([[-1, 1], [0, 0], [1, 1]], [-1, 0, -1])
RelaxedLassoLars(alpha=0.01, copy_X=True, eps=2.220446049250313e-16,
             fit_intercept=True, fit_path=True, max_iter=500,
             normalize=True, precompute='auto', theta=0.5, verbose=False)
>>> print(relasso.coef_)
[ 0.         -0.98162883]

fit(X, y, Xy=None)¶

Fit the model using X, y as training data.

X : array-like, shape (n_samples, n_features): Training data.
y : array-like, shape (n_samples,) or (n_samples, n_targets): Target values.
Xy : array-like, shape (n_samples,) or (n_samples, n_targets),: optional

Xy = np.dot(X.T, y) that can be precomputed. It is useful only when the Gram matrix is precomputed.

self : object: returns an instance of self.

get_params(deep=True)¶

Get parameters for this estimator.

deep : bool, default=True: If True, will return the parameters for this estimator and contained subobjects that are estimators.

params : mapping of string to any: Parameter names mapped to their values.

predict(X)¶

Predict using the linear model.

X : array_like or sparse matrix, shape (n_samples, n_features): Samples.

C : array, shape (n_samples,): Returns predicted values.

score(X, y, sample_weight=None)¶

Return the coefficient of determination R^2 of the prediction.

The coefficient R^2 is defined as (1 - u/v), where u is the residual sum of squares ((y_true - y_pred) ** 2).sum() and v is the total sum of squares ((y_true - y_true.mean()) ** 2).sum(). The best possible score is 1.0 and it can be negative (because the model can be arbitrarily worse). A constant model that always predicts the expected value of y, disregarding the input features, would get a R^2 score of 0.0.

X : array-like of shape (n_samples, n_features): Test samples. For some estimators this may be a precomputed kernel matrix or a list of generic objects instead, shape = (n_samples, n_samples_fitted), where n_samples_fitted is the number of samples used in the fitting for the estimator.
y : array-like of shape (n_samples,) or (n_samples, n_outputs): True values for X.
sample_weight : array-like of shape (n_samples,), default=None: Sample weights.

score : float: R^2 of self.predict(X) wrt. y.

The R2 score used when calling score on a regressor uses multioutput='uniform_average' from version 0.23 to keep consistent with default value of r2_score(). This influences the score method of all the multioutput regressors (except for MultiOutputRegressor).

set_params(**params)¶

Set the parameters of this estimator.

The method works on simple estimators as well as on nested objects (such as pipelines). The latter have parameters of the form <component>__<parameter> so that it’s possible to update each component of a nested object.

**params : dict: Estimator parameters.

self : object: Estimator instance.

The `RelaxedLassoLarsCV` class¶

class relaxed_lasso.RelaxedLassoLarsCV(fit_intercept=True, verbose=False, max_iter=500, normalize=True, precompute='auto', cv=None, max_n_alphas=1000, n_jobs=None, eps=2.220446049250313e-16, copy_X=True)¶

Bases: relaxed_lasso.least_angle.RelaxedLassoLars

Cross-validated Relaxed Lasso, using the LARS algorithm.

fit_intercept : boolean: whether to calculate the intercept for this model. If set to false, no intercept will be used in calculations (e.g. data is expected to be already centered).
verbose : boolean or integer, optional: Sets the verbosity amount
max_iter : integer, optional: Maximum number of iterations to perform.
normalize : boolean, optional, default True: This parameter is ignored when fit_intercept is set to False. If True, the regressors X will be normalized before regression by subtracting the mean and dividing by the l2-norm. If you wish to standardize, please use sklearn.preprocessing.StandardScaler before calling fit on an estimator with normalize=False.
precompute : True | False | ‘auto’: Whether to use a precomputed Gram matrix to speed up calculations. If set to 'auto' let us decide. The Gram matrix cannot be passed as argument since we will use only subsets of X.
cv : int, cross-validation generator or an iterable, optional: Determines the cross-validation splitting strategy. Possible inputs for cv are: - None, to use the default 5-fold cross-validation, - integer, to specify the number of folds. - CV splitter, - An iterable yielding (train, test) splits as arrays of indices.
max_n_alphas : integer, optional: The maximum number of points on the path used to compute the residuals in the cross-validation
n_jobs : int or None, optional (default=None): Number of CPUs to use during the cross validation. None means 1 unless in a joblib.parallel_backend context. -1 means using all processors. See Glossary for more details.
eps : float, optional: The machine-precision regularization in the computation of the Cholesky diagonal factors. Increase this for very ill-conditioned systems.
copy_X : boolean, optional, default True: If True, X will be copied; else, it may be overwritten.

Attributes

coef_ : array, shape (n_features,): parameter vector (w in the formulation formula)
intercept_ : float: independent term in decision function.
coef_path_ : array, shape (n_features, n_alphas_reg, n_alphas_var): the varying values of the coefficients along the path
alpha_ : float: the estimated regularization parameter alpha (for variable selection) Corresponds to alpha_var, i.e. alphas used for variables selection
theta_ : float: the estimated regularization parameter theta (for relaxation)
alphas_ : array, shape (n_alphas,): the different values of alpha along the path
cv_alphas_ : array, shape (n_cv_alphas,): all the values of alpha along the path for the different folds Corresponds to alpha_var, i.e. alphas used for variables selection
mse_path_ : array, shape (n_folds, n_cv_alphas): the mean square error on left-out for each fold along the path (alpha values given by cv_alphas)
n_iter_ : array-like or int: the number of iterations run by Lars with the optimal alpha.

>>> from relaxed_lasso import RelaxedLassoLarsCV
>>> from sklearn.datasets import make_regression
>>> X, y = make_regression(noise=4.0, random_state=0)
>>> relasso = RelaxedLassoLarsCV(cv=5).fit(X, y)
>>> relasso.score(X, y)
0.9991...
>>> relasso.alpha_
0.3724...
>>> relasso.theta_
4.1115...e-13
>>> relasso.predict(X[:1,])
array([[-78.3854...]])

fit(X, y)¶

Fit the model using X, y as training data.

X : array-like, shape (n_samples, n_features): Training data.
y : array-like, shape (n_samples,): Target values.

self : object: returns an instance of self

get_params(deep=True)¶

Get parameters for this estimator.

deep : bool, default=True: If True, will return the parameters for this estimator and contained subobjects that are estimators.

params : mapping of string to any: Parameter names mapped to their values.

predict(X)¶

Predict using the linear model.

X : array_like or sparse matrix, shape (n_samples, n_features): Samples.

C : array, shape (n_samples,): Returns predicted values.

score(X, y, sample_weight=None)¶

Return the coefficient of determination R^2 of the prediction.

The coefficient R^2 is defined as (1 - u/v), where u is the residual sum of squares ((y_true - y_pred) ** 2).sum() and v is the total sum of squares ((y_true - y_true.mean()) ** 2).sum(). The best possible score is 1.0 and it can be negative (because the model can be arbitrarily worse). A constant model that always predicts the expected value of y, disregarding the input features, would get a R^2 score of 0.0.

X : array-like of shape (n_samples, n_features): Test samples. For some estimators this may be a precomputed kernel matrix or a list of generic objects instead, shape = (n_samples, n_samples_fitted), where n_samples_fitted is the number of samples used in the fitting for the estimator.
y : array-like of shape (n_samples,) or (n_samples, n_outputs): True values for X.
sample_weight : array-like of shape (n_samples,), default=None: Sample weights.

score : float: R^2 of self.predict(X) wrt. y.

The R2 score used when calling score on a regressor uses multioutput='uniform_average' from version 0.23 to keep consistent with default value of r2_score(). This influences the score method of all the multioutput regressors (except for MultiOutputRegressor).

set_params(**params)¶

Set the parameters of this estimator.

The method works on simple estimators as well as on nested objects (such as pipelines). The latter have parameters of the form <component>__<parameter> so that it’s possible to update each component of a nested object.

**params : dict: Estimator parameters.

self : object: Estimator instance.

The `RelaxedLasso` class¶

class relaxed_lasso.RelaxedLasso(alpha=1.0, theta=1.0, fit_intercept=True, normalize=False, precompute=False, copy_X=True, max_iter=1000, tol=0.0001, warm_start=False, positive=False, random_state=None, selection='cyclic', eps=0.001, verbose=False)¶

Bases: sklearn.linear_model._coordinate_descent.ElasticNet

Relaxed Lasso model fit with coordinate descent.

Technically the Lasso model is optimizing the same objective function as the Elastic Net with l1_ratio=1.0 (no L2 penalty).

See reference paper: Meinshausen N. (2006): Relaxed Lasso

alpha : float, default=1.0: Constant that multiplies the L1 term. Defaults to 1.0. alpha = 0 is equivalent to an ordinary least square, solved by the LinearRegression object. For numerical reasons, using alpha = 0 with the Lasso object is not advised. Given this, you should use the LinearRegression object.
theta: float, default=1.0: Constant that relaxes the regularization parameter alpha. Value is between 0 and 1 theta = 1 is equivalent to Lasso with regularization alpha theta = 0 is equivalent to an ordinary least square, solved by LinearRegression, applied to a subset of variables that was selected by LassoLars with regularization parameter alpha
eps : float, default=1e-3: Length of the path. eps=1e-3 means that alpha_min / alpha_max = 1e-3
fit_intercept : bool, default=True: Whether to calculate the intercept for this model. If set to False, no intercept will be used in calculations (i.e. data is expected to be centered).
normalize : bool, default=False: This parameter is ignored when fit_intercept is set to False. If True, the regressors X will be normalized before regression by subtracting the mean and dividing by the l2-norm. If you wish to standardize, please use sklearn.preprocessing.StandardScaler before calling fit on an estimator with normalize=False.
precompute : ‘auto’, bool or array-like of shape (n_features, n_features), default=False: Whether to use a precomputed Gram matrix to speed up calculations. If set to 'auto' let us decide. The Gram matrix can also be passed as argument. For sparse input this option is always True to preserve sparsity.
copy_X : bool, default=True: If True, X will be copied; else, it may be overwritten.
max_iter : int, default=1000: The maximum number of iterations
tol : float, default=1e-4: The tolerance for the optimization: if the updates are smaller than tol, the optimization code checks the dual gap for optimality and continues until it is smaller than tol.
warm_start : bool, default=False: When set to True, reuse the solution of the previous call to fit as initialization, otherwise, just erase the previous solution. See the Glossary.
positive : bool, default=False: When set to True, forces the coefficients to be positive.
random_state : int, RandomState instance, default=None: The seed of the pseudo random number generator that selects a random feature to update. Used when selection == ‘random’. Pass an int for reproducible output across multiple function calls. See Glossary.
selection : {‘cyclic’, ‘random’}, default=’cyclic’: If set to ‘random’, a random coefficient is updated every iteration rather than looping over features sequentially by default. This (setting to ‘random’) often leads to significantly faster convergence especially when tol is higher than 1e-4.
verbose : bool or int, default=False: Amount of verbosity.

alphas_ : array, shape (n_alphas_var,) | list of n_targets such arrays Maximum of covariances (in absolute value) at each iteration. n_alphas is either max_iter, n_features, or the number of nodes in the path with correlation greater than alpha, whichever is smaller. Corresponds to alpha_var, i.e. alphas used for variables selection

coefs_ : array, shape (n_features,) or (n_targets, n_features): Dim 0 are coefficients along the path given non zero variables defined by Dim 2 when applying relaxed regularization defined by Dim 1
dual_gaps_ : array, shape (n_alphas_var,): The dual gaps at the end of the optimization for each alpha.
sparse_coef_ : sparse matrix of shape (n_features, 1) or (n_targets, n_features): sparse_coef_ is a readonly property derived from coef_
intercept_ : float or ndarray of shape (n_targets,): independent term in decision function.
n_iter_ : int or list of int: number of iterations run by the coordinate descent solver to reach the specified tolerance.

lars_path lasso_path LassoLars LassoCV LassoLarsCV sklearn.decomposition.sparse_encode

The algorithm used to fit the model is coordinate descent.

To avoid unnecessary memory duplication the X argument of the fit method should be directly passed as a Fortran-contiguous numpy array.

fit(X, y)¶

Fit the model using X, y as training data.

X : array-like, shape (n_samples, n_features): Training data.
y : array-like, shape (n_samples,) or (n_samples, n_targets): Target values.

self : object: returns an instance of self.

get_params(deep=True)¶

Get parameters for this estimator.

deep : bool, default=True: If True, will return the parameters for this estimator and contained subobjects that are estimators.

params : mapping of string to any: Parameter names mapped to their values.

static path(X, y, *, l1_ratio=0.5, eps=0.001, n_alphas=100, alphas=None, precompute='auto', Xy=None, copy_X=True, coef_init=None, verbose=False, return_n_iter=False, positive=False, check_input=True, **params)¶

Compute elastic net path with coordinate descent.

The elastic net optimization function varies for mono and multi-outputs.

For mono-output tasks it is:

1 / (2 * n_samples) * ||y - Xw||^2_2
+ alpha * l1_ratio * ||w||_1
+ 0.5 * alpha * (1 - l1_ratio) * ||w||^2_2

For multi-output tasks it is:

(1 / (2 * n_samples)) * ||Y - XW||^Fro_2
+ alpha * l1_ratio * ||W||_21
+ 0.5 * alpha * (1 - l1_ratio) * ||W||_Fro^2

Where:

||W||_21 = \sum_i \sqrt{\sum_j w_{ij}^2}

i.e. the sum of norm of each row.

The `RelaxedLassoCV` class¶

class relaxed_lasso.RelaxedLassoCV(fit_intercept=True, verbose=False, max_iter=1000, normalize=False, precompute='auto', cv=None, tol=0.0001, max_n_alphas=1000, n_jobs=None, eps=1e-13, copy_X=True, positive=False, n_alphas=100, alphas=None)¶

Bases: relaxed_lasso.coordinate_descent.RelaxedLasso

Cross-validated Relaxed Lasso, using the Coordinate Descent algorithm.

n_alphas : int, default=100: Number of alphas to test along the regularization path
alphas : ndarray, default=None: List of alphas where to compute the models. If None alphas are set automatically
eps : float, default=1e-13: Length of the path. eps=1e-13 means that alpha_min / alpha_max = 1e-13
fit_intercept : boolean: whether to calculate the intercept for this model. If set to false, no intercept will be used in calculations (e.g. data is expected to be already centered).
verbose : boolean or integer, optional: Sets the verbosity amount
max_iter : integer, optional: Maximum number of iterations to perform.
normalize : boolean, optional, default True: This parameter is ignored when fit_intercept is set to False. If True, the regressors X will be normalized before regression by subtracting the mean and dividing by the l2-norm. If you wish to standardize, please use sklearn.preprocessing.StandardScaler before calling fit on an estimator with normalize=False.
precompute : True | False | ‘auto’: Whether to use a precomputed Gram matrix to speed up calculations. If set to 'auto' let us decide. The Gram matrix cannot be passed as argument since we will use only subsets of X.
cv : int, cross-validation generator or an iterable, optional: Determines the cross-validation splitting strategy. Possible inputs for cv are: - None, to use the default 5-fold cross-validation, - integer, to specify the number of folds. - CV splitter, - An iterable yielding (train, test) splits as arrays of indices.
max_n_alphas : integer, optional: The maximum number of points on the path used to compute the residuals in the cross-validation
n_jobs : int or None, optional (default=None): Number of CPUs to use during the cross validation. None means 1 unless in a joblib.parallel_backend context. -1 means using all processors. See Glossary for more details.
copy_X : boolean, optional, default True: If True, X will be copied; else, it may be overwritten.

Attributes

coef_ : array, shape (n_features,): parameter vector (w in the formulation formula)
dual_gaps_ : array, shape (n_alphas_var,): The dual gaps at the end of the optimization for each alpha.
sparse_coef_ : sparse matrix of shape (n_features, 1): sparse_coef_ is a readonly property derived from coef_
intercept_ : float: independent term in decision function.
alpha_ : float: the estimated regularization parameter alpha
theta_ : float: the estimated regularization parameter theta
alphas_ : array, shape (n_alphas,): the different values of alpha along the path
cv_alphas_ : array, shape (n_cv_alphas,): all the values of alpha along the path for the different folds Corresponds to alpha_var, i.e. alphas used for variables selection
mse_path_ : array, shape (n_folds, n_cv_alphas): the mean square error on left-out for each fold along the path (alpha values given by cv_alphas)
n_iter_ : array-like or int: the number of iterations run by Lars with the optimal alpha.

fit(X, y)¶

Fit the model using X, y as training data.

X : array-like, shape (n_samples, n_features): Training data.
y : array-like, shape (n_samples,): Target values.

self : object: returns an instance of self

get_params(deep=True)¶

Get parameters for this estimator.

deep : bool, default=True: If True, will return the parameters for this estimator and contained subobjects that are estimators.

params : mapping of string to any: Parameter names mapped to their values.

static path(X, y, *, l1_ratio=0.5, eps=0.001, n_alphas=100, alphas=None, precompute='auto', Xy=None, copy_X=True, coef_init=None, verbose=False, return_n_iter=False, positive=False, check_input=True, **params)¶

Compute elastic net path with coordinate descent.

The elastic net optimization function varies for mono and multi-outputs.

For mono-output tasks it is:

1 / (2 * n_samples) * ||y - Xw||^2_2
+ alpha * l1_ratio * ||w||_1
+ 0.5 * alpha * (1 - l1_ratio) * ||w||^2_2

For multi-output tasks it is:

(1 / (2 * n_samples)) * ||Y - XW||^Fro_2
+ alpha * l1_ratio * ||W||_21
+ 0.5 * alpha * (1 - l1_ratio) * ||W||_Fro^2

Where:

||W||_21 = \sum_i \sqrt{\sum_j w_{ij}^2}

i.e. the sum of norm of each row.

API Reference¶

The RelaxedLassoLars class¶

The RelaxedLassoLarsCV class¶

The RelaxedLasso class¶

The RelaxedLassoCV class¶

The `RelaxedLassoLars` class¶

The `RelaxedLassoLarsCV` class¶

The `RelaxedLasso` class¶

The `RelaxedLassoCV` class¶