L1, L2 Regularization in XGBoost Regression

Photo by Paolo Chiabrando on Unsplash

Regularization in gradient boosted regression trees are applied to the leaf values and not the feature coefficients like in lasso/ridge regression. For this blog, I will break down the explanation into three steps:

  1. Lasso & Ridge Regression
    - A brief re-cap of lasso and ridge regression
  2. Gradient Boosted Regression Trees
    - Leaf values in boosted regression trees