← All tutorials

Skill Zuno · skillzunos.com · Python ML

XGBoost tutorial

Learn XGBoost on Skill Zuno. This XGBoost tutorial covers New, Easy, Medium, Hard, and Diff levels so you can study practical XGBoost problems on skillzunos.com.

Open XGBoost on Skill Zuno

Also see the XGBoost interview questions.

XGBoost (eXtreme Gradient Boosting) is an optimised library for gradient boosted trees, widely used in industry and in data science competitions. It adds regularisation to the boosting objective, uses second-order gradients, handles missing values and trains quickly in parallel on CPU or GPU. This subject covers XGBoost for regression and general use: its objective, key parameters and APIs.

XGBoost topics covered

  • What makes XGBoost different — A regularised objective, second-order gradients and efficient split finding.
  • The native API and the scikit-learn API — DMatrix with xgb.train, versus XGBRegressor and XGBClassifier.
  • Key parameters — eta, max_depth, min_child_weight, subsample, colsample_bytree, lambda and alpha.
  • Missing values and sparse data — How XGBoost learns a default direction for missing values at each split.
  • Early stopping and evaluation — Using eval sets and early_stopping_rounds to choose the number of trees.
  • Speed and scale — The hist tree method, GPU training and distributed training.

What you can learn

  • XGBoost tutorial for beginners and experienced developers
  • Level-based practice: New, Easy, Medium, Hard, and Diff
  • Python ML topics alongside XGBoost on skillzunos.com

More Python ML tutorials

All tutorials