CatBoost

CatBoost is an open-source gradient boosting library developed by Yandex, specializing in handling categorical features (hence the name Cat-Boost) without manual preprocessing. With over 8,200 stars as of 2026, CatBoost is the third major gradient boosting framework alongside XGBoost and LightGBM. CatBoost introduces several key innovations: ordered boosting (a permutation-driven approach that prevents prediction shift caused by target leakage in traditional gradient boosting, where the same data points are used for both calculating residuals and building the tree, CatBoost uses a permutation-based scheme where residuals for each data point are computed using models trained on previous points only), native categorical feature handling (CatBoost processes categorical variables directly without requiring one-hot encoding or label encoding, using target statistics with smoothing to convert categories to numeric values, and supporting combinations of categorical features automatically), symmetric tree structure (CatBoost uses oblivious decision trees where the same split condition is applied at every node at the same depth, enabling faster inference and reducing overfitting, with the tree being a full binary tree of depth d using exactly d conditions), and fast scoring with SIMD instructions. Key features include: training on GPU (via CUDA, with multi-GPU support), CPU training with multi-threading, cross-validation, feature importance via prediction values change and loss function change, SHAP values for model interpretability, model export to ONNX and CoreML for deployment, categorical feature processing without manual encoding, text feature support (with bag-of-words, Naive Bayes, and BM-25 text encoding), hyperparameter tuning with hyperopt and Optuna integration, missing value handling, class weights, and sample weights. Available for Python, R, and command-line.