vault backup: 2025-01-17 23:55:19

Foundation of data science/notes/9 XGBoost.md

@@ -34,11 +34,11 @@ L(Θ)=∑i=1nl(yi,y^i)+∑k=1TΩ(hk)L(\Theta) = \sum_{i=1}^n l(y_i, \hat{y}_i) +
         - Multiclass Log Loss for multiclass classification.
 2. **Second Term: Regularization Term (Ω(hk)\Omega(h_k))**
     
-    - Adds penalties for model complexity to avoid overfitting: Ω(hk)=γT+12λ∑jwj2\Omega(h_k) = \gamma T + \frac{1}{2} \lambda \sum_j w_j^2
-        - TT: Number of leaves in the tree.
-        - wjw_j: Weights of the leaves.
-        - γ\gamma: Penalizes additional leaves.
-        - λ\lambda: Penalizes large leaf weights (L2 regularization).
+    - Adds penalties for model complexity to avoid overfitting: $\Omega(h_k) = \gamma T + \frac{1}{2} \lambda \sum_j w_j^2$
+        - T: Number of leaves in the tree.
+        - w: Weights of the leaves.
+        - γ Penalizes additional leaves.
+        - λ: Penalizes large leaf weights (L2 regularization).
 
 ---
 
@@ -96,7 +96,7 @@ XGBoost constructs decision trees by finding splits that minimize the loss funct
 
 For a given split, the gain is computed as:
 
-Gain=12[GL2HL+λ+GR2HR+λ−(GL+GR)2HL+HR+λ]−γ\text{Gain} = \frac{1}{2} \left[ \frac{G_L^2}{H_L + \lambda} + \frac{G_R^2}{H_R + \lambda} - \frac{(G_L + G_R)^2}{H_L + H_R + \lambda} \right] - \gamma
+$$\text{Gain} = \frac{1}{2} \left[ \frac{G_L^2}{H_L + \lambda} + \frac{G_R^2}{H_R + \lambda} - \frac{(G_L + G_R)^2}{H_L + H_R + \lambda} \right] - \gamma$$
 
 Where: