http://www.kdd.org/kdd2016/papers/files/rfp0697-chenAemb.pdf
https://homes.cs.washington.edu/~tqchen/pdf/BoostedTree.pdf
【Training Loss measures how well model fit on training data】
【Regularization, measures complexity of model】
【simple】
【predictive】
• Why do we want to contain two component in the objective?
• Optimizing training loss encourages predictive models
Fitting well in training data at least get you close to training data which is hopefully close to the underlying distribution
• Optimizing regularization encourages simple models
Simpler models tends to have smaller variance in future predictions, making prediction stable
【训练损失：量化对点的拟合度】
Training Loss: How will the function fit on the points?
【泛化、正则化：量化复杂度、通用性】
Regularization: How do we define complexity of the function?