陳天奇論文演講-資料下載頁(yè)
2025-08-05 17:57本頁(yè)面
【正文】 ion, calculate ? Use the statistics to greedily grow a tree ? Add to the model ? Usually, instead we do ? is called stepsize or shrinkage, usually set around ? This means we do not do full optimization in each step and reserve chance for future rounds, it helps prevent overfitting Outline ? Review of key concepts of supervised learning ? Regression Tree and Ensemble (What are we Learning) ? Gradient Boosting (How do we Learn) ? Summary Questions to check if you really get it ? How can we build a boosted tree classifier to do weighted regression problem, such that each instance have a importance weight? ? Back to the time series problem, if I want to learn step functions over time. Is there other ways to learn the time splits, other than the top down split approach? Questions to check if you really get it ? How can we build a boosted tree classifier to do weighted regression problem, such that each instance have a importance weight? ? Define objective, calculate , feed it to the old tree learning algorithm we have for unweighted version ? Again think of separation of model and objective, how does the theory can help better anizing the machine learning toolkit Questions to check if you really get it ? Time series problem ? All that is important is the structure score of the splits ? Topdown greedy, same as trees ? Bottomup greedy, start from individual points as each group, greedily merge neighbors ? Dynamic programming, can find optimal solution for this case Summary ? The separation between model, objective, parameters can be helpful for us to understand and customize learning models ? The biasvariance tradeoff applies everywhere, including learning in functional space ? We can be formal about what we learn and how we learn. Clear understanding of theory can be used to guide cleaner implementation. Reference ? Greedy function approximation a gradient boosting machine. . Friedman ? First paper about gradient boosting ? Stochastic Gradient Boosting. . Friedman ? Introducing bagging trick to gradient boosting ? Elements of Statistical Learning. T. Hastie, R. Tibshirani and . Friedman ? Contains a chapter about gradient boosted boosting ? Additive logistic regression a statistical view of boosting. . Friedman T. Hastie R. Tibshirani ? Uses secondorder statistics for tree splitting, which is closer to the view presented in this slide ? Learning Nonlinear Functions Using Regularized Greedy Forest. R. Johnson and T. Zhang ? Proposes to do fully corrective step, as well as regularizing the tree plexity. The regularizing trick is closed related to the view present in this slide ? Software implementing the model described in this slide:
環(huán)評(píng)公示相關(guān)推薦
鄂ICP備17016276號(hào)-1