【正文】 ? 多項式回歸模型可以變換為線性回歸模型 . 例如 y = w0 + w1 x + w2 x2 + w3 x3 借助新變量 : x2 = x2, x3= x3 y = w0 + w1 x + w2 x2 + w3 x3 ? 其他函數(shù) ,如冪函數(shù) , 也可以轉(zhuǎn)化為線性函數(shù) ? Some models are intractable nonlinear (., 指數(shù)相求和 ) ? 可能通過更復(fù)雜的公式綜合計算，得到最小二乘估計 非線性回歸 2022年 1月 4日星期二 Data Mining: Concepts and Techniques 73 ? Generalized linear model: ? Foundation on which linear regression can be applied to modeling categorical response variables ? Variance of y is a function of the mean value of y, not a constant ? Logistic regression: models the prob. of some event occurring as a linear function of a set of predictor variables ? Poisson regression: models the data that exhibit a Poisson distribution ? Loglinear models: (for categorical data) ? Approximate discrete multidimensional prob. distributions ? Also useful for data pression and smoothing ? Regression trees and model trees ? Trees to predict continuous values rather than class labels Other RegressionBased Models 74 Regression Trees and Model Trees ? Regression tree: proposed in CART system (Breiman et al. 1984) ? CART: Classification And Regression Trees ? Each leaf stores a continuousvalued prediction ? It is the average value of the predicted attribute for the training tuples that reach the leaf ? Model tree: proposed by Quinlan (1992) ? Each leaf holds a regression model—a multivariate linear equation for the predicted attribute ? A more general case than regression tree ? Regression and model trees tend to be more accurate than linear regression when the data are not represented well by a simple linear model 75 Prediction: Numerical Data 76 Prediction: Categorical Data 77 Chapter 6. 分類 : 基本概念 ? 分類 : 基本概念 ? 決策樹歸納 ? 貝葉斯分類 ? 基于規(guī)則的分類 ? 模型評價與選擇 ? 提高分類準(zhǔn)確率的技術(shù) :集成方法 Ensemble Methods ? Summary Summary (I) ? Classification is a form of data analysis that extracts models describing important data classes. ? Effective and scalable methods have been developed for decision tree induction, Naive Bayesian classification, rulebased classification, and many other classification methods. ? Evaluation metrics include: accuracy, sensitivity, specificity, precision, recall, F measure, and F223。 measure. ? Stratified kfold crossvalidation is remended for accuracy estimation. Bagging and boosting can be used to increase overall accuracy by learning and bining a series of individual models. 78 Summary (II) ? Significance tests and ROC curves are useful for model selection. ? There have been numerous parisons of the different classification methods。 the matter remains a research topic ? No single method has been found to be superior over all others for all data sets ? Issues such as accuracy, training time, robustness, scalability, and interpretability must be considered and can involve tradeoffs, further plicating the quest for an overall superior method 79 References (1) ? C. Apte and S. Weiss. Data mining with decision trees and decision rules. Future Generation Computer Systems, 13, 1997 ? C. M. Bishop, Neural Networks for Pattern Recognition. Oxford University Press, 1995 ? L. Breiman, J. Friedman, R. Olshen, and C. Stone. Classification and Regression Trees. Wadsworth International Group, 1984 ? C. J. C. Burges. A Tutorial on Support Vector Machines for Pattern Recognition. Data Mining and Knowledge Discovery, 2(2): 121168, 1998 ? P. K. Chan and S. J. Stolfo. Learning arbiter and biner trees from partitioned data for scaling machine learning. KDD39。95 ? H. Cheng, X. Yan, J. Han, and . Hsu, Discriminative Frequent Pattern Analysis for Effective Classification, ICDE39。07 ? H. Cheng, X. Yan, J. Han, and P. S. Yu, Direct Discriminative Pattern Mining for Effective Classification, ICDE39。08 ? W. Cohen. Fast effective rule induction. ICML39。95 ? G. Cong, . Tan, A. K. H. Tung, and X. Xu. Mining topk covering rule groups for gene expression data. SIGMOD39。05 80 References (2) ? A. J. Dobson. An Introduction to Generalized Linear Models. Chapman amp。 Hall, 1990. ? G. Dong and J. Li. Efficient mining of emerging patterns: Discovering trends and differences. KDD39。99. ? R. O. Duda, P. E. Hart, and D. G. Stork. Pattern Classification, 2ed. John Wiley, 2022 ? U. M. Fayyad. Branching on attribute values in decision tree generation. AAAI’94. ? Y. Freund and R. E. Schapire. A decisiontheoretic generalization of online learning and an application to boosting. J. Computer and System Sciences, 1997. ? J. Gehrke, R. Ramakrishnan, and V. Ganti. Rainforest: A framework for fast decision tree construction of large datasets. VLDB’98. ? J. Gehrke, V. Gant, R. Ramakrishnan, and . Loh, BOAT Optimistic Decision Tree Construction. SIGMOD39。99. ? T. Hastie, R. Tibshirani, and J. Friedman. The Elements of Statistical Learning: Data Mining, Inference, and Prediction. SpringerVerlag, 2022. ? D. Heckerman, D. Geiger, and D. M. Chickering. Learning Bayesian works: The bination of knowledge and statistical data. Machine Learning, 1995. ? W. Li, J. Han, and J. Pei, CMAR: Accurate and Efficient Classification Based on Multiple ClassAssociation Rules, ICDM39。01. 81 References (3) ? . Lim, . Loh, and . Shih. A parison of prediction accuracy, plexity, and training time of thirtythree old and new classification algorithms. Machine Learning, 2022. ? J. Magidson. The Chaid approach to segmentation modeling: Chisquared automatic interaction detection. In R. P. Bagozzi, editor, Advanced Methods of Marketing Research, Blackwell Business, 1994. ? M. Mehta, R. Agrawal, and J. Rissanen. SLIQ : A fast scalable classifier for data mining. EDBT39。96. ? T. M. Mitchell. Machine Learning. McGraw Hill, 1997. ? S. K. Murthy, Automatic Construction of Decision Trees from Data: A MultiDisciplinary Survey, Data Mining and Knowledge Discovery 2(4): 345389, 1998 ? J. R. Quinl