《206_handout》.pdf

Machine Learning Techniques ( ) Lecture 6: Support Vector Regression Hsuan-Tien Lin ( ) htlin@csie.ntu.edu.tw Department of Computer Science Information Engineering National Taiwan University ( ) Hsuan-Tien Lin (NTU CSIE) Machine Learning Techniques 0/23 Support Vector Regression Roadmap 1 Embedding Numerous Features: Kernel Models Lecture 5: Kernel Logistic Regression two-level learning for SVM-like sparse model for soft classification, or using representer theorem with regularized logistic error for dense model Lecture 6: Support Vector Regression Kernel Ridge Regression Support Vector Regression Primal Support Vector Regression Dual Summary of Kernel Models 2 Combining Predictive Features: Aggregation Models 3 Distilling Implicit Features: Extraction Models Hsuan-Tien Lin (NTU CSIE) Machine Learning Techniques 1/23 Support Vector Regression Kernel Ridge Regression Recall: Representer Theorem for any L2-regularized linear model N T 1 T min w w erry w z n n w N N n1 optimal w N z . n1 n n —any L2-regularized linear model can be kernelized! regression with squared error erry w Tz y w Tz2 —analytic solution