class3x.pdf介绍了机器学习中线性回归、分类等基本概念，包含了欠饱和与过饱和的介绍，详细介绍

文件名称: class3x.pdf

所属分类: 机器学习

开发工具:

文件大小: 896kb

下载次数: 0

上传时间: 2019-10-05

提供者: qq_39******

下载 (896kb)

不能下载？报告错误

详细说明：介绍了机器学习中线性回归、分类等基本概念，包含了欠饱和与过饱和的介绍，详细介绍了梯度下降的方法，是全英文版的。Tony Jebara, Columbia University Polynomial Basis Functions .To fit a P'th order polynomial function to multivariate data concatenate columns of all monomials up to power p E.g. 2 dimensional data and 2nd order polynomial(quadratic) Max. Throughpu C )x1(2)x1()1(1)1()-1(2)1(2)x(2) r()x(2)x()()(1x2)x(2)1(2) 1a()x(2)x()-s()x()-x(2)x(2)x(2) Tony Jebara, Columbia University Sinusoidal basis functions 15 .More generally, we don't just have to deal With polynomials, use any set of basis fn's: 0.5 P 0d(x)+0 1p「p 5 0 . These are generally called Additive Models Regression adds linear combinations of the basis fn's oFor example: Fourier(sinusoidal), basis .= coS ke 2k 2k+1 eNote, dont have to be a basis per se usually subset 6×0 +6×a +0× 0 Tony Jebara, Columbia University Radial basis functions o Can act as prototypes of the data itself X-X e parameter o= standard deviation 02= covariance controls how wide bumps are 0.5 what happens if too big/small? 05 10 10 .Also works in multi-dimensions ● Called rbf for short 40 x 40 20 00 Tony Jebara, Columbia University Radial basis functions .Each training point leads to a bump function f(x:0 0. ex k=1°k X-Xk rEuse solution from linear regression: 0=XX Xy o Can view the data instead as x a big matrix of sizeN X N exp exp exp 20 X= exp eX X -X 20 2 X 20 eX exp-1x-x 2 20 3 X 2 20 For RBFs, X is square and symmetric, so solution is just Va=0→Xx0=Xy→x0=y→0=xy Tony Jebara, Columbia University Evaluating Our Learned Function .We minimized empirical risk to get 0*K .How well does f(x; 0*) perform on future data? oIt should generalize and have low true risk Rn()=∫P(y1(29)mh o Can't compute true risk instead use Testing Empirical Risk .We randomly split data into training and testing portions N+129N+1)3∴5N+M59N+M .Find 8* with training data: R train N L(3,f(x,;0 .Evaluate it with testing data: R.10-1)N+M test M LIg,f(a; 0 =N+1 Tony Jebara, Columbia University Crossvalidation eTry fitting with different sigma radial basis function widths eSelect sigma which gives lowest Rtest(0*) 木 LOSS test train underfitting overfitting Best sigma . Think of sigma as a measure of the simplicity of the model .Thinner RBFs are more flexible and complex Tony Jebara, Columbia University Regularized risk minimization eMpirical risk Minimization gave overfitting underfitting oWe want to add a penalty for using too many theta values . This gives us the regularized risk 0)=R regularized empirical 0+Penalty(0 f(x;))+2l 2M o Solution for Regularized risk with Least Squares Loss VR =0→Vy-X+, 0 8 regularized 2N 0=XX+XXy Tony Jebara, Columbia University Regularized risk minimization Have D=16 features(or P=15 throughout) eTry minimizing Rreqularized e)to get 0*k with different n nOte that 2=0 give back Empirical Risk Minimization lambda=1. 0e+06 mbda=1.0e+04 lambda=1.0e+02 lambda=1, 0e+00 lambda=1. 0e-02 lambda=1. 0e-04 -2 2

(系统自动生成,下载前可以参看下载内容)