better_deep_learning_mini_courseJason Brownlee 博士的

文件名称: better_deep_learning_mini_course

所属分类: 深度学习

开发工具:

文件大小: 201kb

下载次数: 0

上传时间: 2019-03-15

提供者: weixin_********

下载 (201kb)

不能下载？报告错误

详细说明：Jason Brownlee 博士的最新力作，深度学习速成课程，共七个章节，每个章节用时5--30分钟，阅读速度根据个人的基础因人而异，很实用的技巧，书里代码运行一遍之后，保障有质的提升。Contents Before We get started Lesson 01: Better Deep Learning Framework Lesson 02 batch size 1357 Lesson 03: Learning Rate Schedule Lesson 04 Batch normalization Lesson 05: Weight Regularization Lesson 06: Adding noise 13 Lesson 07: Early Stopping 15 Final word before You go Before We get started Configuring neural network models is often referred to as a dark art. This is because there are no hard and fast rules for configuring a network for a given problem. We cannot analytically calculate the optimal model type or model configuration for a given dataset. Fortunately, there are techniques that are known to address specific issues when configuring and training a neural network that are available in modern deep learning libraries such as Keras. In this crash course you will discover how you can confidently get better performance from your deep learning inodels in seven days. Let's get started Who is this crash-Course for? Before we get started. let's make sure you are in the right place. The list below provides some general guidelines as to who this course was designed for You need to know Your way around basic Python and NumPy The basics of Keras for deep learning you do not need to know You do not need to be a math wiz You do not need to be a deep learning expert This crash course will take you from a developer that knows a little deep learning to a developer who can get better performance on your deep learning project. This crash course assumes you have a working Python 2 or 3 SciPy environment with at least NumPy and Keras 2 installed. If you need help with your environment, you can follow the step-by-step tutorial here How to Setup a Python Environment for Machine Learning and Deep learning Crash-Course Overview crash course is broken down into seven lessons. You could complete one lesson per day (recommended)or complete all of the lessons in one day(hardcore). It really depends on the time vou have available and your level of enthusiasm below are seven lessons that will allow you to confidently improve the performance of your deep learning model Lesson 01: Better Deep Learning framework Lesson 02: Batch Size Lesson 03: Learning Rate schedule Lesson 04: Batch normalization Lesson 05: Weight regularization. Lesson 06: Adding Noise Lesson 07: Early Stopping Fach lesson could take you 60 seconds or up to 30 minutes. Take your time and complete the lessons at your own pace. The lessons expect you to go off and find out how to do things. I will give you hints, but part of the point of each lesson is to force you to learn where to go to look for help(hint, I have all of the answers directly on this blog, use the search box i do provide more help in the form of links to related posts because I want you to build up some confidence and inertia. Post your results online. I'll cheer you on Hang in there, dont give up! Lesson 01: Better Deep Learning Framework In this lesson, you will discover a framework that you can use to systematically improve the performance of your deep learning model. Modern deep learning libraries such as Keras allow you to define and start fitting a wide range of neural network models in minutes with just a few lines of code. Nevertheless, it is still challenging to configure a neural network to get good performance on a new predictive modeling problem. There are three types of problems that are straightforward to diagnose with regard to the poor performance of a deep learning neural network model; they are Problems with Learning. Problems with learning manifest in a model that cannot effectively learn a training dataset or shows slow progress or bad performance when learning the training dataset Problems with Generalization. Problems with generalization manifest in a model that overfits the training dataset and makes poor performance on a holdout dataset Problems with Predictions. Problems with predictions manifest as the stochastic training algorithm having a strong influence on the final model, causing a high variance in behavior and performance The sequential relationship between the three areas in the proposed breakdown allows the issue of deep learning model performance to be first isolated, then targeted with a specific technique or methodology. We call sunmarize techniques that assist with each of these problems as follows Better Learning. Techniques that improve or accelerate the adaptation of neural network model weights in response to a training dataset Better Generalization. Techniques that improve the performance of a neural network model on a holdout dataset Better Predictions. Techniques that reduce the variance in the performance of a final mode You can use this framework to first diagnose the type of problem that you have and then identify a technique to evaluate to attempt to address your problem Your task For this lesson, you must list two techniques or areas of focus that belong to each of the three areas of the framework. Having trouble? note that we will be looking some examples from two of the three areas as part of this mini-course. Post your findings online. I would love to see what you discover Next In the next lesson, you will discover how to control the speed of learning with the batch size Lesson o2 batch size In this lesson, you will discover the iinportance of the batch size when training neural networks Neural networks are trained using gradient descent where the estimate of the error used to update the weights is calculated based on a subset of the training dataset. The number of examples from the training dataset used in the estinate of the error gradient is called the batch size and is an inportant hyperparaneter that influences the dynamics of the learning algorithm The choice of batch size controls how quickly the algorithm learns, for example Batch Gradient Descent. Batch size is set to the number of examples in the training dataset, more accurate estimate of error but longer time between weight updates Stochastic Gradient Descent. Batch size is set to 1, noisy estimate of error but frequent updates to weights e Minibatch gradient descent. batch size is set to a, value more than l and less than the number of training examples, trade-off between batch and stochastic gradient descent. Keras allows you to configure the batch size via the batch -size argument to the fito) function, for example t fit mode l history model fit(train, train, epochs=1000, batch-size=len(train)) Listing 1: Example of batch gradient descent The example below demonstrates a Multilayer Perceptron with batch gradient descent on a binary classification problem example of batch gradient descent sklearn datasets import make circles f keras layers import dense from keras models import Sequential from keras optimizers import SGD from matplotlib import pyplot generate dataset X, y=make_circles(n_samples=1000, noise=0.1, random_state=1) split into train and test m train 500 train, test =X[:n train :], xln train:: train, testy =yL: n_train], y In-train: I define model model= Sequential model. add (Dense(50, input_dim=2, activation='relu)) model. add (Dense(1, activation='sigmoid)) 6 opt SGD(lr=. 01, momentum=0.9) model. compile(loss='binary-crossentropy', optimizer=opt, metrics=['accuracy'1) t fit mode l history = modelfit(train, train, validation-data=(testa, testy), epochs=1000 batch- size=len(train), verbose=0) evaluate the model train_acc = model. evaluate(train, train, verbose=0) test_acc = model, evaluate(test, testy, verbose=o) print( Train: %3f, Test: %3f%(train_acc, test_acc)) plot loss learning curves pyplot. subplot(211) pyplot.title('Cross-Entropy Loss, pad=-40) pyplot. plot(history history ['loss'J, label='train') pyplot. plot(history history['val_loss'], 1 pyplot. legend( plot accuracy learning curves pyplot. subplot(212) pyplot. title('Accur pad pyplot. plot (history history ['acc'], label='train') pyplot. plot(history history['val-acc'l, label='test') pyplot. legend() pyplot. show( Listing 2: Example of MLP with batch gradient descent for binary classification Your task For this lesson, you must run the code example with each type of gradient descent(batch minibatch, and stochastic)and describe the effect that it has on the learning curves during training. Post your findings online. I would love to see what you can come up with Next In the next lesson, you will discover how to fine tune a model during training with a learning rate schedule Lesson 03: Learning Rate Schedule In this lesson, you will discover how to configure an adaptive learning rate schedule to fine tune the model during the training run. The amount of change to the model during each step of this search process, or the step size, is called the learning Tale and provides perhaps the most important hyperparameter to tune for your neural network in order to achieve good perfornance on your problen. Configuring a fixed learning rate is very challenging and requires careful experimentation. An alternative to using a fixed learning rate is to instead vary the learning rate over the training process. Keras provides the ReduceLROnPlateau learning rate schedule that will adjust the learning rate when a plateau in model performance is detected, e. g.no change for a given number of training epochs. For example define learning rate schedule ReducelROnPlateau(monitor='val-loss', factor=0. 1, patience=5, min_delta=lE-7, verbose=1) Listing 3: Example of a learning rate schedule callback This callback is designed to reduce the learning rate after the model stops inproving with the hope of fine-tuning inodel weights during training. The example below demonstrates a Multilayer Perceptron with a learning rate schedule on a binary classification problen, where the learning rate will be reduced by an order of magnitude if no change is detected in validation loss over training epochs example of a learning rate schedule from sklearn datasets import make_circles raslayers import Dense port Sequential from keras optimizers import SGD from keras callbacks import ReducelronPlateau from matplotlib import pyplot generate dataset X, y=make_circles(n_samples=1000, noise=0. 1, random_state=1) split into train and test train,test = X[:n train, ] XIn train:,: 1 rainy, testy =y[: ntrain], yIn_train: I define model model Sequential( model. add(Dense(50, input_dim=2, activation=relu)) model. add(Dense(1, activation=sigmoid)) compile model opt SGD(lr=0. 01, momentum=0. 9) model. compile(loss='binary-crossentropy', optimizer-opt, metrics=['accuracy'1) define learning rate schedule

(系统自动生成,下载前可以参看下载内容)