Feed Forward Neural Networks

Read It: Feed Forward Neural Networks

[00:18:59.64] OK, so then let's focus on feed forward neural networks here, guys. So, the overall workflow for creating a neural network is we're going to operate in this general framework here. And we start with the data set. And say our Rex data set-- we've got, say, 5,000 data points. Again, I'm rounding off here, but let's say 5,000 data points.  

[00:19:26.09] We need to split this data set into two subsets-- one called the training data and one called the validation data. Sometimes that validation data set is called the testing data set set. So, we've got our training set and our validation set. I'm going to stick with the term "validation," by the way, because that's what the Python tools that we use call it. So, I'll keep consistent with that.  

[00:19:49.71] So for example, we might say we'll just randomly take 80% of our observations and put them in the training set, and the other 20% go in the validation set. And that's a typical split. Usually, you want more in the training set and less in the validation set.  

[00:20:16.17] So once you've made that split of the data, then you want to specify your network architecture and various settings. And that's kind of some abstract terminology there but let me go down this list and indicate the sorts of things that you want to choose to design in your neural network. So, one thing you want to make a choice about is how many hidden layers you want to include. I'll explain what I mean by hidden layers in a little bit in a picture.  

[00:20:47.02] But basically, how many different processing steps do you want to have in your network? How many nodes per layer do you want to have? And again, I'll have a picture that illustrates what this means. So basically, each node will have its own function in there.  

[00:21:06.25] So we want to decide how many nodes to have in each layer. And then we need to decide on what functions to use in those nodes. These are called activation functions. And even those activation functions might have some constants in them, like hyperparameters-- some things that we need to set in them.  

[00:21:27.42] We need to have some metric for fitting. I said we want to minimize the dissimilarity between these two, but how do we measure the dissimilarity? With linear regression, I said we use the sum of the squared errors. And so perhaps that's one thing we could do. But there's various other loss functions that we could choose from. And I'll go over those as well in this lecture.  

[00:21:53.60] And then something that's really important, too, is this number of epochs. Epochs is really just a fancy term for the number of iterations that we go through to train our neural network. But what I mean by that is, OK, we might propose this architecture. And the neural network is going to go through with some initial guess about some optimal-- I'm just going to call them beta values to use here.  

[00:22:24.75] And I'll make some initial guess on the predictions. But most likely, these are going to be very different than the ground truth. And so, then it's going to look at that difference, and it's going to go back and adjust those datas. And then it's going to do it again.  

[00:22:39.00] And so this question is, how many times do we want to repeat doing that? How many times do I want to keep going back and tuning those coefficients, basically turning the dials in our network function, to get the dissimilarity lower and lower and lower? So, we need to specify all those things. 

[00:23:04.11] And then once we do that, we then fit the network. So, then we actually go through the process of finding those optimal beta values, adjusting those dials. Now, the key thing here is that we only fit this to the training data. So, for example, 80% of our-- randomly selected 80% of our initial data set.  

[00:23:27.86] So we're only doing this based on the training data. This is completely blind to the validation data. So, you're saying, let's design this network, get it to fit really well to most of the data, and then-- I'll go back to these. And then ultimately what we'll do is we'll see how good that end model does on predicting the validation data set because that's the key thing is it's not necessarily how well a model can fit to existing data. This is all geared towards prediction.  

[00:24:12.67] All those applications I showed you before were forecasting and prediction. We want to see how good this is at predicting things. So, we're just going to test out its predictive accuracy on data that we've already measured. And we can say, it's got such and such predictive error.  

[00:24:32.10] Let me wind back a little bit. So, I wanted to find some other things here. So, when we talk about features, we're just talking about the inputs. This is just neural network lingo for the explanatory variables-- our x's, our independent variables.  

[00:24:52.22] Labels-- this is neural network lingo for the y's, the response variables, the dependent variables. So, our features are the inputs, and our labels are the output. And so, our training data will have some set of inputs and then this ground truth.  

[00:25:17.15] And so when we're fitting, I said we're trying to find these optimal betas. And I said betas because that's analogous. That's what we did with linear regression. We were trying to find-- those were the coefficients.  

[00:25:29.41] But also in neural network lingo, we're going to replace betas with something called weights, but they're essentially the same thing. They're these coefficients that we can adjust to lend more value to one part of the calculation than others. And so, we train on these training features and training labels. We get our best fitting model.  

[00:26:01.20] We have our network. We think this is the best thing that we have. And then we just throw in our validation features. We get some predictions, and we compare that to our validation labels. And that final validation error is going to be the difference between the testing labels and our output prediction here.  

---

Try It: OPTION 1 GOOGLE COLAB

1. Click the Google Colab file used in the video here.
2. Go to the Colab file and click "File" then "Save a copy in Drive", this will create a new Colab file that you can edit in your own Google Drive account.
3. Once you have it saved in your Drive, try to implement the following code to import a file of your choice by mounting your Google Drive:

from google.colab import drive

drive.mount('/content/drive')

import pandas as pd

df = pd.read_csv('yourfilename.csv')

df # print the dataframe

Once you have implemented this code on your own, come back to this page to test your knowledge.

---

OPTION 2 : DATACAMP

 Try It: OPTION 2 DataCamp - Apply Your Coding Skills

Dictionaries are a quick way to create a variable from scratch. However, their functionality is limited, so we will often want to convert those dictionaries into DataFrames. Try to code this conversion in the cell below. Hint: Make sure to import the Pandas library.

---

Assess It: Check Your Knowledge