fast.ai: Deep Learning Part 1 (v2) (2018)
We learn today how to classify dogs from cats. Rather than understanding the mathematical details of how this works, we start by learning the nuts and bolts of how to get the computer to complete the task, using ‘fine-tuning’, perhaps the most important skill for any deep learning practitioner. In a later lesson we’ll learn about how fine-tuning actually works “behind the scenes”.
An important point discussed is how the data for this lesson needs to be structured. This is the most important step for you to complete—if your data is not structured correctly you will not be able to train any models.
(autogenerated subtitles follow, may contain gibberish/bad format - please proofread to improve - remove this note once proofread)
Hi everybody welcome to practical, deep learning for coders. This is part one of our two-part course. I'm presenting this from the data Institute in San Francisco will be doing seven lessons in this part of the course most of them will be about a couple of hours long. This first one may be a little bit shorter, practical, deep learning for coders is all about getting you up and running, with deep learning in practice, getting world-class results and it's a really coding focused approach, as the name suggests, but we're not going to dumb it down By the end of the course, you all have learned all of the theory in details that are necessary to rebuild all of the world-class results, we're learning about from scratch. Now I should mention that our videos are hosted on YouTube, that we strongly recommend watching them via our website, at course fast a I, although they're exactly the same videos, the important thing about watching them through our website is that you'll get all of the information you Need about kind of updates to libraries, my own locations, further information, frequently asked questions and so forth. So, if you're currently on YouTube watching this, why don't you switch over to crosstalk fast at AI now and start watching through there and make sure you read all of the material on the page before you start just to make sure that you've got everything you need? The other thing to mention is that there is a really great strong community at forums faster.
I, from time to time you will find that you get stuck, you may get stuck very early on. You may not get stuck for quite a while, but at some point you might get stuck with understanding why something works the way it does, or there may be some computer problem that you have or so forth, on forums don't fast at AI. There are thousands of other learners talking about every lesson and lots of other topics. Besides, it's the most active, deep learning community on the internet by far so definitely register there.
(autogenerated subtitles follow, may contain gibberish/bad format - please proofread to improve - remove this note once proofread)
And start getting involved, you'll get a lot more out of this course. If you do that, so we're going to start by doing some coding. This is an approach. We're going to be talking about in the moment called the top-down approach to study. But let's learn it by doing it, so let's go ahead and try and actually train a neural network. Now, in order to train a neural network, you almost certainly want a GPU GPU of is a graphics processing, a graphics, processing unit. It's the things that companies use to help you play games better. They let your computer render the game much more quickly than your CPU. Okay, we'll be talking about them more shortly, but for now I'm going to show you how you can get access to a GPU. Specifically, you're going to need an NVIDIA GPU, because only NVIDIA GPUs support something called cooter cooter is the language and framework that nearly all deep learning, libraries and practitioners use to do their work. Obviously, it's not ideal that we're stuck with one particular vendors cards and over time we hope to see more competition in this base. But for now we do need an NVIDIA GPU. Your laptop almost certainly doesn't have one unless you specifically went out of your way to buy like a gaming laptop. So almost certainly you will need to rent one, and the good news is that renting access paying by the second or a GPU based computer is pretty easy and pretty cheap. I'm going to show you a couple of options.
The first option, I'll show you, which is probably the easiest, is called Crestle. If you go to kress allcom and click on
(autogenerated subtitles follow, may contain gibberish/bad format - please proofread to improve - remove this note once proofread)
Sign up or if you've been there before sign-in, you will find yourself at this screen, which has a big button. That says start jupiter and another switch called enable GP you. So if we make sure that is set to true the Nabal GPU is on and we click start Jupiter and we click start Jupiter. It's going to launch us into something called Jupiter notebook, Jupiter notebook in a recent survey of tens of thousands of data. Scientists was rated as the third most important tool in the data scientist toolbox. It's really important that you get to learn it well, and all of our courses will be run through Jupiter. Yes, Rachel, you have a question or a comment. I just wanted to point out that you get, I believe, 10 3 hours. So if you wanted to tricresyl out yeah, he might have changed that recently to less hours, but you can check the FAQ or the pricing, but you certainly get some three hours. The pricing varies because this is actually runs on top of Amazon Web Services. So at the moment, it's 60 cents an hour. The nice thing is, though, that you can always turn it turn it on. You know start your Jupiter without the CP, without the GPU running and pay you a tenth of that price, which is pretty cool, so Jupiter notebook is something we'll be doing all of this course in and so to get started here. We're going to find our particular course so we're go to courses and would go to fast, a o2 and there they are.
Things have been moving around a little bit, so it may be in a different spot for you when you look at this and we'll make sure all the information current information is on the website. Now. Having said that, that's you know. The crystal approach is, you know, as you can see, it's basically instant and and easy, but if you've got you know an extra hour,
(autogenerated subtitles follow, may contain gibberish/bad format - please proofread to improve - remove this note once proofread)
Or so to get going and even better option is something called paper space paper space unlike Chris or doesn't run on top of Amazon. They have their own machines and if I click on so here's here's paper space. And so, if I click on new machine, I can pick which one of the three data centers to use so pick that one closest to you so I'll say a West Coast and then I'll say Linux and I'll say Ubuntu 16. And then it says, choose machine and you can see there's various different machines. I can choose from and pay-by-the-hour. So this is pretty cool for 40 cents an hour, so it's cheaper than cresol. I get a machine, that's actually going to be much faster than Crescent 67. Our machine or for 65 cents an hour way way way faster, all right, so I'm going to actually show you how to get started with the paper space approach, because that actually is going to do everything from scratch. You may find, if you trailers, do the 65 cents an hour, one that it may require you to contact paper space to say like. Why do you want it and it's just an anti fraud, the thing so, if you say faster AI there, then they'll quickly get you up and running. So I'm going to use the cheapest one here, 40 cents an hour. You can pick how much draw it. You want and note that you pay for a month of storage as soon as you start the machine up.
Alright, so don't start and stop lots of machines, because each time you pay for that month of storage, I think the 250 gig $ 7 a month option is pretty good, but you only need 50 gig. So if you're trying to minimize the price you can go there, the only other thing you need to do is turn on public IP, so that we can actually log into this, and we can turn off auto snapshot to save the money or not having backups all Right, so if you then click on create your paper space about a minute later, you will find that your machine will pop up. Here is my Ubuntu 1604 machine. If you check your email, you will find that they have emailed you a password, so you can copy that and you can go to your machine and enter your password now to paste the password you would press ctrl shift V or on Mac against Apple shift V. So it's slightly different to normal pasting or of course you can just type it in, and here we are now we can make a little bit more room here by clicking on those little arrows. There can zoom in a little bit, and so, as you can see, we've got like a terminal, that's sitting inside our browser, it's just kind of quite a handy way to do it so now we need to configure this further course and so the way you can Figure it for the course is you type curl, HTTP colon, slash, slash files, dot, fastai, slash setup, slash paper, space, type, okay, and so that's then, going to run a script which is going to set up all of the crudo drivers, special Python, Reaper, Python distribution.
We use called anaconda all of the libraries or other courses and the data we use for the first part of the course ok, so that takes an hour or so and when it's finished, running you'll need to reboot your computer. So to reboot, not your own computer, but your pages paste computer, and so to do that.  So what you'll find is if you've now got an anaconda three directory. That's where your python is. You've got a data directory which contains the data for the first part of this course. First lesson, which is their dogs and cats and you've got a fastai directory, and that contains everything for this course. So what you should do is CD fastai and from time to time you should go get Paul and that will just make sure that all of your fastai stuff is up-to-date and also from time to time. You might want to just check that your Python libraries are up-to-date, and so you can type Condor and update to do that. Alright, so make sure that you've CD into fastai and then you can type Jupiter notebook all right there. It is so we now have a Jupiter notebook servant running and we want to connect to that, and so you can see here it says copy paste this URL into your browser when you connect.
So if you double click on it, then that will actually that will actually copy it for you, then you can go and paste it, but you need to change this local host to be the paper space IP address. So if you click on the little arrows to go smaller, you can see the IP addresses here so I'll, just copy that and paste it where it used to say local host okay. So it's now HTTP and then my IP and then everything else a copied before and so there it is so this is the faster I get repo and our courses are all in courses and in there the deep learning part one is DL one and in there You will find lesson one by Mb, ipython
(autogenerated subtitles follow, may contain gibberish/bad format - please proofread to improve - remove this note once proofread)
Notebook so here we are ready to go depending whether you're using Crestle or paper space or something else. If you check course it's not fast today, I will keep putting additional videos and links to information about how to set up other. You know good Jupiter, notebook providers as well, so to run a cell in Jupiter notebook, you select the cell and you hold down shift and press Enter or if you've got the tool bar showing you can just click on the little Run button. So you'll notice that some cells contain code and some contain text and some contain pictures and some contain videos, so this environment basically has you know it's a it's a way that we can give you access to and a way to run experiments and to kind of Tell you what's going on so pictures. This is why it's like a super popular tool in data science, a data science is kind of all about running experiments. Really so, let's go ahead and click run and you'll see that cell turn into a star the one turn into a star for a moment and then it finished running okay. So let's try the next one. This time, instead of using the toolbar, I'm going to hold down shift and press ENTER, and you can see again it turn into a star and then it said to so. If I hold down shift and keep pressing enter, it just keeps running each so right. So I can put anything I like, for example, one plus one is two.
So what we're going to do is we're going to yes Rachel. This is just a side note, but I wanted to point out that we're using Python 3 here, yes, thank you, pythons are still using Python, mmhmm yeah um and it is important to switch to Python 3. You know now well for Class A I you require it, but you know increasingly, a lot of libraries are removing support for Python thanks Rachel. Now it mentions here that you can download the data set for this lesson from this location if you're using crystal or the paper space script that we just used to set up that this will already be and made available for you. Okay, if you're not you'll, need to wget it as now, Crestle is quite a bit slower than paper space and also it there are some particular things. It doesn't support that we really need, and so there are a couple of extra steps if you're using cresol, you have to run two more cells right, so you can see these are commented out there quite hashes at the start. So if you remove the hashes from these and run these two additional cells that just runs the stuff that the stuff that you only need for crystal I'm using paper space, so I'm not going to run it. Ok, so inside our data. So we set up this path to data, slash dogs, cats, that's pre, set up for you and so inside there you can see here. I can use an exclamation mark to basically say I don't want to run Python, but I want to run bash right.
I want to run shell, so this runs a bash command and the bit inside the curly brackets actually refers, however, to a python variable so inserts that python variable into the batch command. So here's the contents of our folder there's a training set and a validation set. If you're not familiar with the idea of training, sets and validation sets, it would be a very good idea to check out our practical machine learning course, which tells you a lot about this kind of stuff. If, like yeah the basics of how to setup and run machine learning projects more generally, would you recommend that people take that course before this one actually a lot of students? Who would you know, as they went through these, as said they'll they've liked doing them together? So you can kind of check it out and see the machine learning course yeah. They cover with some similar stuff, but all in different directions. So people have done both seen, you know say they find it. They they each support each other. I wouldn't say it's a prerequisite, but you know if I do, if I say something like hey, this is the training set and this is a validation set and you're going. I don't know what that means, at least Google, but do a quick read. You know, because we're assuming that you know the very basics of kind of what machine learning is and does to some extent - and I have a whole blog post on this topic as well.
Okay and we'll make sure that you link to that from Pastor day night and as we just wanted to say in general with fasting, our philosophy is to kind of learn things on an as-needed basis, yeah, exactly don't try and learn everything that you think you might Need first, otherwise, you'll never get around elite learning the stuff. You actually want to learn exactly that shows up in deep learning. I think particularly a lot yes, okay, so in our validation, folder, there's a cat's folder and a dog's folder, and then inside the validation cats. Folder is a whole bunch of JPEGs. The reason that it's set up like this is that this is kind of the most common standard approach for how image classification, datasets, shared and provided, and the idea is that each folder tells you the label. So there's each of these images is labeled cats and each of the images and the dogs folder is labelled dogs, okay. This is how chaos works as well, for example. So this is a pretty standard way to share image classification files, so we can have a look. So if you go plot dot a.m. show, we can see an example of the first of the cats. If you haven't seen this before this is a Python 3.6 format string. So you can google for that. If you haven't seen it, it's a very convenient way to do string formatting and we use it a lot. So there's no cat but we're going to mainly be interested in the underlying data that makes up that cat.
So, specifically, it's an image whose shape that is the dimensions of the array is 198 by 1/7. 9X3. Is it's a three dimensional array plus a quarter rank three tensor and here are the first four rows and four columns of that image. So, as you can see, each of those cells has three items in it, and this is the red, green and blue pixel values between naught and 255.  So that's that that's will be. Our idea is to take these kinds of numbers and use them to predict whether those kinds of numbers represent a cat or a dog based on looking at lots of pictures of cats and dogs. So that's a pretty hard thing to do and at the point in time when this this this data set actually comes from a caracal competition, the dogs versus cats, caracal competition and when it was released in, I think it was 2012. The state of the art was 80 % accuracy, so computers weren't really able to at all accurately recognize dogs versus cats. So let's go ahead and train
(autogenerated subtitles follow, may contain gibberish/bad format - please proofread to improve - remove this note once proofread)
A model so here are the three lines of code necessary to train a model, and so let's go ahead and run it. So I'll click on this on the cell, I'll press shift enter and then we'll wait a couple of seconds for it to pop up and there it goes okay and it's training, and so I've asked it to do three epochs. So that means it's going to look at every image three times in total or look at the entire set of images three times. That's what we mean by an epoch and as we do it's going to print out the accuracy is: that's lasted for three numbers. It prints out on the validation set. Okay, the first three numbers we'll talk about later. In short, they're, the value of the loss function, which is in this case the cross-entropy loss for the training set and the validation set, and then right at the start. Here is the epoch number, so you can see it's getting about 90 % see and it took 17 seconds. So you can see. We've come a long way since 2012 and in fact, even in the competition, this actually would have won the caracal competition of that time. The best in the caracal competition was 98.9 and we're getting about 99 %. So this play surprised you that we're getting a you know: Kaggle winning as of 20 and of 2012 early 2013 kaggle winning image classifier in 17 seconds that and three lines of code, and I think that's because, like a lot of people assume that deep learning takes A huge amount of time and lots of resources and lots of data and as you'll learn in this course that in general, rule isn't true.
One of the ways we've made it much simpler is that this code is written on top of a library we built imaginatively called fastai. The faster a library is basically a library which takes all of the best practices approaches that we can find, and so each time a paper comes out, you know we that looks interesting. We test it out if it works well for a variety of data sets and we can figure out how to tune it. We implement it in fastai and so faster. I kind of curates all this stuff and packages up for you and much of the time, but most the time kind of automatically figures out the best way to handle things so the first day our library is why we were able to do this in just three Lines of code and the reason that we were able to make the faster I library work so well is because it interns it's on top of something called pytorch, which is a really flexible, deep learning and machine learning and GPU computation library written by Facebook. Most people are more familiar with tensorflow than pytorch, because google markets that pretty heavily but most of the top researchers I know nowadays at least the ones that are at Google - have switched across to pytorch. Yes, Rachel and we'll be covering some pie torts later in the course yeah. It's I mean one of the things that hopefully you'll really like about last AI, is that it's really flexible, that you can use all these kind of curated best practices as much as little as you want, and so really easy to hook in at any point and Write your own data augmentation write, your own loss function, write your own network architecture whatever, and so we'll do all of those things in this course. So
(autogenerated subtitles follow, may contain gibberish/bad format - please proofread to improve - remove this note once proofread)
What does this model look like? Well, what we can do is we can take a look at. So what are the? What is the validation set, dependent variable? The Y look like, and it's just a bunch of zeros and ones, okay, so the zeros. If we look at data dot classes, the zeros represent cats, the ones represent dogs you'll, see here, there's basically two objects. I'm working with one is an object, called data which contains the validation and training data and another one is the object, called learn which contains the model right so anytime. You want to find something out about the data. We can look inside data, so we're going to get predictions through our validation set, and so to do that we can call learned predict, and so you can see here the first ten predictions and what it's giving you as a prediction for dog and a prediction for Cat now, the way PI, Torche, generally works and therefore fastai also works - is that most models return the log of the predictions rather than the probabilities themselves, we'll learn why that is later in the course. So for now recognize that to get your probabilities, you have to get e to the power of you'll. See here we're using numpy NP is none play if you're not familiar with numpy. That is one of the things that we assume that you have some familiarity with. So be sure to check out the material on cost. I passed at AI to learn the basics of number.
It's the way that handles all of the fast numerical programming array computation that kind of thing okay, so we can get the probabilities using that using MP, dot X and there's a few functions here that you can look at yourself if you're interested, that's just some flooding Functions that we'll use - and so we can now plot some random, correct images, and so here are some images that it's correct about. Okay, and so remember, one is a dog, so anything greater than 0.5 is dog and zero is a cat. So this is what 10 to the negative 5. Obviously a cat here are some which are incorrect. Alright, so you can see that some of these, which it thinks are incorrect, obviously are just the you know, images that shouldn't be there at all, but clearly this one, which it called a a dog, is not at all a dog. So there are some obvious mistakes. We can also take a look at which cats is it. The most confident are cats which dogs are the most doglike, the most confident dogs, perhaps more. Interestingly, we can also see which cats is that the most confident are actually dogs, so which ones it is it? The most wrong about and same thing for the ones the dog said, it really thinks of cats. And again some of these are just pretty weird. I guess there is a dog in there. Yes, Rachel, I see suit. You want to see more about why you would want to look at your data, yeah sure so yeah.
So finally, I just mentioned the last one we've got here is to see which ones have the probability closest to 0.5. So these are the ones that the model knows. It doesn't really know what to do with, and some of these it's not surprising so yeah I mean this is kind of like always. The first thing I do after I build a model is to try to find a way to like visualize what it's built, because if I want to make the model better, then I need to take advantage of the things that's doing well and fix the things. That's doing badly, and so in this case and off this is the case. I've learned something about the data set itself, which is that there are some things that are in here, that probably shouldn't be, but I've also like it's also clear that this model has room to improve like to me. That's pretty obviously a dog, but one thing I'm suspicious about here is this: image is very kind of fat and short and, as we all learn the way these algorithms work is it kind of grabs, a square piece at a time. So this rather makes me suspicious that we're going to need to use something called data augmentation that we'll learn about learn about later to handle this properly. Okay. So that's it right! We've now built we've now built an image classifier and something that you should try now is to grab some data yourself. Some pictures of two or more different types of thing put them in different folders and run the same three lines of code on them.
Okay and you'll find that it will work for that as well as long as that. They are pictures of things like the kinds of things that people normally take photos of right. So if their microscope microscope pictures or pathology pictures or CT scans or something, this won't work very well as well learn about later. There are some other things we didn't need to do to make that work, but for things that look like normal photos, these you can run exactly the same three lines of code and just point your path variable somewhere else to get your own image classifier. So, for example, one student took those three lines of code downloaded for Google Images, ten examples of pictures of people playing cricket, ten examples of people playing baseball and build a classifier of those images which was new perfectly correct. The same student actually also tried downloading seven pictures of Canadian currency, seven pictures of American currency and again, in that case the model was a hundred percent accurate. So you can just go to Google Images if you like and download a few things of a few different classes and see see what works and tell us on.
(autogenerated subtitles follow, may contain gibberish/bad format - please proofread to improve - remove this note once proofread)
The forum, both your successes and your failures, so what we just did was to train a neural network, but we didn't first of all, tell you what a neural network is or what training means or anything. Why is that? Well, this is the start of our top-down approach to learning, and basically the idea is that, unlike the way math and technical subjects, I usually talk where you learn every little element piece by piece and you don't actually get to put them all together and build your Own image, classifier until third year of graduate school, our approach is to say from the start: hey, let's show you how to train an image classifier and you can start doing stuff and then gradually we dig deeper and deeper and deeper, and so the idea is that Throughout the course you're going to see like new problems that we want to solve so, for example, in the next lesson, we'll look at well what, if we're, not looking at normal kinds of photos, but we're looking at satellite images and we'll see why it is that This approach that we're learning today doesn't quite work as well, and what things do we have to change, and so we'll learn enough about the theory to understand why that happens, and then we'll learn about the libraries and how we can change change things with the libraries To make that work better, and so during the course we are gradually going to learn to solve more and more problems as we do so we all need to learn more and more parts of the library more and more bits of the theory until by the end, We're actually going to learn how to create a world plus neural net architecture from scratch and our own training loop from scratch and so were actually built everything ourselves.
So that's the general approach, yes, Rachel and sometimes also call this the whole game, which is inspired by harvard professor david perkins yeah, and so the idea with the whole game is like. This is more like how you would learn, baseball or music with baseball. You would get taken to a ball game, you would learn what baseball is, you would start playing it, and it would only be years later that you might learn about the physics of how curveball works, for example. Well, with music, we put a instrument in your hand, and you start banging the drum or hitting the xylophone, and it's not until years later, that you learn about the circle of fifths and understand how to construct a cadence example so yeah, so that this is kind Of the approach we're using it's very inspired by David Perkins and other writers of Education, so what that does mean is to take advantage of this as we peel back the layers, we want you to keep
(autogenerated subtitles follow, may contain gibberish/bad format - please proofread to improve - remove this note once proofread)
Like looking under the hood yourself as well like experiment a lot now, because this is a very code driven approach, so here's basically what happens right. We start out looking today at convolutional neural networks for images and then in a couple of lessons, we'll start to look at how to use neural nets, to look at structured data and then look at language data and then look at recommendation system data. And then we kind of then take all of those depths and we go backwards through them in reverse order. So now you know by the end of that fourth piece: you will know by the end of lesson four: how to create a world-class image. Classifier, a world-class structured data analysis program, world-class language, classifier, broad class recommendation system and then we're going to go back over all of them again and learn in depth about like well. What exactly did it do and how to do work, and how do we change things around and use it in different situations for for recommendation systems, structured data images and then, finally, back to language? So that's how it's going to work. So what that kind of means is that most students find that they tend to watch the videos two or three times but not like watch lesson, one two or three times and listen to two or three times in verse and three three times but like they do. The whole thing into end lessons one through seven and then go back and start lesson one again, that's an approach which a lot of people find when they want to go back and understand all the details enough.
That can work pretty well. So I would say you know, aim to get through to the end of lesson. Seven, you know as as quickly as you can, rather than aiming to fully understand every detail from this data. So basically, the plan is that, in today's lesson, you'll learn in as few lines of code as possible with as few details as possible. How do you actually build an image classifier with deep learning to do this to, in this case, say: hey here are some pictures of dogs as opposed to pictures of cats. Then we're going to learn how to look at different kinds of images, and particularly we're going to look at images of from satellites and we're going to say for a satellite image. What kinds of things might you be seeing in that image and there could be multiple things that we're looking at so a multi-label classification problem from there we'll move to something which is perhaps the most widely applicable for the most people, which is looking at what we Call structured data so data about data that kind of comes from databases or spreadsheets. So we're going to specifically look at this data set of predicting sales, the number of things that are sold at different stores on different dates, based on different holidays and and so on and so forth, and so we're going to be doing its sales forecasting exercise.
After that, we're going to look at language and we're going to figure out what this person thinks about the movie is on be given and will be able to figure out how to create. Just like we create image classifiers for any kind of image will learn to create in NLP classifiers, to classify any kind of language in lots of different ways. Then we'll look at something called collaborative filtering, which is used mainly for recommendation systems. We're going to be looking at this data set that showed four different people for different movies. What rating did they give it, and here are some of the movies, and so this is maybe an easier way to think about it is there are lots of different users and lots of different movies and then for each one? We can look up for each user. How much they liked that movie and the goal will be, of course, to predict for user movie combinations we haven't seen before. Are they likely to enjoy that movie or not? And that's the really common approach used for like deciding what stuff to put on your homepage? When somebody's visiting you know what book might they want to read or what film might they want to see or so forth from there? We could have then dig back into language a bit more and we're going to look at actually we're gon na look at the writings of Nietzsche, the philosopher and learn how to create our own Nietzsche philosophy from scratch character by character.
So this here, perhaps that every life values a blood of intercourse when it senses there is unscrupulous who's very right sense to impulse love is not actually Nietzsche. That's actually like some character by character, generated text that we built with this recurrent neural network and then, finally, we're going to loop all the way back to computer vision. Again, we're going to learn how not just to recognize cats from dogs how to actually find like where the cat is with this kind of hate map, and we're also going to learn how to write our own architectures from scratch. So this is an example of a resonate, which is the kind of network that we are using in today's lesson for computer vision and so we'll actually end up building the network and the training loop from scratch and so they're. Basically, the the steps that we're going to be taking from here and at each step we're going to be getting into increasing amounts of detail about how to actually do these things yourself. So we've actually heard that from our students of past courses about what they've found and one of the things that we've heard, a lot of students say is that there's been too much time on theory and research and not enough time running the code and even after We tell people about this morning where they still come to the end of the course not and say I wish I had taken more seriously that advice, which is to keep running code, so these are actual quotes from our forum. In retrospect, I should have spent the majority of my time on the actual code and the notebooks see what goes in see what comes out now.
This idea that you can create world-class models in a code, first approach, learning what you need as you go, is very different to a lot of the advice. You're read out there, such as this person on the forum hacker news who claimed that the best way to become an m/l engineer is to learn all of math and C and C++ learn. Parallel programming learn ml algorithms, implement them yourself using plain C and finally start doing ml. So we would say, if you want to become an effective practitioner, do exactly the opposite of of this. Yes, Rachel yeah, I'm just highlighting that this is. We think this is bad advice, and this can be very discouraging for a lot of people to come across this yeah yeah. It's it's it's it's! You know. We now have thousands and more tens of thousands of people that have done this course and have lots and lots of examples of people who are now running. Research, labs or Google brain residence, or you know, have created patents based on deep learning and so forth. Who have done it by doing this course, so the top-down approach works super well now one thing to mention is like we've: we've now already learned how you can actually train a world-class image classifier in 17 seconds. I should mention by the way the first time you run that code. There are two things it has to do that take more than 17 seconds. One is that it downloads a pre trained model from the internet, so you'll see the first time you run it it'll, say downloading model, so that takes a minute or two also the first time you run it.
It pre computes and caches some of the intermediate information that it needs and that takes about a minute and a half as well. So if the first time you run it, it takes three or four minutes to download and pre compute stuff, that's normal. If you run it again, you should find it takes 20 seconds or so so image classifiers. You know you may not feel like you need to recognize cats versus dogs very often on a computer. You can probably do it yourself pretty well, but what's interestingly interesting, is that these image classification, algorithms, are really useful for lots and lots of things, for example alphago, which became which beat the go world champion. The way it worked was to use something at its heart that looked almost exactly like our dogs, vs. cats, image classification, algorithm. It looked at thousands and thousands of go boards, and at for each one, there was a label saying whether that go board ended up being the winning or the losing player, and so it learnt basically an image classification that was able to look at a go board And figure out whether it was a good group or a bad code board, and that's really the key most important step in playing Gowell is to know which, which move is better. Another example is one of our earlier students who actually got a couple of patterns for this work.
Looked at anti-fraud, he had lots of all of his customers mouths movements because they they provided kind of these user tracking software to help avoid fraud, and so he took the the mouse paths basically of the users on his customers. Websites turn them into pictures of where their mouse moved and how quickly it moved and then built a image classifier that took those images as input and as output. It was was that a fraudulent transaction or not and turned
(autogenerated subtitles follow, may contain gibberish/bad format - please proofread to improve - remove this note once proofread)
Out to go, you know really great results for his company, so image classifiers are like much more flexible than you might imagine so so this is how you know some of the ways you can use deep learning, specifically for image recognition and it's worth understanding that deep Learning is not, you know, just a word. That means the same thing as machine learning right like what is it that we're actually doing here when we're doing deep learning. Instead, deep learning is a kind of machine learning, so machine learning was invented by this guy Arthur Samuels, who was pretty amazing in the late 50s. He got this IBM mainframe to play checkers better than he can and the way he did it was. He invented machine learning. He got the mainframe to play against itself lots of times and figure out which kinds of things led to victories and which kinds of things didn't and use that to kind of almost write. Its own program and Arthur Arthur Samuels actually said in 1962 that he thought that one day the vast majority of computer software would be written using this machine learning approach rather than written by hand by writing the loops and so forth by hand. So I guess that hasn't happened yet, but it seems to be in the process of happening. I think one of the reasons it didn't happen for a long time is because traditional machine learning actually was very difficult and very knowledge and time intensive.
So, for example, here's something called the computational, pathologist or C path from backwater and II back and II back back when he was at Stanford, he's now moved on to somewhere on the East Coast, no Harvard, I think, and what he did was he took these pathology Slides of breast cancer biopsies right and he worked with lots of pathologists to come up with ideas about what kinds of patterns or features might be associated with so long-term survival versus versus dying quickly. Basically, and so he came up with these ideas like well, they came up with these ideas, like relationship between epithelial, nuclear neighbors, relationship between epithelial and stromal objects and so forth, and so they came up with all of these ideas of features. These are just a few of the hundreds that they thought of and then lots of smart computer programmers wrote specialist algorithms to to calculate all these different features and then those those features were passed into a logistic regression to predict survival and it ended up working very Well - and it ended up that the survival predictions were more accurate than pathologists own survival, predictions work and so machine learning can work really well. But the point here is that this was a an approach that took lots of domain experts and computer experts.
Many years of work to actually to build this thing right, so we really want something something better, and so, specifically, I'm going to show you something which, rather than being a very specific function with all this very domain-specific, feature engineering, we're going to try and create an Infinitely flexible function, a function that could solve any problem right. It would solve any problem if only you set the parameters of that function correctly, and so then we need or purpose way of setting the parameters of that function, and we would need that to be fast and scalable right now. If we had something that had these three things, then you wouldn't need to do this incredibly time and domain knowledge intensive approach anymore. Instead, we can learn all of those things with this with this algorithm. So, as you might have guessed, the algorithm in question, which has these three properties, is called deep learning or it's not an algorithm, then maybe we will call it a class of algorithms. Let's look at each of these three things in turn, so the underlying function that deep learning uses is something called the neural network. Now the neural network we're going to learn all about it and implemented ourselves from scratch later on in the course, but for now all you need to know about it is that it consists of a number of simple linear layers, interspersed with a number of simple nonlinear Layers and when you in dispersed these layers in this way,
(autogenerated subtitles follow, may contain gibberish/bad format - please proofread to improve - remove this note once proofread)
You get something called the universal approximation theorem and the universal approximation theorem says that this kind of function can solve any given problem to arbitrarily close accuracy. As long as you add enough parameters, so it's actually provably shown to be an infinitely flexible function. Okay, so now we need some way to fit the parameters so that this infinitely flexible neural network solves some specific problem, and so the way we do, that is using a technique that probably most of you will have come across before at some stage called gradient descent And with gradient descent, we basically say: okay well for the different parameters. We have how: how good are they at solving my problem and, let's figure out a slightly better set of parameters and a slightly better set of parameters and j6v follow down the the surface of the loss function downwards. It's kind of like a marble going down, find the minimum and, as you can see here, depending on where you start, you end up in different places. These things a court local minima. Now, interestingly, it turns out that for neural networks, particularly in particular, there aren't actually multiple different local minima, there's, basically just there's, basically just one right or think of it. Another way there are different parts of the space which are all equally good, so gradient. Descent, therefore turns out to be actually an excellent way to solve this problem of fitting parameters to neural networks.
The problem is, though, that we need to do it in a reasonable amount of time, and it's really only thanks to GPUs that that's become possible, so GPUs. This shows over the last few years, how many gigaflops per second can you get out of a GPU? That's the red and green versus a CPU. That's the blue right, and this is on a log scale. So you can see that, generally speaking, the GPUs are about 10 times faster than the CPUs, and, what's really interesting is that nowadays, not only is the Titan X about 10 times faster than the e5 to $ 6.99 CPU, but the Titan X well actually better one To look at would be the GTX 1080i GPU costs about 700 bucks, whereas the CPU, which is 10 times slower costs over $ 4,000. So GPUs turn out to be able to solve these neural network parameter fitting problems incredibly quickly and also incredibly cheaply. So they've been absolutely key in bringing these three pieces together, then there's one more piece which is I mentioned, that these neural network, so you can intersperse multiple sets of linear and then nonlinear layers. In the particular example, that's drawn here, there's actually only one what we call hidden layer, one layer in the middle and something that we learned in the last few years is that these kinds of neural networks, although they do support the universal approximation theorem, they can solve Any given problem arbitrarily closely, they require an exponentially increasing number of parameters to do so, so they don't actually solve the fast and scalable for even reasonable size problems.
But we've since discovered that, if you create add multiple hidden layers, then you get super linear scaling. So you can add a few more hidden layers to get multiplicatively more accuracy, 2ma duplicative, lis, more complex problems, and that is where it becomes called deep learning. So deep learning means a neural network with multiple hidden layers. So when you put all this together, there's actually really amazing what happens? Google started investing in deep learning. In 2012, they actually hired Geoffrey Hinton who's kind of the father of deep learning and his top student Alex Bogusky, and they started trying to build a team that team became known as Google brain and because things with these three properties are so incredibly powerful. And so incredibly flexible, you can actually see over time how many projects at Google use deep learning. My graph here only goes up through a bit over a year ago, but it's I know it's been continuing to grow exponentially since then as well, and so what you see now is around Google that deep learning is used in like every part of the business, and So it's really interesting to see how the this kind of simple idea that we can solve machine learning problems using a an algorithm that has these properties when a big company invests heavily in actually making that happen. You see this incredible growth in how much it's used.
So, for example, if you use the inbox by Google software, then when you receive an email from somebody, it will often tell you here are some replies that I could send for you, and so it's actually using deep learning here to read the original email and to Generate some suggested replies and so like this is a really great example of the kind of stuff that previously just wasn't possible. Another great example would be: Microsoft is also a little bit more recently invested heavily in deep learning, and so now you can use Skype. You can speaking to it in English and ask it at the other end to translate it in real time to Chinese or Spanish and then, when they talk back to you in Chinese or Spanish, Scott will in real-time translated the speech in in their language into English Speech in real-time and again, this is an example of stuff which we can only do thanks to deep learning and something is really interesting to think about how deep learning can be combined with human expertise. So here's an example of low drawing something just sketching it out and then using a program called neural doodle. This is from a couple of years ago then say: please, take that sketch and render it in the style of an artist and so here's the picture that have been created, rendering it, as you know, impressionist painting, and I think this is a really great example of How you can use deep learning to help combine human expertise and what computers are good at so I, a few years ago decided to try this myself like what would happen if I took think learning and tried to use it to solve a really important problem, and So the problem I picked was diagnosing lung cancer.
It turns out, if you can find lung nodules earlier, there's a 10 times higher probability of survival. So it's a really important problem to solve, so I got together with three other people. None of us had any medical background and we grabbed a data set of CT scans. We used to compilation or neural network much like the dogs vs. cats, one we trained at the start of today's lesson to try and predict which CT scans had malignant tumors in them, and we ended up after a couple of months with something with a much lower False negative rate and a much lower false positive rate than a panel with four radiologists, and we went on to build this in a start-up in just into a company called analytic, which has really become pretty successful. And since that time, the idea of using deep learning for medical imaging has become hugely popular and is
(autogenerated subtitles follow, may contain gibberish/bad format - please proofread to improve - remove this note once proofread)
Being used all around the world, so what I've generally noticed is that you know the vast majority of of kind of things that people do in the world currently aren't using deep learning and then each time somebody says: oh, let's try using deep learning to improve performance At this thing they nearly always get fantastic results and then suddenly everybody in that industry starts using it as well. So there's just lots and lots of opportunities here at this particular time to use deep learning to help with all kinds of different stuff. So I've jotted down a few ideas here. These are all things which I know you can use deep learning for right now to get good results from, and you know, are things which people spend a lot of money on or have a lot of. You know important business opportunities, there's lots more as well that these are some examples of things that maybe your company, you could think about applying deep learning for so, let's talk about what's actually going on what actually happened when we trained that deep learning model earlier and So, as I briefly mentioned, the thing we created is something called a convolutional neural network or CNN, and the key piece of a convolutional neural network is the convolution. So here's a great example from our website. I've got the URL up here, explained visually, it's called, and the explained visually website has an example of a convolution kind of.
In fact, this over here in the bottom left is a very zoomed in picture of somebody's face, and over here on, the right is an example of using a convolution on that image. You can see here. This particular thing is obviously finding edges the edges of his head about top and bottom edges in particular. Now, how is it doing that? Well, if we look at each of these are all three by three areas. This is moving over it's taking each three by three area of pixels, and here are the pixel values right for each thing in that 3x3 area and it's multiplying each one of those 3 by 3 pixels by each one of these 3 by 3 kernel values. In a convolution, this specific set of 9 values is called a kernel. It doesn't have to be 9, it could be 4 by 4 or 5 by 5 or 3 by 2 or whatever right, in this case, it's a 3 by 3 kernel and in fact, a deep learning. Nearly all of our kernels are 3 by 3. So in this case the kernel is 1. 1 /, 1, 2. 1. So we take each of the black through white pixel values and we multiply, as you can see, each of them by the corresponding value in the kernel and then we add them all together. And so, if you do that for every 3x3 area, you end up with the values you see over here. On the right hand, side, okay, so very low values become black, very high values become white, and so you can see when we're at an edge where it's black at the bottom and white at the top. We're obviously going to get higher numbers over here and vice versa. Okay, so that's a convolution! So, as you can see, it is a linear operation and so based on that definition of a neural net I described before this can be a layer in our neural network.
It is a simple linear operation and we're going to look much more at convolutions later, including building a little spreadsheet that implements them ourselves.
(autogenerated subtitles follow, may contain gibberish/bad format - please proofread to improve - remove this note once proofread)
So the next thing we're going to do is we're going to add a nonlinear layer. So a non-linearity as it's called, is something which takes an input value and turns it into some different value in a nonlinear way, and you can see this orange picture here is an example of a nonlinear function. Specifically, this is something called a sigmoid, and so a sigmoid is something that has this kind of S shape, and this is what we used to use as our nonlinearities in neural networks. A lot actually nowadays we're nearly entirely use. Something else called a rally or rectified linear unit, a rail. U is simply take any negative numbers and replace them with 0 and leave any positive numbers as they are so, in other words, in code that would be y equals max X, comma, 0, so max X, comma 0 simply says, replace the negatives with 0. Now, regardless of whether you use a sigmoid or a RAL, you or something else, the key point about taking this combination of a linear layer, followed by a element-wise nonlinear function, is that it allows us to create arbitrarily complex shapes, as you see in the bottom right And the reason, why is that - and this is all from Michael Nelson's, neural networks and deep learning com - really fantastic, interactive book as you change the values of your linear functions, it basically allows you to kind of like build these arbitrarily tall or thin blocks and then Combine those blocks together, and this is actually the essence of the universal approximation theorem.
This idea that when you have a linear layer feeding into a non-linearity, you can actually create these arbitrarily complex shapes. So this is the key idea behind why neural networks can solve any computable problem. So then we need a way as we described to actually set these parameters. So it's all very well, knowing that we can move three, a meters around manually to try to create different shapes, but we have some specific shape. We want how do we get to that shape, and so, as we've discussed earlier, the basic idea is to use something called gradient descent. This is an extract from a notebook actually one of the fastai lessons, and it shows actually an example of using gradient descent to solve a simple linear regression problem, but I can show you the basic idea. Let's say you were just you had a simple quadratic. All right - and so you are trying to find the minimum of this quadratic and so in order to find the minimum you start out by randomly picking some point all right. So we say: okay, let's pick, let's pick here and so you go up there and you calculate the value of your quadratic at that point. So what you now want to do is try to find a slightly better point. So what you could do is you can move a little bit to the left and a little bit to the right to find out which direction is down and what you'll find out is that moving a little bit to the left decreases the value of the function. So that looks good right and so, in other words, we're calculating the derivative.
There's a function at that point right, so that tells you which way is down it's the gradient, and so now that we know that going to the left is down. We can take a small step in that direction to create a new point, and then we can repeat the process and say: okay, which way is down now, and we can now take another step and another step and another step, another step, another step. Okay and each time we're getting closer and closer, so the basic approach here is to say: okay, we start we're at some point. We've got some value X, which is our current guess right, that's at time, step n! So then, our new guest at time, step n plus 1, is just equal to our previous guess, plus the derivative right times some small number, because we want to take a small step. We need to pick a small number, because if we picked a big number right, then we say: okay, we know we want to go to the left. Let's jump a big long way to the left. We could go all the way over here and we actually end up worse all right and then we do it again now we're even worse again right. So if you have too high a step size, you can actually end up with divergence rather than convergence. So this number here we're going to be talking about it - a lot during this course and we're going to be writing all this stuff out in code from scratch ourselves. But this number here is called the learning rate. Okay, so you can see here.
This is an example of basically starting out with some random line and then using gradient descent to gradually make the line better and better and better. So what happens when you combine these ideas right, the convolution, the non-linearity and gradient descent, because they're all tiny small, simple little things? It doesn't sound that exciting. But if you have enough of these kernels right with enough layers, something really interesting happens and
(autogenerated subtitles follow, may contain gibberish/bad format - please proofread to improve - remove this note once proofread)
We can actually draw them, so here's the so. This is a really interesting paper by Matt, Siler and Rob Fergus and what they did a few years ago was they figured out how to basically draw a picture of what each layer in a deep learning net Network learned, and so they showed that layer. One of the network here are nine examples of convolutional filters from layer, one of a trained network, and they found that some of the filters kind of learnt these diagonal lines or simple dat or grid patterns. Some of them learnt these simple gradients right and so for each of these filters they show nine examples of little pieces of actual photos which activate that filter quite highly right, so you can see layer, one these learnt, or ever these these are learnt using gradient descent. These filters were not programmed, they will learnt using gradient descent, so in other words, we were learning these nine numbers, so layer two then was going to take these as inputs and combine them together and so layer two had you know. This is like my in kind of attempts to draw one of the examples of the filters in layer two they're pretty hard to draw, but what you can do is say if the each filter, what are examples of little bits of images that activated them and you Can see by layer two we've got basically something that's being activated nearly entirely by little bits of sunset.
Some things, that's being activated by circular objects, something that's being activated by repeating horizontal lines, something that's being activated by corners, so you can see how we're basically combining layer. One features together. So if we combine those features together and again, these are all convolutional filters. Won't through gradient descent by the third layer, it's actually learn to recognize the presence of text. Another filter has learnt to recognize the presence of petals. Another filter has learnt to recognize the presence of human faces right. So just three layers is enough to get some pretty rich behavior so, but by the time we get to layer, five we've got something that can recognize the eyeballs of insects and birds and something that can recognize unicycle wheels. Alright. So so this is kind of where we start with something incredibly simple right, but if we use it as a big enough scale, thanks to the universal approximation theorem and the use of multiple hidden layers in deep learning, we actually get the very, very rich capabilities so That is what we used when we actually trained our little dog vs cat recognizer. Okay. So let's talk more about this dog vs cat recognizer, so we've learnt the idea of like we can look at the pictures that come out of the other end to see what the model is. Classifying well like, as I find badly or which ones it's unsure about, but let's talk about like this key thing I mentioned, which is the learning rate, so I mentioned we have to set this thing.
I just caught it L before the learning rate and you might have noticed, there's a couple of numbers, these kind of magic numbers here, the first one is the learning rate right. So this number is: how much do you want to multiply the gradient by when you're, taking each step in your gradient descent? We already talked about why you wouldn't want it to be too high right, but probably also it's obvious to see why you wouldn't want it to be too low. If you had it too low, you would take like a little step and you'd be a little bit closer and little bits too little step little step, and it would take lots and lots and lots of steps, and it would take too long so setting this number. Well is actually really important and for the longest time this was driving deep learning, researchers crazy, because they didn't really know a good way to set this reliably. So the good news is last year a researcher came up with an approach to quite reliably set the learning rate. Unfortunately, almost nobody noticed so almost no deep learning, researchers I know about actually are aware of this approach, but it's incredibly successful and it's incredibly simple and I'll show you the idea right. It's built into the fastai library as something called @ lr find or the learning rate finder, and it comes from this paper. I was actually 2015 paper. Sorry, cyclic or learning rates for training, neural networks by a terrific researcher called Leslie Smith and I'll, show you Leslie's idea.
So Leslie's ideas started out with the same basic idea that we've seen before, which is, if we're going to optimize something pick. Some random point take its gradient all right and then, specifically, he said, take a tiny, tiny step like tiny step, so a learning rate of like 10 e next, seven all right and then do it again again, but each time increase the learning rate like double it. So then we try like to wean X, 7, 14 X, 7, 18 X, 7, 10 in X, 6 right and so gradually your steps are getting bigger and bigger right, and so you can see what's going to happen, it's gon na like start doing almost nothing right And it's going to then: suddenly the loss function is going to improve very quickly right, but then it's going to step even further again and then even further again, all right. Let's draw the rest of that line to be clear, all right and so suddenly it's then going to shoot off and get much worse right. So the idea, then, is to go back and say: okay at what point did we see like the best improvement? So here we've got our best improvement right, and so I would say: ok, let's use that learning rate right. So, in other words, if we were to plot the learning rate over time, it was increasing like so alright, and so what we then want to do is we want to plot the learning rate against the loss right. So when I say the loss, I basically mean like how accurate is the model. How close? In this case, the loss would be.
How far away is the predictive prediction from the from the goal? Okay, and so if we plotted the learning rate against the loss, we'd say like okay: initially, it didn't do very much right for small learning rates and then it suddenly improved a lot and then it suddenly got a lot worse. So that's the basic idea and so we'd be looking for the point where this graph is dropping quickly right, we're not looking for its minimum point, we're not saying like where was it the lowest, because that could actually be the point where it's just jumped too far. We want at what point: was it dropping the fastest? So if you go so, if you create your learn object in the same way that we did before we'll be learning more about this these details shortly. If you then call LR find method on that, you'll see that it'll start training a model like it did before, but it'll generally stop before it gets to 100 %. Okay, because if it notices that the loss is getting a lot worse, then it'll stop automatically that's what you can see here. It stopped at 84 %, and so then you can call one said that gets you the learning rate scheduler, that's the object, which actually does this learning rate finding and that object has a plot learning rate function, and so you can see here over by iteration. You can see the learning rate alright, so you can see each step. The learning rate is getting bigger and bigger. You can do it this way. We can see it's increasing exponentially.
Another way that Leslie Smith, the researcher suggests, is to do it linearly. So I'm actually currently researching with both of these approaches to see which works best. Recently, I've been mainly using exponential, but I'm starting to look more using Linea at the moment, and so if we then call shed but plot that does the plot that I just described down here. Learning rate versus loss all right and so we're looking for the highest learning rate we can find where the loss is still improving, clearly well right, and so in this case I would say 10 to the negative 2x that 10 to the negative 1. It's not improving right 10 to the negative 3. It is also improving, but I'm trying to find the highest learning rate I can - or it's still clearly improving so I'd say 10 to the negative 2 okay. So you might have noticed that when we ran our model before we had 10 to the negative to 0.01. So that's why we picked that learning rate. So, there's really only one other number that we have to pick, and that was this number three and so that number three controlled. How many epochs did we run so an epoch means going through our entire data set of images and using each each time.  We grab like 64 images at a time and use them to try to improve the model a little bit using gradient descent right and using all of the images once is called one epoch, and so at the end of each epoch, we print out the accuracy and Validation and training loss at the end of the epoch, so the question of how many epochs should be run is kind of the one other question that you need to answer to run these three lines of code, and the answer really to me is like, as many As you like, what you might find happen is, if you run it for too long, the accuracy you'll start getting worse all right and we'll learn about that.
Why later it's something called overfitting right, so you can run it for a while run lots of epochs. Once you see it getting worse, you know how many epochs you can run and the other thing that might happen is if you've got like a really big model or a lot lots and lots of data. Maybe it takes so long, you don't have time, and so you just run enough epochs that fit into the time you have available. So the number of epochs you run. You know that's a pretty easy thing to set. So there are the only two numbers you're gon na have to see it, and so the goal this week will be to make sure that you can run not only these three lines of code on the data that I provided, but to run it on a set Of images that you either have on your computer or that you get from work well that you download from Google and like try to get a sense of like which kinds of images this is seem to work well, for which ones doesn't it work? Well, for what kind of learning rates do you need for different kinds of images? How many epochs do you need? How does the number of the learning rate change the accuracy you get and so forth? Like really experiment, and then you know try to get a sense of like what's inside this data object, you know what are the y-values look like? What are these places mean if you're not familiar with numpy, you know really practice a lot with numpy so that
(autogenerated subtitles follow, may contain gibberish/bad format - please proofread to improve - remove this note once proofread)
By the time you come back for the next lesson. You know we're going to be digging into a lot more detail, and so you'll really feel ready to do that now. One thing that's really important to be able to do. That is that you need to really know how to work with numpy, the faster, a library and so forth, and so I want to show you some tricks in Jupiter notebook to make that much easier. So one trick to be aware of is, if you can't quite remember how to spell something right. So if you're not quite sure what the method you want is you can always hit tab and you'll get a list of methods that start with that letter right and so that's a quick way to find things. If you then can't remember what the arguments are to a method hit shift tab, all right, so hitting shift tab tells you the arguments to the method so shift. Tab is like one of the most helpful things I know so, let's take in P, X, P shift tab, and so now you might be wondering like okay. Well, what does this function do and how does it work? If you press shift tab twice, then it actually brings up the documentation, shows you what the parameters are and shows you what it returns and gives you examples. Okay, if you press it three times, then it actually pops up a whole little separate window with that information. Okay, so shift tab is super helpful. One way to grab that window straight away is, if you just put question mark at the start, then it just brings up that little documentation window now. The other thing to be aware of is increasingly during this course we're going to be looking at the actual source code of fastai itself and learning how it's built and why it's built.
That way, it's really helpful to look at source code. In order to you know, understand what you can do and how you can do it. So if you, for example, wanted to look at the source code for learned, I predict you can just put two question marks. Okay, and you can see it's popped up. The source code right, and so it's just a single line of code, you're very often find that fastai methods, like they they're, designed to never be more than about half a screen full of code and they're, often under six lines. So you can see this case. It's calling predicted with tags, so we could then get the source code for that in the same way, okay and then that's calling a function called predicted with tags, so we could get that documentation for that in the same way, and then so here yeah and then Finally, that's what it does it either rates through a data, loader gets the predictions and then passes them back and so forth. Okay, so question mark question. Mark is how to get source code, but the single question mark is how to get documentation and shift-tab is how to bring up parameters or press it more times to get the docs. So that's really helpful.
(autogenerated subtitles follow, may contain gibberish/bad format - please proofread to improve - remove this note once proofread)
Another really helpful thing to know about is how to use Jupiter notebook well, and the button that you want to know is H. If you press H, it will bring up the keyboard shortcuts palette, and so now you can see exactly what Jupiter notebook can do and how to do it. I personally find all of these functions useful, so I generally tell students to try and learn four or five different keyboard shortcuts a day. Try them out see what they do see, how they work, and then you can try practicing in that session and one very important thing to remember when you're finished, with your work for the day go back to a paper space and click on that little button, which Stops and starts the machine. So after it's stopped you'll see it says, connection closed and you'll see it's off. If you leave it running, you'll be charged for it same thing with Crestle be sure to go to your cresol instance and stop it. You can't just turn your computer off or
(autogenerated subtitles follow, may contain gibberish/bad format - please proofread to improve - remove this note once proofread)
Close the browser you actually have to stop an increase or or in paper space and don't forget to do that or you'll end up being charged until you finally do remember. Okay, so I think that's all the information that you need to get started. Please remember about the forum's okay, if you get stuck at any point check them out, but before you do make sure you read the information on course dot fast at AI for each lesson right, because that is going to tell you about like things that have changed. Okay, so if there's been some change, witch Cupid, a notebook provider we suggest using or how to set up paper space or anything like that and that'll all be on course, doc. Bastard, AI, okay, thanks very much for watching and look forward to seeing you in the next lesson.


BY EDWIN JAHIEL
RAISE THE RED LANTERN (DAHONG DENGLONG GAIGAO GUA) (1991) (China,Taiwan, Hong Kong) *** Directed by Zhang Yimou. Written by Ni Zhen from the 1989 novel by Su Tong ³Wives and Concubines.² Camera, Zhao Fei. Cast: Gong Li, Ma Jingwu, He Caifei, Qan Quifen, Jin Shuyan. An Orion Classics release. In Mandarin with subtitles. 125 minutes. Rated PG. Art Theater.
The first Chinese movie ever nominated for Best Foreign Film in last yearıs Oscars, ³Red Lantern² is the work of Zhang Yimou. He is a prize-winning cinematographer who, within ten years, also established himself, along with Chen Kaige, as the leading filmmaker of the Chinese New Wave called ³The Fifth Generation²of the mid-Eighties. Zhangıs ³Red Sorghum² received the Golden Bear at the Berlin Festival, his ³Ju Dou² the Golden Hugo, at the Chicago Festival, and ³Red Lantern² won five prizes in Venice, including the Silver Lion.
³Raise the Red Lantern² is set in North China during the 1920s. 19-year old Songlian (Gong Li, Zhangıs fetish actress), drops out of the University after one semester and her fatherıs death. Trapped by the lack of money and the impossibility for a woman to become something, she can either marry a poor man or become the fourth concubine of a wealthy one. She opts for the latter.
The man is always anonymously referred to as The Master and the fact that he is never photographed in close-up, underlines his psychological distance from the women. The non-family lives in a geometric compound where each of the concubines has her own house around an inner courtyard.
The filmıs title refers to the red lanterns that, like the light outside a studio where recording is in progress, are lit before the house in which the Master chooses to spend the night. The mistress elected is prepared for this by, among other ceremonies, a foot massage that presumably enhances her libido.
The womenıs numbers correspond to their age. The First Mistress is an older woman who is out of the sexual rivalry. The Second is relatively younger. The Third is a pert, capricious, ex-opera singer.
Number Four, Songlian, is immediately immersed in intrigue. The ³raison dıetre² of the mistresses is solely the competition for the Masterıs favors. In opaque, devious ways, the women, who call each other ³Sister,² compete for this distinction with manoeuvres, schemes, temporary alliances and realignments. Songlianıs sullen, disappointed young maid, who had hoped to become Number Four, adds to the tension.
There is a terrible emptiness in this household. Life is a void that, paradoxically, has an epicenter. This is nightly sex , alluded to but never seen graphically, but even so, sex without any hint of sensuality, feeling or pleasure. Everything revolves around rank, status, the pecking order. No one really does anything. We see no books. We get minimal music, a gramophone, some flutes, a game of mah-jong. The women do eat regularly, yet even the ordering of food is another weapon yet in this war of one-upmanship.
The claustrophobia is visually symbolized by the look of the place. The courtyard is rectilinear and its geometry is repeatedly photographed from a static, impassive point of view. The rooms have a strangely anonymous lavishness which leads to depressing sterility, similar to the non-person nature of the characters.
On the contrary, when the camera takes to the criss-crossing roofs of the compound, it follows their sinuous, bisecting lines, in a way that corresponds to the convoluted game that the ³sisters² plot and play, a game that leads to tragedy.
The movieıs deliberate pace and color coding oddly stress its implacable ³No Exit² theme. The feudal Masterıs mates are also his inmates, locked up in a stockade of gilded cages, and prisoners too of their own narrow, inescapable mentality.
In passing, the film shows the touching Chinese respect for education. Songlianıs single semester at the University gives her prestige. Says one mistress to guests: ³She was a University student, I was just an opera singer.² Another ³sister² asks Songlian : ³Please cut my hair.² ³But I have never done it.² ³You went to the University, youıll do it better.²
³Raise the Red Lantern² takes place well before China raised the Red flag. The Revolution, however, replaced one kind of tyranny with another. This explains why the movie, like Zhangıs earlier ³Ju Dou,² (also about the oppression of women), is a ³pictura non grata,² unreleased in China.
It was made with devious, face-saving financing from Taiwan (via Hong Kong), to a large degree thanks to Hou Hsiao-Hsien, Taiwanıs premier film director. He is listed in the credits as executive producer and may well have influenced the filmıs esthetics.
" Le mauvais gout mene au crime" (Stendhal)
Edwin Jahiel's movie reviews are at http://www.prairienet.org/ejahiel
The review above was posted to the
rec.arts.movies.reviews newsgroup (de.rec.film.kritiken for German reviews).
The Internet Movie Database accepts no responsibility for the contents of the
review and has no editorial control. Unless stated otherwise, the copyright
belongs to the author.
Please direct comments/criticisms of the review to relevant newsgroups.
Broken URLs inthe reviews are the responsibility of the author.
The formatting of the review is likely to differ from the original due
to ASCII to HTML conversion.
Related links: index of all rec.arts.movies.reviews reviews


{% extends "blog/blog_base.html" %}
{% load wagtailcore_tags wagtailimages_tags wagtailroutablepage_tags %}
{% block canonical_url %}
{% endblock %}
{% block article_content %}
{% include "utils/includes/social_share.html" %}
{% include "blog/includes/blog_header.html" %}
{% include "blog/includes/blog_body.html" with body=self.body only %}
{% include "utils/includes/related_links.html" with related_links=self.related_links.all only %}
{% endblock %}


#include <serializablefactory.h>
Factory class for Serializable objects. You can request a new object from the factory by passing in an ID. The factory returns a base-class pointer (allocated on the heap) that points to a new object of the type specified by the ID
Definition at line 23 of file serializablefactory.h.
Public Types | |
| typedef SerializablePtr(*) | CreateObjectFunPtr () |
Static Public Member Functions | |
| static bool | canDeserialize (const std::vector< uint8 > &buffer, int read_pos=0) |
| static SerializablePtr | newObject (std::istream &s) |
| static SerializablePtr | newObject (const std::vector< uint8 > &buffer, int &read_pos_inout) |
| template<class T> | |
| static boost::shared_ptr< T > | newObject () |
| template<class T> | |
| static boost::shared_ptr< T > | newObject (Serializable::ObjectID object_id) |
| static SerializablePtr | newObject (Serializable::ObjectID object_id) |
| static SerializablePtr | newObjectWithStatus (std::istream &s, DeserializationStatusFlags &status_inout) |
| static SerializablePtr | newObjectWithStatus (const std::vector< uint8 > &buffer, int &read_pos_inout, DeserializationStatusFlags &status_inout) |
| static SerializablePtr | newObjectWithStatus (Serializable::ObjectID object_id, DeserializationStatusFlags &status_inout) |
| static CreateObjectFunPtr | registerObjectType (Serializable::ObjectID object_id, CreateObjectFunPtr creation_fn_ptr) |
| static CreateObjectFunPtr | unregisterObjectType (Serializable::ObjectID object_id) |
| typedef SerializablePtr(*) CreateObjectFunPtr() |
Typedef: pointer to function that creates a Serializable object.
Definition at line 163 of file serializablefactory.h.
| static bool canDeserialize | ( | const std::vector< uint8 > & | buffer, | |
| int | read_pos = 0 | |||
| ) | [static] |
Returns true of there is enough data to deserialize the object at read_pos.
First checks if there is enough data for an object header and if so, then determines the data size from the header and checks that there is enough data in the buffer.
| static SerializablePtr newObject | ( | std::istream & | s | ) | [static] |
Create a new SerializableObject from an open input stream.
| s | Input stream to read the serialized object data from. This function assumes the current read position is set to the start of a serialized object in the stream. This method reads one Serializable object from the stream and returns it, updating the stream's read position accordingly. If the object's ID is unrecognised, that object will be skipped over, and a null pointer is returned |
| SerializableError | if reading from the stream failed, or an error occured whilst unserializing the buffer, or the stream is at the end of the file. |
| static SerializablePtr newObject | ( | const std::vector< uint8 > & | buffer, | |
| int & | read_pos_inout | |||
| ) | [static] |
Create a new Serializable object from the contents of a buffer.
| buffer | Buffer of bytes to initialise the object from | |
| read_pos_inout | The index into buffer to start reading from. Updated by this function to point to the position following the object unless an exception is thrown. Left unmodified if the object id is unknown or an exception is thrown. |
| SerializableError | when: insufficient data in buffer, if an error occured when unserializing from the buffer. Caused by malformed data within a packet. read_pos is left unmodified. |
| static boost::shared_ptr<T> newObject | ( | ) | [static] |
Create and return a new, default initialized Object of the specified type.
The object must define the object id as a static constant named THIS_OBJECT_ID.
| SerializableError | if the ID of the new object does not match object_id |
Definition at line 64 of file serializablefactory.h.
| static boost::shared_ptr<T> newObject | ( | Serializable::ObjectID | object_id | ) | [static] |
Create and return a new, default initialized Object with the specified ID.
| object_id | ID of the object to create |
| SerializableError | if the ID of the new object does not match object_id |
Definition at line 50 of file serializablefactory.h.
| static SerializablePtr newObject | ( | Serializable::ObjectID | object_id | ) | [static] |
Creates a new, default initialized Serializable Object with the specified ID.
| object_id | ID of the object to create |
| SerializableError | if the ID of the new object does not match object_id |
| static SerializablePtr newObjectWithStatus | ( | std::istream & | s, | |
| DeserializationStatusFlags & | status_inout | |||
| ) | [static] |
Create a new SerializableObject from an open input stream.
This function assumes the current read position is set to the start of a serialized object in the stream. This method reads one Serializable object from the stream and returns it, updating the stream's read position accordingly. If the object's ID is unrecognised, that object will be skipped over, and a null pointer is returned.
| s | Input stream to read the serialized object data from. | |
| status_inout | Status flags updated to reflect the deserialization status. |
| SerializableError | if the factory for the object id creates an object with a different object id. |
| static SerializablePtr newObjectWithStatus | ( | const std::vector< uint8 > & | buffer, | |
| int & | read_pos_inout, | |||
| DeserializationStatusFlags & | status_inout | |||
| ) | [static] |
Create a new Serializable object from the contents of a buffer.
| buffer | Buffer of bytes to initialise the object from. | |
| read_pos_inout | The index into buffer to start reading from. Updated by this function to point to the position following the object unless there was insufficient or invalid data to deserialize the object. | |
| status_inout | Status flags updated to reflect the deserialization status. |
| SerializableError | if the factory for the object id creates an object with a different object id. |
| static SerializablePtr newObjectWithStatus | ( | Serializable::ObjectID | object_id, | |
| DeserializationStatusFlags & | status_inout | |||
| ) | [static] |
Creates and a new, default initialized Serializable Object with the specified ID.
| object_id | ID of the object to create. | |
| status_inout | DeserializationStatusFlags to possibly update. If the object_id is unknown then the DESERIALIZATION_UNKNOWN_OBJECT_ID bit flag is set. |
| SerializableError | if the ID of the new object does not match object_id |
| static CreateObjectFunPtr registerObjectType | ( | Serializable::ObjectID | object_id, | |
| CreateObjectFunPtr | creation_fn_ptr | |||
| ) | [static] |
Register a Serializable object type.
| object_id | A unique object identifier | |
| creation_fn_ptr | Pointer to a creation function which creates objects of the type associated with object_id |
| SerializableError | under the following invalid conditions:
|
| static CreateObjectFunPtr unregisterObjectType | ( | Serializable::ObjectID | object_id | ) | [static] |
Unregisters an object id.
| object_id | The id that may have been previously registered with registerObjectType() |


This canvas is drawn from the three planar images below:
Benchmark
Try moving the sliders to apply a multiplier to each plane separately.
See source code of this demo.


Implements the methods needed for a pure test double. RSpec::Mocks::Mock includes this module, and it is provided for cases where you want a pure test double without subclassing RSpec::Mocks::Mock.
Extends the TestDouble module onto the given object and initializes it as a test double.
@example
module = Module.new RSpec::Mocks::TestDouble.extend_onto(module, "MyMixin", :foo => "bar") module.foo #=> "bar"
# File lib/rspec/mocks/test_double.rb, line 17 def self.extend_onto(object, name=nil, stubs_and_options={}) object.extend self object.send(:__initialize_as_test_double, name, stubs_and_options) end
This allows for comparing the mock to other objects that proxy such as ActiveRecords belongs_to proxy objects. By making the other object run the comparison, we’re sure the call gets delegated to the proxy target.
# File lib/rspec/mocks/test_double.rb, line 32 def ==(other) other == __mock_proxy end
@private
# File lib/rspec/mocks/test_double.rb, line 37 def inspect "#<#{self.class}:#{sprintf '0x%x', self.object_id}<EMAIL_ADDRESS>end
@private
# File lib/rspec/mocks/test_double.rb, line 49 def respond_to?(message, incl_private=false) __mock_proxy.null_object? && message != :to_ary ? true : super end
@private
# File lib/rspec/mocks/test_double.rb, line 42 def to_s inspect.gsub('<','[').gsub('>',']') end
Generated with the Darkfish Rdoc Generator 2.


SFTSV protein RNA virus :–. endonuclease polymerase domain using structure CBD LACV (). influenza (Figure Figure binding (Supplementary Supplementary product data genome replication mGTP activity cryo-EM position promoter structural used viral {border-style C-terminal SAXS active bunyavirus residues initiation px;} site template #ffffff;} RdRp novo presence map mechanism model protein. Figures Table buffer primer region transcription vRNA NaCl, assay compared conserved core different functional protein, size volume apo-L assays full-length proteins reaction shown similar (residues However, RVFV S). segment terminal GTP denaturing end missing observed separated synthesis S–S). Virol.; X-ray complementary conformation described expression possible products single structurally structures Alignment Appl. additional cap-binding complex contains function interaction involved like pmol primers resulting scattering sites thumb visualized B). Crystallogr.; HEPES(NaOH) LASV cap cap-snatching distinct domain, final hook indicated min presented prime-and-realign priming purified px; relative replication. sequences stability standard termini vitro {font-family (,). (A) (DA) (GE :e. File). PA-C biochemical characterization contain density detected differences domains electron font-size high including interactions linker mobility models palm parts performed resolution ribbon shift small-angle specific (B) (C) (D) (Figures (Thermo (w/v) Acids Biol. D). Na-phosphate SA). affinity beamline bound cap-dependent cells chromatography determination determined divalent double; established fever fingers higher highly indicating insertion ions isolated likely loading loop magnesium model. mutant negative number observe present protein–RNA purification role studies subsequently synthesized thrombocytopenia titration understanding vRBL visible EMBL Healthcare) Healthcare). Leibniz PAGE RNA, RNAs Res.; Therefore, Thermal addition analysis analyzed arenavirus autoradiography based border-color border-width buffer. characterize closely collected column concentrations consistent containing crystal defined degradation development electrophoresis expressed gel gene genomic incubated independent inhibitor known labeled length ligand liquid mGTP. obtained polyacrylamide potential previously provide provided radioactively recently related ring second software solid; summary, supplemented syndrome tested {font-style {font-weight °C. μg/μl (compare A). Acta C). Cryst.; Data Fisher) GuntherS., Hamburg, III ITC Nucleic P-labeled PETRA RNasin Reactions RosenthalM., SB). Sci. Structural Typhoon U/μl added assay. band bold bovine build building bunyaviruses calorimetry cap-analogue case cellular chemically class cleavage cleft comprehensive concentrated concentration conclude conformational contrast courier; demonstrate dependent differ diluted dithiothreitol, domain. em; emerging enzymatic equivalent especially filtered followed form frozen future gels glycerol important inactive include information initiated initiates initio insect insights interacting internally isothermal larger location low-pass low-resolution lower mGTP, manganese min. necessary needed nucleotides order phosphor polymerase. program proposed pt;} published realignment refined reported representation resolved resulted sNSV scanner screen segmented segments sequence serum severe so-called ssRNA strand subsequent thank unable viruses wild-type β-sheet


Provides external JAR dependencies for the 'DatabaseConnector' package.
| Version: | 1.0.0 |
| Imports: | rJava |
| Published: | 2018-05-14 |
| Author: | Martijn Schuemie [aut, cre], Marc Suchard [aut], Observational Health Data Science and Informatics [cph], Microsoft Inc. [cph] (SQL Server JDBC driver), PostgreSQL Global Development Group [cph] (PostgreSQL JDBC driver), Starschema Ltd. [cph] (BigQuery JDBC driver), Oracle Inc. [cph] (Oracle JDBC driver), Amazon Inc. [cph] (RedShift JDBC driver) |
| Maintainer: | Martijn Schuemie <schuemie at ohdsi.org> |
| BugReports: | https://github.com/OHDSI/DatabaseConnectorJars/issues |
| License: | Apache License version 1.1 | Apache License version 2.0 [expanded from: Apache License] |
| Copyright: | See file COPYRIGHTS DatabaseConnectorJars copyright details |
| URL: | https://github.com/OHDSI/DatabaseConnectorJars |
| NeedsCompilation: | no |
| Materials: | README NEWS |
| CRAN checks: | DatabaseConnectorJars results |
| Reference manual: | DatabaseConnectorJars.pdf |
| Package source: | DatabaseConnectorJars_1.0.0.tar.gz |
| Windows binaries: | r-devel: DatabaseConnectorJars_1.0.0.zip, r-release: DatabaseConnectorJars_1.0.0.zip, r-oldrel: DatabaseConnectorJars_1.0.0.zip |
| OS X binaries: | r-release: DatabaseConnectorJars_1.0.0.tgz, r-oldrel: DatabaseConnectorJars_1.0.0.tgz |
| Reverse imports: | DatabaseConnector |
Please use the canonical form https://CRAN.R-project.org/package=DatabaseConnectorJars to link to this page.


TABLE OF INTERACTIONS WITH
METFORMIN
From the French ANSM drug interactions document of September 2016, p. 150
METFORMIN RxNorm: 6809 ATC: A10BA02 A10BD13 A10BD11 A10BD05 A10BD03 A10BD10 A10BD07 A10BD02 A10BD08 |
ALCOHOL (DRINK OR INGREDIENT) |
Increased risk of lactic acidosis when there is acute alcohol intoxication, particularly in cases of fasting or undernourishment, or in cases of liver failure |
Not recommended Avoid taking alcoholic beverages and medications containing alcohol. |
METFORMIN RxNorm: 6809 ATC: A10BA02 A10BD13 A10BD11 A10BD05 A10BD03 A10BD10 A10BD07 A10BD02 A10BD08 |
LOOP DIURETICS CLASS CODE: C03CA-001 |
Lactic acidosis due to the metformin, set into action by a possible functional renal insufficiency, linked to the loop diuretics |
Precaution for use Do not use metformin when the creatinemia is greater than 15mg/l (135 Ámol/l) for men, and 12 mg/l (110Ámol/l) for women. |
METFORMIN RxNorm: 6809 ATC: A10BA02 A10BD13 A10BD11 A10BD05 A10BD03 A10BD10 A10BD07 A10BD02 A10BD08 |
DOLUTEGRAVIR RxNorm: 1433868 ATC: J05AX12 |
Increase of the plasma concentrations of metformin by the dolutegravir and of its undesirable effects of the lactic acidosis type |
Precaution for use --with the patient having moderate renal insufficiency (stage 3a, clearance of the creatinine between 45 and 59 ml/min), clinical and biological monitoring , and possible adjustment of the dosage of metformin |
METFORMIN RxNorm: 6809 ATC: A10BA02 A10BD13 A10BD11 A10BD05 A10BD03 A10BD10 A10BD07 A10BD02 A10BD08 |
IODINATED X-RAY CONTRAST MEDIA CLASS CODE: VO8A-001 |
Lactic acidosis linked to elevated concentrations of metformin in relation to the functional renal insufficiency induced by the radiological exam. |
CONTRAINDICATION The treatment with metformin should be suspended at the time of the radiological exam and only started again 2 days later. |