How a Phone Call Led to OS X And The Return of Steve Jobs
by Charlotte HenryIt was a not particularly interesting phone call between an Apple exec and a midlevel manager at NeXT. However, as Cult of Mac’s ‘Today in Apple History’ noted, it started a chain overs that led to the creation of OS X and the return of Steve Jobs.
Garrett L. Rice’s communication with Ellen Hancock, Apple’s chief technology officer, is the first formal step in a long process. It ultimately leads to Apple buying NeXT, the creation of OS X, and Steve Jobs returning home to the company he co-founded… By November 1996, Jobs was speaking with Amelio again (albeit only very recently). Jobs advised that Be was not the right choice for Apple. The November 25 phone call from NeXT’s Rice presented the option Jobs surely wanted all along: that Apple acquire the rights to put a version of OpenStep on Macs. By early December, Jobs visited Apple HQ for the first time since his ouster. A deal would bring both NeXT and Jobs aboard — the best decision Apple made in years.
Check It Out: How a Phone Call Led to OS X And The Return of Steve Jobs


Responsive web design is a web design method that enables web to fit the screens of different devices automatically, displaying the content in a way that people feel comfortable. This greatly reduces users’ operations like panning, zooming and scrolling when browsing the web.
Mobile first, as the name suggests, means that we start the product design from the mobile end which has more restrictions, then expand its features to create a tablet or desktop version. It's important because it helps to save product design time and improve designers’ productivity, even though it forces designers to pay more attention to the content of a product, which helps them to created neat and practical designs.
Frameworks are a kind of programming abstraction that holds together the user-written code.
A website wireframe is a page schematic or screen blueprint, is a visual guide that represents the skeletal framework of a website. We use it for the purpose of arranging elements to best accomplish a particular purpose.
Desktop Home Page
Mobile Home Page
Desktop blog Page
Mobile blog Page


We provide whole loads of shipping methods to make your consumer journey very easy and happy by providing you items at your doorstep without any extra shipping charges, we have lakhs of happy customers.
Secure Payment System
We provide a convinient and secure payment method of providing electronically generated mails for every orders you make so that there is no difficulty in the payment.
24/7 Support
We not only provide service to Indians but we make sure that people all over the world can easily access our website anytime anywhere without any inconvinience.


about total management?
Total Management is a free, tableless, W3C-compilant web design layout by Template World. This template has bee tested and proven compatible with all major browser environments and operating systems. You are free to modify design to suit your tastes in any way you lika.
We only ask you to not remove "Design by Template World" and the link http://www.templateworld.com from the footer of the template.
If you are interested in seeing more of our free web template designs feel free to visit our website, Template World. We intend to add at least 25 new free templates in the coming month.


{{$file := .Get 0}}
{{ with .Page.File }}
{{ $path := printf "%s%s" .Dir $file}}
{{ with $.Site.GetPage $path }}{{ .Page.Content }}{{else}}
Include : File "{{$path}}" not found !
param was "{{$file}}"
{{ end }}
{{else}}
Include : Current page file does not exist !
param was "{{$file}}"
{{end}}


Ryan Milton
A veteran of the United States Navy and currently a full-stack web developer. Having led teams designed to accomplished
specific tasks, he knows just how to motivate and encourage people from all background to work cohesively. His journey
into tech started with tearing computers apart as a child and desinging gadgets with raspberry pis. His next step
is to become a part of your team and drive your company to it's next milestone.
LinkedIn
| Github
Joey Eisenzimmer
Joey Eisenzimmer has been working most in digital marketing as a project manager. Prior to that, he completed service
in the U.S. Marine Corps, then moved to completing his MBA, and is now training to become a full stack web developer
at Code Fellows. Joey enjoys learning new technologies, building new applications, and leading teams. He considers
himself a life long learner and enjoys bringing his knowledge and skills to organizations and teams whenever possible.
LinkedIn
| Github
David Chambers
David is a veteran of the United States Air Force and is currently a full-stack JavaScript developer. He has more
than 25 years of professional leadership experience building teams in both non-profit and for-profit industries.
He's a big picture, high functioning, passionate individual who wants to see his company, its clients, and partners
succeed by achieving their full potential.
LinkedIn
| Github
Nathan Vail
Nathan is a full-stack JavaScript developer and former U.S. Army Bomb Squad Officer. He has led teams and managed
time, people, and resources on three continents to deliver measurable, national-level results. Nathan is looking
for opportunities to build coding expertise on a collaborative, results-driven team.
LinkedIn
| Github


Tue Dec 17 11:49:07 CET 2013
| Log Message | Time |
|---|---|
| Initial Rule added:sell 0.01 BTC on mtgox if price goes below 74.0 USD | Dec 17,2013 11:49 |
| Initial Rule added:buy 1.9 LTC on btce if price goes below 2.3 USD | Dec 17,2013 11:49 |
| Initial Rule added:sell 0.4 BTC on bitstamp if price goes above 150.4 USD | Dec 17,2013 11:49 |
| Initial Rule added:buy 0.037 BTC on mtgox if price goes above 10.0 USD | Dec 17,2013 11:49 |
| Initial Rule added:buy 0.02 BTC on mtgox if price goes above 1.0 USD | Dec 17,2013 11:49 |
| Initial Rule added:buy 0.11 BTC on btce if price goes above 10.0 USD | Dec 17,2013 11:49 |
| Initial Rule added:buy 0.001 BTC on btce if price goes above 10.0 USD | Dec 17,2013 11:49 |
| Initial Rule added:buy 0.001 LTC on btce if price goes below 50.0 USD | Dec 17,2013 11:49 |
The bot will not execute the query, there is no connection to mtgox | Dec 17,2013 11:49 |
The bot will not execute the query, there is no connection to btce | Dec 17,2013 11:49 |
The bot will not execute the query, there is no connection to Bitstamp | Dec 17,2013 11:49 |
| BitStamp, error while getting permissions: no connection | Dec 17,2013 11:49 |
mtgox offline : Price checker cannot update values. | Dec 17,2013 11:49 |
btce offline : Price checker cannot update values. | Dec 17,2013 11:49 |
bitstamp offline : Price checker cannot update values. | Dec 17,2013 11:49 |


Fullstack Web Developer & Former Optical Store Manager/ Contact Lens Specialist
| Language: | JavaScript, Python, HTML5 and CSS3 |
| Frameworks & Libraries: | Node.js, Express, Flask, Django, ReactJS, Angular 2+, AngularJS, jQuery and Bootstrap |
| Databases: | MySQL, PostgreSQL, SQLite and MongoDB |
| Other: | NPM, Mongoose, AJAX and RESTful Architecture, Git, Scrum Agile Development |
| Exposure: | Ruby on Rails, Backbone.js, RSpec, RxJS, Chai, AWS (EC2, S3), NGINX, Curl |
Hi, I'm Bas Sondag - I moved to the San Francisco Bay Area in 2015. In my previous life I was a manager of an optical store and contactlens specialist for 16 years with the chain Eye Wish Opticiens in the Netherlands.
Immersing myself in a new country was not exciting enough. I also wanted to become a professional software engineer and make a complete career switch. Upon further research I enrolled in Coding Dojo's Onsite Flex Bootcamp.
In 1100 hours I became a fully functional developer in MEAN, Python and Ruby on Rails. It gave me a pretty amazing feeling that I can create web applications from front end to back end all by myself. Check out my bootcamp projects.
Recently I have worked on projects for Privacy In Practice Consulting, Warpcache, Brigadehub and Letters4Animals. I also work as a freelance developer. Looking for developer support on a project? Send me an email.
Feel free to contact me if you want more details about my code, or to provide feedback on my work and projects. I'd love to hear your suggestions.


Location Information
Hours: 6am-7pm local time
Salmon Cookies Seattle
837 Salmon Street
Seattle, WA
(555)555-5555
Salmon Cookies Tokyo
873 Salmon Street
Tokyo, Japan
(555)555-5555
Salmon Cookies Dubai
873 Salmon Street
Dubai, UAE
(555)555-5555
Salmon Cookies Paris
873 Salmon Street
Paris, France
(555)555-5555
Salmon Cookies Lima
873 Salmon Street
Lima, Peru
(555)555-5555


[Update: Now available] Philips debuts compact Android TV for the kitchen w/ Google Assistant
by Ben SchoonPhilips is now selling a 24-inch TV that’s running on Android TV, designed especially for the kitchen, and has a built-in Google Assistant speaker, too. Here’s what you need to know.
Available today for $299, this new 24-inch Philips Android TV is unique on the market. The product isn’t designed for your living room or bedroom, but rather for areas like the kitchen, garage, home gym, and other areas around the home. The device is designed to replace the use of something like a tablet in those rooms.
Like most TVs using Google’s Android TV platform, this new 24-inch option from Philips is on top of the latest interface and offers a smart homescreen, Google Assistant integration, and apps such as YouTube, Google Play Movies, Netflix, and more. It’s unclear if Prime Video is supported.
The TV has a smaller footprint than most by using a centered stand that houses the speakers. The dual speakers are used, of course, for whatever content is being played on screen, but there are also far-field mics on the device that allow it to double as a Google Assistant speaker.
Philips imagines that most people will be using this product in the kitchen for following recipes on YouTube or watching their favorite shows while cooking. To that end, the product has integrated Netflix and YouTube buttons on the included remote. There are also two HDMI plugs on the TV for plugging in other devices.
The company says that this product will be available today from major retailers in the US and Canada, but so far, we can’t find it for sale. We’ll update this article when the product appears online.
Update 11/13: A week after Philips said the TV was set to go for sale, the product has finally surfaced from two retailers. Specifically, both QVC and HSN are offering the TV both for the $299 price tag. These listings also confirm a few more specifications including the 720p resolution of the display, 8W speakers, and the dimensions too.
More on Android TV:
- Nvidia Shield TV (2019) Review: Smaller, cheaper, still the best streaming device
- Xiaomi Mi Box S beta update delivers Android TV w/ Pie, Amazon Prime Video
- YouTube for Android TV rolls out refreshed UI w/ new sections
FTC: We use income earning auto affiliate links. More.


Chapter 4 Docker, Postgres, and R
We always load the tidyverse and some other packages, but don’t show it unless we are using packages other than tidyverse
, DBI
, RPostgres
, and glue
.
4.1 Verify that Docker running
Docker commands can be run from a terminal (e.g., the Rstudio Terminal pane) or with a system()
command. In this tutorial, we use system2()
so that all the output that is created externally is shown. Note that system2
calls are divided into several parts:
- The program that you are sending a command to.
- The parameters or commands that are being sent
stdout = TRUE, stderr = TRUE
are two parameters that are standard in this book, so that the comand’s full output is shown in the book.
The docker version
command returns the details about the docker daemon that is running on your computer.
## [1] "Client:"
## [2] " Version: 18.06.1-ce"
## [3] " API version: 1.38"
## [4] " Go version: go1.10.3"
## [5] " Git commit: e68fc7a"
## [6] " Built: Tue Aug 21 17:21:31 2018"
## [7] " OS/Arch: darwin/amd64"
## [8] " Experimental: false"
## [9] ""
## [10] "Server:"
## [11] " Engine:"
## [12] " Version: 18.06.1-ce"
## [13] " API version: 1.38 (minimum version 1.12)"
## [14] " Go version: go1.10.3"
## [15] " Git commit: e68fc7a"
## [16] " Built: Tue Aug 21 17:29:02 2018"
## [17] " OS/Arch: linux/amd64"
## [18] " Experimental: true"
4.2 Clean up if appropriate
Remove the cattle
and sql-pet
containers if they exists (e.g., from a prior experiments).
if (system2("docker", "ps -a", stdout = TRUE) %>%
grepl(x = ., pattern = 'cattle') %>%
any()) {
system2("docker", "rm -f cattle")
}
if (system2("docker", "ps -a", stdout = TRUE) %>%
grepl(x = ., pattern = 'sql-pet') %>%
any()) {
system2("docker", "rm -f sql-pet")
}
The convention we use in this book is to assemble a command with glue
so that the you can see all of its separate parts. The following chunk just constructs the command, but does not execute it. If you have problems executing a commnad, you can always copy the command and execute in your terminal session.
docker_cmd <- glue(
"run ", # Run is the Docker command. Everything that follows are `docker run` parameters.
"--detach ", # (or `-d`) tells Docker to disconnect from the terminal / program issuing the command
"--name cattle ", # tells Docker to give the container a name: `cattle`
"--publish 5432:5432 ", # tells Docker to expose the Postgres port 5432 to the local network with 5432
" postgres:10" # tells Docker the image that is to be run (after downloading if necessary)
)
# We name containers `cattle` for throw-aways and `pet` for ones we treasure and keep around. :-)
Submit the command constructed above:
## run --detach --name cattle --publish 5432:5432 postgres:10
## [1] "41adc617f8fbf4803eb5288f1cfb2321332a999fe3da408581c020e6bd51f446"
Docker returns a long string of numbers. If you are running this command for the first time, Docker is downloading the Postgres image and it takes a bit of time.
The following comand shows that postgres:10
is still running:
## [1] "CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES"
## [2] "41adc617f8fb postgres:10 \"docker-entrypoint.s…\" 1 second ago Up Less than a second <IP_ADDRESS>:5432->5432/tcp cattle"
4.3 Connect, read and write to Postgres from R
Create a connection to Postgres waiting up to 10 seconds for the database to come up in Docker.
wait_for_postgres <- function(seconds_to_test){
for (i in 1:seconds_to_test) {
db_ready <- DBI::dbCanConnect(RPostgres::Postgres(),
host = "localhost",
port = "5432",
user = "postgres",
password = "postgres")
if ( !db_ready ) {Sys.sleep(1)}
else {con <- DBI::dbConnect(RPostgres::Postgres(),
host = "localhost",
port = "5432",
user = "postgres",
password = "postgres")
}
if (i == seconds_to_test & !db_ready) {con <- "it's not there"}
}
con
}
con <- wait_for_postgres(10)
Show that you can connect but that Postgres database doesn’t contain any tables:
## character(0)
Write mtcars
to Postgres
List the tables in the Postgres database to show that mtcars
is now there:
## [1] "mtcars"
## [1] "mpg" "cyl" "disp" "hp" "drat" "wt" "qsec" "vs" "am" "gear"
## [11] "carb"
Download the table from the DBMS to a local data frame:
| mpg | cyl | disp | hp | drat | wt | qsec | vs | am | gear | carb |
|---|---|---|---|---|---|---|---|---|---|---|
| 21.0 | 6 | 160 | 110 | 3.90 | 2.620 | 16.46 | 0 | 1 | 4 | 4 |
| 21.0 | 6 | 160 | 110 | 3.90 | 2.875 | 17.02 | 0 | 1 | 4 | 4 |
| 22.8 | 4 | 108 | 93 | 3.85 | 2.320 | 18.61 | 1 | 1 | 4 | 1 |
| 21.4 | 6 | 258 | 110 | 3.08 | 3.215 | 19.44 | 1 | 0 | 3 | 1 |
| 18.7 | 8 | 360 | 175 | 3.15 | 3.440 | 17.02 | 0 | 0 | 3 | 2 |
| 18.1 | 6 | 225 | 105 | 2.76 | 3.460 | 20.22 | 1 | 0 | 3 | 1 |
4.4 Clean up
Afterwards, always disconnect from the DBMS, stop the docker container and (optionally) remove it.
dbDisconnect(con)
# tell Docker to stop the container:
system2("docker", "stop cattle", stdout = TRUE, stderr = TRUE)
## [1] "cattle"
# tell Docker to remove the container from it's library of active containers:
system2("docker", "rm cattle", stdout = TRUE, stderr = TRUE)
## [1] "cattle"
If we stop
the docker container but don’t remove it (with the rm cattle
command), the container will persist and we can start it up again later with start cattle
. In that case, mtcars
would still be there and we could retrieve it from R again. Since we have now removed the cattle
container, the whole database has been deleted. (There are enough copies of mtcars
in the world, so no great loss.)


{{ forms.text(form.gluu_fqdn) }} {{ forms.text(form.country_code) }} {{ forms.text(form.state) }} {{ forms.text(form.city) }} {{ forms.email(form.email) }} {{ forms.text(form.org_name) }} {{ forms.password(form.admin_pw) }} {{ forms.password(form.admin_pw_confirm) }} {% if persistence_backend in ("hybrid", "ldap") %} {{ forms.password(form.ldap_pw) }} {{ forms.password(form.ldap_pw_confirm) }} {% endif %} {{ forms.radio(form.migration_enabled) }} {% set hide_element = False if form.migration_enabled.data == 'Y' else True %} {% if form.migration_enabled.data == 'Y' %}


Broaden the genres of music you listen to!
Get introduced to new artists, songs, and releases that match your music preferences.
Compare your music tastes with other friends and match with people who are similar to you!
See how your recent track features measure up to your friends' playlists.
Ever wonder how your playlist stacks up?
Measure your favorite tracks by their energy, danceability, and more!
Recent Trends is a section to analyze the top 10 recent songs. Features are measured on a 0.0 - 1.0 scale.
Top Tracks is a section to analyze the top 10 tracks you listen to. Features are measured on a 0.0 - 1.0 scale.
Top Artists are the top 10 artists you listen to on a daily basis. Take a look at who some of your favorite artists are!


Sign Up
Find Fellow Explorers
MEAN Google Maps API
List Your Favorite Hidden Urban Landscapes
Username
All fields required
Location Type
Public Park
Urban Landscape
Foresty Wildery Goodness
Appreciation Time (Minutes)
Favorite Scenic Feature
Latitude
Longitude
Is the Location Verified
Submit


|
rotten.com
Q. Where do you get your pictures? A. Everywhere! People send them to us, sometimes we buy them, people hunt for pictures for us, and also we acquire reference materials that contain them. Occasionally, we will even take pictures ourselves.
Q. Are they real? A. Yes, the pictures that appear to be dead bodies are in fact real. If a picture is represented as humor, such as the "Circular File" exhibit, then anything goes. We see a lot of fake pictures, and can spot them fairly easily. Real pictures of this nature aren't particularly rare; they are merely hidden from the public in most cases.
Q. Why do you do it? A. Our staff loves to upset people, it keeps our stools regular. Since the site has actually acquired a life of its own, it really is out of our hands. Well, that's true until we switch to an adult contemporary music format.
Q. Aren't you ashamed? A. No, not really. We don't really show anything that the major television networks don't show you, it's just a matter of degree. Perhaps when CNN admits its shame, so will we. They make a fortune on the misery of fellow man, and few complain about that.
Q.  But there isn't any part of the Bible that has anything to say about running this kind of website. People tell us this, we ask them to find supporting verses (we will even let them use Old Testament fire and brimstone) and then we never hear from that person again. Because, there simply are no Bible passages to support this viewpoint.
Q. Oh I love your site but I'm offended by a particular picture. Please take it down. A. Different people have different triggers, so we will often receive a complaint like this. Well, many of our pictures receive complaints like this; honoring them all would make us boring. If you send us such a message, it isn't likely we will listen to you. Just move along.
Q. Do people claim to find people they know in pictures? A. Yes, from time to time that happens. Sometimes we know exactly where a picture came from, and someone in a totally different place will claim that a particular picture is a relative. In fact multiple people have claimed to know the circumstances of the exact same picture, and this is of course nuts. If you see someone you think you know, it is highly likely that it isn't that person.
Q. Can't you provide more details about pictures? A. Often, unfortunately, we can't. When we end up with a police photo, or whatnot, there is generally no accompanying documentation. It's great when we do have such information, but because our sources are secretive, shaky, or even completely anonymous, the best that we can do is just show the picture.
Q. Do you ever take stuff down for legal reasons? A. Yes. If a sufficiently compelling argument is put forward by a duly licensed attorney, we have been known to remove images. Is that selling out? Maybe. It just isn't worth fighting most battles, particularly with Goliath corporations. We'll just put up different pictures. Any demand to remove pictures had better have at least some merit though; we haven't removed all pictures or features that have been demanded of us.
Q. Why aren't you in jail already? A. The founding fathers of the United States had the foresight to provide us with loopholes to excuse our bad behavior. By not violating any criminal codes, somehow, we manage to stay out of jail.
Q. Is Rotten a hate site? A. No, of course not. We love everyone! Especially you! There might be a bit of self loathing mixed in however. Our self esteem committee is working on that one.
Q. Where can I submit pictures? A. You can email digital pictures to<EMAIL_ADDRESS>High quality JPG's are the preferred format, but any format will be taken if necessary. If you have original pictures or materials to send: Rotten.com, PO Box 4436, Mountain View CA, 94040, USA. We love receiving your weird photos, zines, or artwork in the snail mail. Good stuff gets a tshirt in return; mediocre stuff gets laughed at!
Q. Do you have tshirts? A. Yes, finally. Look here!
Q. We want to advertise, do you accept banners? A.
Q. Can I link to rotten.com? A. Sure! You don't have to ask. We encourage everyone and anyone to link to us and spread the word. Spraypaint our URL on city buses, carve it into school desks with your switchblades, and splash it on hacked webpages. Free tshirts to anyone with rotten.com tattoos.
Q. What happened to the story about Princess Diana? A. It's here, we took the link off the main page because it's old news now. Really, really old news.
Q. I have a picture of Kurt Cobain! [enclosed] A. It is highly unlikely that the picture is of Kurt Cobain, since we had a copy of it long before he ever killed himself.
Q. How do you think someone would feel if their dead body was posted up on the internet? A. I would think they feel nothing, what with being dead and all.
Q. Do you do interviews? A. Rarely. If you are a well established organization and want to fly me somewhere, I might do one. Up until this point I have shunned nearly all media attention -- exception of the New York Times -- but I think I am more open to this sort of discussion than before.
Q. Are you single? A. Well, DUH.
Q.  I am going to notify AOL if you dont write back! A. I'm sure AOL would love to hear from you.
Q. I don't understand how you people can continue live with yourselves after condoning a site which praises death, murder, and suicide. A. We praise murder and suicide? That's news to us. Perhaps you are delusional. Therapy?
Q. Why do you put up a website that are shown perforate people by pistols and shotguns? [disjointed English in an email from Germany] A. Why do you live in a country that exterminated Jews?
Q. I have been to your web site a few times and I don't see how you can do that to people who have passed on, especially children! I think the elephantis and things that people may have never seen like that are pretty nasty but interesting none the less. I think what you are doing is horrible how could you show these dead bodies "no" how dare you show these dead bodies. Why are you showing them for people to have a good laugh or for people to get grossed out. I accidentally came across the dead child photo and it almost brought me to tears. My sister in law showed me, before I could reject, the picture of the dead child with his jaw ripped off, I lost sleep for three days not only because of what happened to that poor child but that some sick person would publicise something like that. I hope you realize what you are doing is wrong before you meet your maker, he may not be very forgiving of this. A. Why on earth do you keep visiting if you find it so upsetting?
Q. Hello!  The President was giving the sign of "I Love You". If you ask any intelligent person who has ever studied American Sign Language to any degree will tell you that it is no the sign of satan. It is "I Love You". The sign of satan requires that the thumb be tucked. If you will notice in you carefully taken photo, it is sticking out! This otherwise is an awsome site and I visit often and refer friends to it. I realize you must have had a brain-fart. I happen to be fluent in sign language by the way. A. We've received many dozens of complaints from sign language afficionados over the years regarding that picture, we just leave that picture up to irritate them. It works very well.
Q. I was wondering how on earth your site hasn't been banned, and how on earth you people sleep at night knowing that your opening giving the world access to this brutal imagery you have on display. What you are showing is quite blatantly illegal. I guess you people are intelligent as I understand this site has been around for some time and until now you have gotten yourselves out of trouble when neccessary. I feel I should warn you that I intend to investigate ways in which I can help have this site closed down. The problem is that anyone could stumble across this site accidently, like I did, and that includes kids, and besides anyone who takes pleasure in this depravity needs help, which I am sure includes all those involved at your site. I would appreciate a response to this in which I want you to attempt to justify your ideas and site content to me.  A. Our content is clearly labelled for what it is -- and this is not required under *any* U.S. law, nor can it be -- and also, we don't have any pictures on our main page, also not required by law. If you go to our page and click on a link, it's your fault, you asked to see it. Now as to the fact that you don't like the pages, I'm sorry. It is not a crime. We still do have the 1st amendment. These pictures aren't any different than what you see in mainstream media, just we show you a lot of them at once. So while we will continue to wallow in our depravity, you'll just have to lie awake nights. Oh well! Cheers.
Q. Hi, I just wanted to ask you obviously very sick, thoughtless human beings whether God plays a role in your life at all. The immoral, sick, disgusting, corruptive, horrible, sinful, evil stuff that you put up for all to see shows me the horror that exists within the human mind. Do you not want something better for yourselves?  Why would you create something as horrible as your website and let other human beings have access to it when you very well should know what a vulnerable existance we have? Anything bad can pollute and influence and corrupt and hurt us. This is the existance that we have created for ourselves but it is not the existance God wants us to live. He wants us to be happy, to show love and kindness to one another, to help one another out whenever we can, and what you're doing is sick and disturbing and only takes human existance further and further away from He who had put us here. This world and existance is not ours, so please be more kind and loving and respectful with your actions.  Please get rid of it and start spreading LOVE and beauty with all that you do. Thank you and remember, God loves you so please try and make Him proud of us and worthy of having been brought into existance. And don't forget....you have been touched by an angel. A. This truly does bring a tear to my eye. In fact I am reminded this moment of Psalm 137:8, "O Daughter of Babylon, doomed to destruction, happy is he who repays you for what you have done to us - he who seizes your infants and dashes their brains against the rocks." Rejoice in the Lord.
In particular, email is subject to ridicule; we'll also be going through old email looking for a few gems.
|


This dialogue allows you to add and edit HTTP breakpoints.
A breakpoint is defined by the following fields:
String
is checked in the
HTTP message: URL
, Request Header
, Request Body
,
Response Header
, or Response Body
.String
is interpreted, Regex
or Contains
, for regular expression or exact match, respectively, in the Location
.
The regular expression does not need to match the whole content of the Location
.Match
should be
the inverse.String
should be
ignored.
Note: ZAP will warn and prevent adding breakpoints with a fragment identifier component (#
), if
the breakpoint has match Contains
and location URL
. Such breakpoint would not work because the
fragment identifier is not sent to the server.
| History tab | 'Break...' right click menu item | |
| Sites tab | 'Break...' right click menu item |
| UI Overview | for an overview of the user interface | |
| Dialogs | for details of the dialogs or popups |


Outline
Introductions
TypeScript for the New Programmer
Getting Started Part 1: Customized Introductions to TypeScript
TypeScript for the New Programmer
- What is JavaScript?
- Runs everywhere
- Untyped, error-prone language
- Super common
- What is TypeScript?
- Adds types to JavaScript
- Your existing code works
- What's a type? (For newbies)
- A type is a kind of value
- Types implicitly define what operations make sense on them
- Lots of different kinds, not just primitives
- We can make descriptions for all kinds of values
- Inference 101
- Examples
- TypeScript can figure out types most of the time
- Two places we'll ask you what the type is: Function boundaries, and later-initialized values
- Co-learning JavaScript
- You can+should read existing JS resources
- Just paste it in and see what happens
- Consider turning off 'strict'
TypeScript for the JavaScript Programmer
- What is TypeScript?
- What's a type? (For newbies)
- checkJs / ts-check
- Try this out on your existing files, see what happens!
- You can use JS Doc if you like
- Inference 101
- All JS is TS
- What this means (behavior)
- What this means (type errors)
- c.f. https://stackoverflow.com/questions/41750390/what-does-all-legal-javascript-is-legal-typescript-mean
TypeScript for the Java/C# Programmer
- Rethinking the OOP mold: Classes, namespaces
- OOP isn't mandatory in JS - no static classes, etc
- Therefore, classes will come late in this guide
- But we still got your back
- Preferred patterns - unions, etc vs inheritance
- JSON/POJOs as a data container vs classes
- Rethinking the OOP mold: Structural Types
- Types are a domain of values, not a class or built-in primitive
- Structural types are structural, yes, structural, see, structural
- Classes are structural
- Classes are structural
- Inference 101
- Co-learning JavaScript
TypeScript for the Functional Programmer
- Welcome Home
- Inference 301
TypeScript for the Veteran Programmer
- --> right to page 1
Chapter 1: Basics
The What & Why of Static Type Checking
- TypeScript models runtime behavior
- e.g. typeof operator exists
- Some operations make sense, some don't
- We try to catch nonsense operations early
- "static" means "before your code runs"
- Detect ahead of time whether your program makes sense
- "Checking" means "detecting errors"
- Examples of typo-catches
- Examples of wrong-value errors
- Examples of invalid operations
- Sometimes code you meant to write looks wrong; we'll tell you how to inform TS what you meant to do
- Tooling: Your editor will be extremely helpful now
- Completion, navigation, etc
- Only suggest things that make sense
What happens when I run tsc?
- Type checking
- Type errors won't block emit
- Feel free to run the JS anyway
- Remove type annotations
- TS doesn't add runtime checking
- TS won't cause your program to not work at runtime
- Downleveling
- Briefly explain ES3/5/6/+
Getting set up
- Install editor of your choice (VS Code or VS Code)
- checkJs / ts-check
- tsc --init
- Basic compiler options: strict, target, lib, module
- How to decide these settings
Chapter 2: Everyday Types
Built-in Types
- string, number, boolean, any, and array
- Show inference and rules
- Examples of allowed and disallowed operations
- Type annotations on variable declarations
Functions
- Parameter type annotations
- Return type annotations and inference
- Inference in function expressions
Object Types
- Anonymous object types
- Type aliases and Interfaces
Union Types
- Simple concept: "This or that"
- Unions of primitives
- Introduce concept of type narrowing and demonstrate it with typeof
Type assertions
- Sometimes you know more than the checker
e as T
vs<T>e
- Not all assertions are valid
Literal Types
- String literals
- Numeric literals
- Boolean's just an alias for
true
|false
- Combine these with unions for e.g.
"GET" | "POST"
- Why do I have to sometimes write
"foo" as "foo"
null and undefined
- --strictNullChecks: turn it on
- Undefined in optional parameters and properties
- Demonstrate basic narrowing to check for them
- Postfix unary
!
Where do types come from?
- Built-in (lib.d.ts)
- Inference
- Your own types
- Packaged types
- DefinitelyTyped
Chapter 3: TypeScript Understands Code
Introduction: Narrowing
Examples of built-in narrowing
- typeof
- instanceof
- truthy/falsy
- equality
Discriminated unions
- Combining literal types with unions yields a powerful pattern
- Show an example
- Show narrowing with the discriminant
- never
User-defined type guards
Chapter 4: More on Functions
Function type expressions
Call/construct signatures
Generics
- "Passing a value through"
- Stress that a type parameter should always appear in 2 places
- Generic constraints
- Using
T
in the body
Overloads
- How to write them
- The implementation signature is not visible
- When not to write them
- Union types
- Generics
- Optional parameters
- etc
Rest and destructuring parameters
- Where do type annotations go?
Other types to know about
- void, object, unknown, never
Chapter 5: More on Object Types (Daniel)
Optional properties
interfaces and type aliases
- Differences; pros/cons
Generic object types
- Arrays - you've been using them all along
- ReadonlyArray
- Always manifest the type parameter
readonly modifier
Tuples
Index signatures
- numeric
- string
Chapter 6: Types from Extraction (Ryan)
typeof (in type positions)
keyof
- Talk about what it does with index signatures
Indexed access types
import
Chapter 7: Types from Transformation (Daniel)
Introduction
Conditional types
Mapped types
- Built-in mapped types ReadOnly, Exclude, etc
Intersection types
- Talk about nominal/tagging
Chapter 8: Classes (Ryan)
Basics
- Hey it's just ES6
- Methods and properties
Class heritage
- inherits
- ES6 thing again
- We don't have inherited typing yet
- implements
- Again, just a check
- Be wary of anys
public, private, protected
Generics
Static
- Instance type parameters
Managing
this
- Arrow functions
this
parameters- The
this
type this
-related flags
Parameter properties
Chapter 9: Modules (Ryan)
Overview of Choices
- ES6 (read MDN)
- CommonJS
- AMD
- SystemJS
- UMD
- See the appendix because oh my god
Import forms
Synthetic defaults
Module resolution
WHERE TO PUT THIS
Why do I have to sometimes write
"foo" as "foo"
BigInt
Chapter 10: Enums (Daniel)
Numeric Enums
- Oh god there are so many kinds
String enums
- Compare and contrast with literal unions
- Consider not using them
Chapter 11: Namespaces (Daniel)
Basics
- Actually two kinds (instantiated vs not)
Merging
Appendix: Commandline flags (Ryan)
Appendix: Symbols / unique symbols
Appendix: JS Doc in lieu of .ts
Appendix: Decorators
Appendix: Project references
Appendix: Writing Definition Files (Ryan)
Basics
Declaration Merging
UMD Globals
Appendix: JSX with React (Ryan)
Appendix: Module interop (Daniel)
WTF is even going on
allowSyntheticDefaultImports
esModuleInterop


{% extends 'nt_training/base.html' %} {% block title %}About{% endblock %} {% block content %}
The Nottingham New Theatre's training records have been stored in various different paper systems for as many years as the theatre has had training. This website provides a way to search through that, browse easily, and be more accessible than a binder of paper in the tech box or workshop.
Whether you're an experienced techie, a seasoned pro with a drill, a director, brand new to the theatre, or anything in between, this website will hopefully give you a better idea of how tech and the workshop at the theatre works, and serve as a reminder for anything you've forgotten.
Being trained in something gives everyone - fellow backstage people, as well as directors, producers, and so on - something to see how you're getting on. If you're looking to become a lighting designer, or want to get involved with the designing and construction of set, having the training ticked off is a sure fire way to let everyone know you know your stuff.
We try to make training as fun and practical as we can so that everyone involved enjoys themselves as much as we enjoy doing the shows themselves. (Or at least almost that much.)
Just ask! The various committee members involved in training are happy to explain pretty much any area of training to anyone who asks, provided we're not already up a ladder with five minutes until the show opens!
Get in touch with one of the emails below for the relevant person for the area that you want to be signed off in. They'll ask you a few questions to check you know your stuff - fairly informally, don't worry - and fill in any gaps for you.


{% extends "layout/base.html" %} {% block content %}
{{myresult}}
Similar item postings removed in two days:
HISTOGRAM OF POSTED PRICES FOR ADVERTS REMOVED IN TWO DAYS


Front End Developer
I am Yansong shi who is a Front End Developer and is passionate of designing beautiful website and satisfying all requirements of customers for their website.
Experience with Front end technologies like HTML, CSS, JavaScript and modern JS framework especially in React.js and related technologies
Experience with Back end technologies like Node.js and Java
Experience with Database like MonogoDB and MySql
IT Support
IT Support
useryansong@gmail.com


We are unable to send the invites out.
You are able to invite and remove contributors. Only the lead organisation and lead applicant cannot be removed.
To change the lead applicant please email Innovate UK<EMAIL_ADDRESS>You are able to invite contributors from your own organisation.


{% extends "base.html" %} {% load i18n %} {% block content %} {% if user.is_authenticated %}
Text-Captcha
Image-Captcha
Unsolved
Solved
What are users supposed to identify in your image captcha?
Please enter one keyword


Blame Stella for breaking your web application
Now in public beta Follow on GitHubStella is an agile testing tool for web applications. Software development has changed in a way that demands a new kind of toolset. Stella integrates quickly into your existing development process to help you run functional and performance tests throughout your product lifecycle.
Not your grandparents' EC2 deployment tool
Now in public beta Follow on GitHubRudy is a development and deployment tool for Amazon EC2. Define infrastructure and processes using a powerful Ruby DSL* and Rudy takes care of the rest (like creating security groups and keypairs).
* domain specific language


Contact
Home
Accomodations
Activities
Travel Arrangements
Pictures
Contact
First name:
Last name:
Email:
Enter text here...
Fantastic Palm Springs
9334 Beach Road
Florida, 58342
(000)-867-5309


About Ankara
The Capital of Turkey: Ankara The capital of Turkey, Ankara offers many eye-catching options for both visitors and residents. The city has buildings wit national values such as Anıtkabir and the building of National Assembly of Turkey as well as several important museums. Ethnography Museum and Museum of Anatolian Civilizations are among the must-see places to understand our history. Ankara, always referred to as the civil servant city, attracts attention with its calm city life. The seriousness of being the capital is felt in every street of the city. In addition Ankara is making a whole new face with new projects. Cafes, restaurants and entertainment areas attract the attention of families as well as young people. The Youth Park is one of the popular meeting points of Ankara. Those who want to take a short walk can visit places such as Güvenpark and Ankara Botanic Park. Atatürk Forest Farm continues to be the lungs of the capital. Although away from the sea, the capital offers a different fun to everyone.
All you wanna learn about Turkey
I hope You liked this video about our country.As you see It shows us how much incredible history,people and natural beauties. It feels like This country will accept you whoever you are like Mevlana says.Because We know how much important being human and living in this world. We respect every religion,every culture,every different people.We accept people whoever they are and whereever they are coming from.It is the most important thing being human.Everybody should come and visit this country when they have time.You missed lots of things in your life without seeing this country.


| 1 | /* | |
| 2 | * Licensed to the Apache Software Foundation (ASF) under one or more | |
| 3 | * contributor license agreements. See the NOTICE file distributed with | |
| 4 | * this work for additional information regarding copyright ownership. | |
| 5 | * The ASF licenses this file to You under the Apache License, Version 2.0 | |
| 6 | * (the "License"); you may not use this file except in compliance with | |
| 7 | * the License. You may obtain a copy of the License at | |
| 8 | * | |
| 9 | * http://www.apache.org/licenses/LICENSE-2.0 | |
| 10 | * | |
| 11 | * Unless required by applicable law or agreed to in writing, software | |
| 12 | * distributed under the License is distributed on an "AS IS" BASIS, | |
| 13 | * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | |
| 14 | * See the License for the specific language governing permissions and | |
| 15 | * limitations under the License. | |
| 16 | */ | |
| 17 | package org.apache.commons.collections4.set; | |
| 18 | ||
| 19 | import java.util.Iterator; | |
| 20 | import java.util.NavigableSet; | |
| 21 | ||
| 22 | import org.apache.commons.collections4.Predicate; | |
| 23 | ||
| 24 | /** | |
| 25 | * Decorates another NavigableSet to validate that all additions | |
| 26 | * match a specified predicate. | |
| 27 | *
 | |
| 28 | * This set exists to provide validation for the decorated set. | |
| 29 | * It is normally created to decorate an empty set. | |
| 30 | * If an object cannot be added to the set, an IllegalArgumentException is thrown. | |
| 31 | *
 | |
| 32 | * One usage would be to ensure that no null entries are added to the set. | |
| 33 | *
 | |
| 34 | * NavigableSet set = | |
| 35 | * PredicatedSortedSet.predicatedNavigableSet(new TreeSet(), | |
| 36 | * NotNullPredicate.notNullPredicate()); | |
| 37 | *
 | |
| 38 | * | |
| 39 | * @since 4.1 | |
| 40 | * @version $Id: PredicatedNavigableSet.java 1686855 2015-06-22 13:00:27Z tn $ | |
| 41 | */ | |
| 42 | public class PredicatedNavigableSet<E> extends PredicatedSortedSet<E> implements NavigableSet<E> { | |
| 43 | ||
| 44 | /** Serialization version */ | |
| 45 | private static final long serialVersionUID = 20150528L; | |
| 46 | ||
| 47 | /** | |
| 48 | * Factory method to create a predicated (validating) navigable set. | |
| 49 | *
 | |
| 50 | * If there are any elements already in the set being decorated, they | |
| 51 | * are validated. | |
| 52 | * | |
| 53 | * @param <E> the element type | |
| 54 | * @param set the set to decorate, must not be null | |
| 55 | * @param predicate the predicate to use for validation, must not be null | |
| 56 | * @return a new predicated navigable set. | |
| 57 | * @throws NullPointerException if set or predicate is null | |
| 58 | * @throws IllegalArgumentException if the set contains invalid elements | |
| 59 | * @since 4.0 | |
| 60 | */ | |
| 61 | public static <E> PredicatedNavigableSet<E> predicatedNavigableSet(final NavigableSet<E> set, | |
| 62 | final Predicate<? super E> predicate) { | |
| 63 |
1
1. predicatedNavigableSet : mutated return of Object value for org/apache/commons/collections4/set/PredicatedNavigableSet::predicatedNavigableSet to ( if (x != null) null else throw new RuntimeException ) → NO_COVERAGE |
return new PredicatedNavigableSet<E>(set, predicate); |
| 64 | } | |
| 65 | ||
| 66 | //----------------------------------------------------------------------- | |
| 67 | /** | |
| 68 | * Constructor that wraps (not copies). | |
| 69 | *
 | |
| 70 | * If there are any elements already in the set being decorated, they | |
| 71 | * are validated. | |
| 72 | * | |
| 73 | * @param set the set to decorate, must not be null | |
| 74 | * @param predicate the predicate to use for validation, must not be null | |
| 75 | * @throws NullPointerException if set or predicate is null | |
| 76 | * @throws IllegalArgumentException if the set contains invalid elements | |
| 77 | */ | |
| 78 | protected PredicatedNavigableSet(final NavigableSet<E> set, final Predicate<? super E> predicate) { | |
| 79 | super(set, predicate); | |
| 80 | } | |
| 81 | ||
| 82 | /** | |
| 83 | * Gets the navigable set being decorated. | |
| 84 | * | |
| 85 | * @return the decorated navigable set | |
| 86 | */ | |
| 87 | @Override | |
| 88 | protected NavigableSet<E> decorated() { | |
| 89 |
1
1. decorated : mutated return of Object value for org/apache/commons/collections4/set/PredicatedNavigableSet::decorated to ( if (x != null) null else throw new RuntimeException ) → NO_COVERAGE |
return (NavigableSet<E>) super.decorated(); |
| 90 | } | |
| 91 | ||
| 92 | //----------------------------------------------------------------------- | |
| 93 | ||
| 94 | @Override | |
| 95 | public E lower(E e) { | |
| 96 |
1
1. lower : mutated return of Object value for org/apache/commons/collections4/set/PredicatedNavigableSet::lower to ( if (x != null) null else throw new RuntimeException ) → NO_COVERAGE |
return decorated().lower(e); |
| 97 | } | |
| 98 | ||
| 99 | @Override | |
| 100 | public E floor(E e) { | |
| 101 |
1
1. floor : mutated return of Object value for org/apache/commons/collections4/set/PredicatedNavigableSet::floor to ( if (x != null) null else throw new RuntimeException ) → NO_COVERAGE |
return decorated().floor(e); |
| 102 | } | |
| 103 | ||
| 104 | @Override | |
| 105 | public E ceiling(E e) { | |
| 106 |
1
1. ceiling : mutated return of Object value for org/apache/commons/collections4/set/PredicatedNavigableSet::ceiling to ( if (x != null) null else throw new RuntimeException ) → NO_COVERAGE |
return decorated().ceiling(e); |
| 107 | } | |
| 108 | ||
| 109 | @Override | |
| 110 | public E higher(E e) { | |
| 111 |
1
1. higher : mutated return of Object value for org/apache/commons/collections4/set/PredicatedNavigableSet::higher to ( if (x != null) null else throw new RuntimeException ) → NO_COVERAGE |
return decorated().higher(e); |
| 112 | } | |
| 113 | ||
| 114 | @Override | |
| 115 | public E pollFirst() { | |
| 116 |
1
1. pollFirst : mutated return of Object value for org/apache/commons/collections4/set/PredicatedNavigableSet::pollFirst to ( if (x != null) null else throw new RuntimeException ) → NO_COVERAGE |
return decorated().pollFirst(); |
| 117 | } | |
| 118 | ||
| 119 | @Override | |
| 120 | public E pollLast() { | |
| 121 |
1
1. pollLast : mutated return of Object value for org/apache/commons/collections4/set/PredicatedNavigableSet::pollLast to ( if (x != null) null else throw new RuntimeException ) → NO_COVERAGE |
return decorated().pollLast(); |
| 122 | } | |
| 123 | ||
| 124 | @Override | |
| 125 | public NavigableSet<E> descendingSet() { | |
| 126 |
1
1. descendingSet : mutated return of Object value for org/apache/commons/collections4/set/PredicatedNavigableSet::descendingSet to ( if (x != null) null else throw new RuntimeException ) → NO_COVERAGE |
return predicatedNavigableSet(decorated().descendingSet(), predicate); |
| 127 | } | |
| 128 | ||
| 129 | @Override | |
| 130 | public Iterator<E> descendingIterator() { | |
| 131 |
1
1. descendingIterator : mutated return of Object value for org/apache/commons/collections4/set/PredicatedNavigableSet::descendingIterator to ( if (x != null) null else throw new RuntimeException ) → NO_COVERAGE |
return decorated().descendingIterator(); |
| 132 | } | |
| 133 | ||
| 134 | @Override | |
| 135 | public NavigableSet<E> subSet(E fromElement, boolean fromInclusive, E toElement, boolean toInclusive) { | |
| 136 | final NavigableSet<E> sub = decorated().subSet(fromElement, fromInclusive, toElement, toInclusive); | |
| 137 |
1
1. subSet : mutated return of Object value for org/apache/commons/collections4/set/PredicatedNavigableSet::subSet to ( if (x != null) null else throw new RuntimeException ) → NO_COVERAGE |
return predicatedNavigableSet(sub, predicate); |
| 138 | } | |
| 139 | ||
| 140 | @Override | |
| 141 | public NavigableSet<E> headSet(E toElement, boolean inclusive) { | |
| 142 | final NavigableSet<E> head = decorated().headSet(toElement, inclusive); | |
| 143 |
1
1. headSet : mutated return of Object value for org/apache/commons/collections4/set/PredicatedNavigableSet::headSet to ( if (x != null) null else throw new RuntimeException ) → NO_COVERAGE |
return predicatedNavigableSet(head, predicate); |
| 144 | } | |
| 145 | ||
| 146 | @Override | |
| 147 | public NavigableSet<E> tailSet(E fromElement, boolean inclusive) { | |
| 148 | final NavigableSet<E> tail = decorated().tailSet(fromElement, inclusive); | |
| 149 |
1
1. tailSet : mutated return of Object value for org/apache/commons/collections4/set/PredicatedNavigableSet::tailSet to ( if (x != null) null else throw new RuntimeException ) → NO_COVERAGE |
return predicatedNavigableSet(tail, predicate); |
| 150 | } | |
| 151 | ||
| 152 | } | |
Mutations | ||
| 63 |
1.1 |
|
| 89 |
1.1 |
|
| 96 |
1.1 |
|
| 101 |
1.1 |
|
| 106 |
1.1 |
|
| 111 |
1.1 |
|
| 116 |
1.1 |
|
| 121 |
1.1 |
|
| 126 |
1.1 |
|
| 131 |
1.1 |
|
| 137 |
1.1 |
|
| 143 |
1.1 |
|
| 149 |
1.1 |


Welcome to my development portfolio. Have a look at my work, enjoy yourself, and if you have any questions you can email me.
This is an example html resume that I styled with css. Someday this will be my resume.
This is a website for a fictitious resort in Milwaukee, Wisconsin. The site spans multiple pages.
I attempted to rebuild the Wikipedia homepage from scratch using Bootstrap and custom css. Content was copy-pasted.
This is a website for a fictitious animal shelter.
This webpage is a blog for the fictitious character Bob Loblaw from the TV show Arrested Development.
I built this webpage as a one-stop reference for the Git commands I was introduced to in the first week of class at Epicodus.
Brad Copenhaver grew up in rural Pennsylvania and now lives in Portland, Oregon. He has traveled the world, lived abroad, and watched every episode of the first twelve seasons of The Simpsons. He loves to write about himself in the third person, and his favorite color is orange. Brad is learning to program at Epicodus in order to further his career in User Experience Design.
Brad earned a master's degree in Human-Computer Interaction from Carnegie Mellon University in 2009. The program was a joint program with the University of Madeira and involved classes at both universities. In case you didn't know, Madeira is a Portuguese, subtropical island in the Atlantic Ocean, and Brad lived there for a year while he was studying.
Way back in the early 2000's, Brad attended Washington and Jefferson College and earned a Bachelor of Science in Psychology. He also minored in Math and IT: New Media studies. His time at W&J taught Brad a lot about life away from home, meeting new people, and working together. Of course that was so long ago that most of those lessons have been revised by his now more mature and jaded point of view.
Brad rides his bike as often as possible for transportation and recreation. It's a great way to stay in shape and save money.
Mt. Tabor Park is one of the biggest parks on the east side of Portland. Brad helps keep the park healthy with the Friends of Mt. Tabor Park Weed Warriors on the last Saturday of every month from March through October.


Seven Dead, 16 More Injured in Fire After Gas Leak in Egypt
The residents got different degrees of burns, with some of the locals saying they might be more casualties than reported by the state media.
At least seven people were killed and 16 more injured during a fire in a Nile Delta village in Egypt, the local media reported on Wednesday.
Witnesses told Reuters that there might be more victims than reported.
The fire started after an attempted theft of gas from the pipeline caused a gas leak in Beheira governorate. It was confirmed by the Petroleum Pipelines Company head, Abdel-Moneim Hafez.
No further details have been provided so far.


| File | Statements | Branches | Functions | Lines | |||||
|---|---|---|---|---|---|---|---|---|---|
| src/ | 100% | 2/2 | 100% | 0/0 | 100% | 0/0 | 100% | 2/2 | |
| src/app/ | 1.22% | 19/1561 | 0% | 0/712 | 1.36% | 5/369 | 1.29% | 19/1471 | |
| src/app/ag-pagination/ | 0% | 0/16 | 0% | 0/6 | 0% | 0/6 | 0% | 0/15 | |
| src/app/article-details/ | 0% | 0/9 | 0% | 0/4 | 0% | 0/6 | 0% | 0/7 | |
| src/app/article-form-page/ | 0% | 0/24 | 0% | 0/2 | 0% | 0/10 | 0% | 0/23 | |
| src/app/article-form/ | 97.06% | 33/34 | 25% | 1/4 | 92.86% | 13/14 | 96.88% | 31/32 | |
| src/app/article-list/ | 91.41% | 149/163 | 43.75% | 7/16 | 83.08% | 54/65 | 92.31% | 144/156 | |
| src/app/article-preview/ | 0% | 0/6 | 100% | 0/0 | 0% | 0/5 | 0% | 0/6 | |
| src/app/assets/scripts/ | 3.5% | 74/2113 | 1.98% | 24/1212 | 2.11% | 12/570 | 1.56% | 23/1478 | |
| src/app/form-elements/ | 0% | 0/26 | 0% | 0/6 | 0% | 0/12 | 0% | 0/26 |


Across: 1. Defying the orthodox, he swapped parts 3 and 5 in musical performance (9) 6. He was among leaders of colonial looting, in value especially (5) 9. Horde delays, every second in a way seeming strange (5) 10. Support from public cracked a cold case (9) 11. Dangerous organic matter in selvedge? (7) 12. As mutiny stirred, city finally fell in wave of destruction (7) 13. Jerk? That's an unfair generalisation! (4) 14. Taking on 9 cool mobs: a state is nigh (10) 18. State laws get nasty when broken by this cryptic author (4 6) 19,18dn. For fabric like denim, I live in an old 25 structure (4 7) 22. Retinal failure is the pits (7) 25. Alright to go back and talk about a city? (7) 26. Camouflaged coin (melted and made ingot) (9) 27. Flash model staple (5) 28. Originally mighty yarns, these heroic stories! (5) 29. These plan creatively and they have good memories (9) Down: 1. Hog swimming in sacred river in 25 (7) 2. Simple things, like coin found among little scraps of litter (9) 3. Only one local leader to rally around: cricket captain (6) 4. 9 cereal caked for want of some strict control (9) 5. Allowed it to sit under silicon core (5) 6. City reconquered by 6 across, with some astronomical cut taken (8) 7. Land of revolution, at heart unafraid: Nirvana! (5) 8. Genius contrived to capture new following (7) 15. Astronmically powerful player, notorious element of 19a 18d (5 4) 16. I sat in loo in such a state? (9) 17. A communication between banks leading to signing contracts (8) 18. see 19 across 20. 9 starred as business people (7) 21. Central 6 down ransacked in heartless cash grab (6) 23. Having one's manners understood? (5) 24. Kicking-out reflex I learnt to control (5)


{{ site.copyright.holder }} ("us", "we", or "our") operates the website (the "Service").
We take your privacy seriously. To better protect your privacy this page details our policies regarding the collection, use, and disclosure of personal data when you use our Service and the choices you have associated with that data.
Our definition of personal data is based on the privacy laws and regulations of the EU, including the General Data Protection Regulation (GDPR).
These are widely regarded as the strongest privacy protections in the world. We consider any information about you or your behavior that can be traced back to you as personal data.
We do not log or share any personal information about our visitors.
This website tracks basic information about their visitors, required by normal internet traffic.
This information includes, but is not limited to, IP addresses, browser details, timestamps and referring pages. None of this information can personally identify specific visitors to this website. The information is tracked for routine administration and maintenance purposes.
Where necessary, this website uses cookies to store information about a visitor’s preferences and history in order to better serve the visitor and/or present the visitor with customized content.
This website does not contain any third party advertisement.
It is hosted on {{ site.privacy_policy.third_party.hosting }}.
We have included links on this website for your use and reference.
We are not responsible for the privacy policies on the above mentioned third parties. You should be aware that the privacy policies of these websites may differ from our own.
Third parties may use cookies, scripts and/or web beacons to track visitors' activities on this website in order to display advertisements and other useful information. Such tracking is done directly by the third parties through their own servers and is subject to their own privacy policies. This website has no access or control over these cookies, scripts and/or web beacons that may be used by third parties.
Learn how to opt out of Google’s cookie usage.
The security of your personal information is important to us, but remember that no method of transmission over the Internet, or method of electronic storage, is 100% secure. While we strive to use commercially acceptable means to protect your personal information, we cannot guarantee its absolute security.
This Privacy Policy is effective as of {{ site.time | date: '%F'}} and will remain in effect except with respect to any changes in its provisions in the future, which will be in effect immediately after being posted on this page.
We reserve the right to update or change our Privacy Policy at any time and you should check this Privacy Policy periodically. If we make any material changes to this Privacy Policy, we will notify you either through the email address you have provided us, or by placing a prominent notice on our website.
For any questions or concerns regarding the privacy policy, please send us an email to {{ site.email }}.


%matplotlib inline
# Common imports
from IPython.display import Image
from pydot import graph_from_dot_data
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
from sklearn.model_selection import train_test_split
from sklearn.tree import export_graphviz
from sklearn.preprocessing import StandardScaler, OneHotEncoder
from sklearn.compose import ColumnTransformer
from pydot import graph_from_dot_data
from sklearn.datasets import load_breast_cancer
from sklearn.svm import SVC
from sklearn.linear_model import LogisticRegression
from sklearn.tree import DecisionTreeClassifier
from sklearn.ensemble import RandomForestClassifier
from sklearn.preprocessing import LabelEncoder
from sklearn.model_selection import cross_validate
import scikitplot as skplt
from sklearn.preprocessing import StandardScaler
import os
# Where to save the figures and data files
PROJECT_ROOT_DIR = "Results"
FIGURE_ID = "Results/FigureFiles"
DATA_ID = "DataFiles/"
if not os.path.exists(PROJECT_ROOT_DIR):
os.mkdir(PROJECT_ROOT_DIR)
if not os.path.exists(FIGURE_ID):
os.makedirs(FIGURE_ID)
if not os.path.exists(DATA_ID):
os.makedirs(DATA_ID)
def image_path(fig_id):
return os.path.join(FIGURE_ID, fig_id)
def data_path(dat_id):
return os.path.join(DATA_ID, dat_id)
def save_fig(fig_id):
plt.savefig(image_path(fig_id) + ".png", format='png')
# Load the cancer data
cancer = load_breast_cancer()
X_train, X_test, y_train, y_test = train_test_split(cancer.data,cancer.target,random_state=0)
print(X_train.shape)
print(X_test.shape)
#Scale the data
scaler = StandardScaler()
scaler.fit(X_train)
X_train_scaled = scaler.transform(X_train)
X_test_scaled = scaler.transform(X_test)
#define methods
# Logistic Regression
logreg = LogisticRegression(solver='lbfgs')
logreg.fit(X_train_scaled, y_train)
print("Test set accuracy Logistic Regression with scaled data: {:.2f}".format(logreg.score(X_test_scaled,y_test)))
# Decision Trees
deep_tree_clf = DecisionTreeClassifier(max_depth=None)
deep_tree_clf.fit(X_train_scaled, y_train)
print("Test set accuracy with Decision Trees and scaled data: {:.2f}".format(deep_tree_clf.score(X_test_scaled,y_test)))
# Support Vector Machine
svm = SVC(gamma='auto', C=100)
svm.fit(X_train_scaled, y_train)
print("Test set accuracy SVM with scaled data: {:.2f}".format(logreg.score(X_test_scaled,y_test)))
# Random forests
#Instantiate the model with 500 trees and entropy as splitting criteria
Random_Forest_model = RandomForestClassifier(n_estimators=500,criterion="entropy")
Random_Forest_model.fit(X_train_scaled, y_train)
print("Test set accuracy with Random Forests and scaled data: {:.2f}".format(Random_Forest_model.score(X_test_scaled,y_test)))
y_pred = Random_Forest_model.predict(X_test_scaled)
skplt.metrics.plot_confusion_matrix(y_test, y_pred, normalize=True)
plt.show()
y_probas = Random_Forest_model.predict_proba(X_test_scaled)
skplt.metrics.plot_roc(y_test, y_probas)
plt.show()
skplt.metrics.plot_cumulative_gain(y_test, y_probas)
plt.show()
The basic idea is to combine weak classifiers in order to create a good classifier. With a weak classifier we often intend a classifier which produces results which are only slightly better than we would get by random guesses.
This is done by applying in an iterative way a weak (or a standard classifier like decision trees) to modify the data. In each iteration we emphasize those observations which are misclassified by weighting them with a factor.
Boosting is a way of fitting an additive expansion in a set of elementary basis functions like for example some simple polynomials. Assume for example that we have a function
$$ f_M(x) = \sum_{i=1}^M \beta_m b(x;\gamma_m), $$where \( \beta_m \) are the expansion parameters to be determined in a minimization process and \( b(x;\gamma_m) \) are some simple functions of the multivariable parameter \( x \) which is characterized by the parameters \( \gamma_m \).
As an example, consider the Sigmoid function we used in logistic regression. In that case, we can translate the function \( b(x;\gamma_m) \) into the Sigmoid function
$$ \sigma(t) = \frac{1}{1+\exp{(-t)}}, $$where \( t=\gamma_0+\gamma_1 x \) and the parameters \( \gamma_0 \) and \( \gamma_1 \) were determined by the Logistic Regression fitting algorithm.
As another example, consider the cost function we defined for linear regression
$$ C(\boldsymbol{y},\boldsymbol{f}) = \frac{1}{n} \sum_{i=0}^{n-1}(y_i-f(x_i))^2. $$In this case the function \( f(x) \) was replaced by the design matrix \( \boldsymbol{X} \) and the unknown linear regression parameters \( \boldsymbol{\beta} \), that is \( \boldsymbol{f}=\boldsymbol{X}\boldsymbol{\beta} \). In linear regression we can simply invert a matrix and obtain the parameters \( \beta \) by
$$ \boldsymbol{\beta}=\left(\boldsymbol{X}^T\boldsymbol{X}\right)^{-1}\boldsymbol{X}^T\boldsymbol{y}. $$In iterative fitting or additive modeling, we minimize the cost function with respect to the parameters \( \beta_m \) and \( \gamma_m \).
The way we proceed is as follows (here we specialize to the squared-error cost function)
We could use any of the algorithms we have discussed till now. If we use trees, \( \gamma \) parameterizes the split variables and split points at the internal nodes, and the predictions at the terminal nodes.
To better understand what happens, let us develop the steps for the iterative fitting using the above squared error function.
For simplicity we assume also that our functions \( b(x;\gamma)=1+\gamma x \).
This means that for every iteration \( m \), we need to optimize
$$ (\beta_m,\gamma_m) = \mathrm{argmin}_{\beta,\lambda}\hspace{0.1cm} \sum_{i=0}^{n-1}(y_i-f_{m-1}(x_i)-\beta b(x;\gamma))^2=\sum_{i=0}^{n-1}(y_i-f_{m-1}(x_i)-\beta(1+\gamma x_i))^2. $$We start our iteration by simply setting \( f_0(x)=0 \). Taking the derivatives with respect to \( \beta \) and \( \gamma \) we obtain
$$ \frac{\partial {\cal C}}{\partial \beta} = -2\sum_{i}(1+\gamma x_i)(y_i-\beta(1+\gamma x_i))=0, $$and
$$ \frac{\partial {\cal C}}{\partial \gamma} =-2\sum_{i}\beta x_i(y_i-\beta(1+\gamma x_i))=0. $$We can then rewrite these equations as (defining \( \boldsymbol{w}=\boldsymbol{e}+\gamma \boldsymbol{x}) \) with \( \boldsymbol{e} \) being the unit vector)
$$ \gamma \boldsymbol{w}^T(\boldsymbol{y}-\beta\gamma \boldsymbol{w})=0, $$which gives us \( \beta = \boldsymbol{w}^T\boldsymbol{y}/(\boldsymbol{w}^T\boldsymbol{w}) \). Similarly we have
$$ \beta\gamma \boldsymbol{x}^T(\boldsymbol{y}-\beta(1+\gamma \boldsymbol{x}))=0, $$which leads to \( \gamma =(\boldsymbol{x}^T\boldsymbol{y}-\beta\boldsymbol{x}^T\boldsymbol{e})/(\beta\boldsymbol{x}^T\boldsymbol{x}) \). Inserting for \( \beta \) gives us an equation for \( \gamma \). This is a non-linear equation in the unknown \( \gamma \) and has to be solved numerically.
The solution to these two equations gives us in turn \( \beta_1 \) and \( \gamma_1 \) leading to the new expression for \( f_1(x) \) as \( f_1(x) = \beta_1(1+\gamma_1x) \). Doing this \( M \) times results in our final estimate for the function \( f \).
Let us consider a binary classification problem with two outcomes \( y_i \in \{-1,1\} \) and \( i=0,1,2,\dots,n-1 \) as our set of observations. We define a classification function \( G(x) \) which produces a prediction taking one or the other of the two values \( \{-1,1\} \).
The error rate of the training sample is then
$$ \mathrm{\overline{err}}=\frac{1}{n} \sum_{i=0}^{n-1} I(y_i\ne G(x_i)). $$The iterative procedure starts with defining a weak classifier whose error rate is barely better than random guessing. The iterative procedure in boosting is to sequentially apply a weak classification algorithm to repeatedly modified versions of the data producing a sequence of weak classifiers \( G_m(x) \).
Here we will express our function \( f(x) \) in terms of \( G(x) \). That is
$$ f_M(x) = \sum_{i=1}^M \beta_m b(x;\gamma_m), $$will be a function of
$$ G_M(x) = \mathrm{sign} \sum_{i=1}^M \alpha_m G_m(x). $$In our iterative procedure we define thus
$$ f_m(x) = f_{m-1}(x)+\beta_mG_m(x). $$The simplest possible cost function which leads (also simple from a computational point of view) to the AdaBoost algorithm is the exponential cost/loss function defined as
$$ C(\boldsymbol{y},\boldsymbol{f}) = \sum_{i=0}^{n-1}\exp{(-y_i(f_{m-1}(x_i)+\beta G(x_i))}. $$We optimize \( \beta \) and \( G \) for each value of \( m=1:M \) as we did in the regression case. This is normally done in two steps. Let us however first rewrite the cost function as
$$ C(\boldsymbol{y},\boldsymbol{f}) = \sum_{i=0}^{n-1}w_i^{m}\exp{(-y_i\beta G(x_i))}, $$where we have defined \( w_i^m= \exp{(-y_if_{m-1}(x_i))} \).
First, for any \( \beta > 0 \), we optimize \( G \) by setting
$$ G_m(x) = \mathrm{sign} \sum_{i=0}^{n-1} w_i^m I(y_i \ne G_(x_i)), $$which is the classifier that minimizes the weighted error rate in predicting \( y \).
We can do this by rewriting
$$ \exp{-(\beta)}\sum_{y_i=G(x_i)}w_i^m+\exp{(\beta)}\sum_{y_i\ne G(x_i)}w_i^m, $$which can be rewritten as
$$ (\exp{(\beta)}-\exp{-(\beta)})\sum_{i=0}^{n-1}w_i^mI(y_i\ne G(x_i))+\exp{(-\beta)}\sum_{i=0}^{n-1}w_i^m=0, $$which leads to
$$ \beta_m = \frac{1}{2}\log{\frac{1-\mathrm{\overline{err}}}{\mathrm{\overline{err}}}}, $$where we have redefined the error as
$$ \mathrm{\overline{err}}_m=\frac{1}{n}\frac{\sum_{i=0}^{n-1}w_i^mI(y_i\ne G(x_i)}{\sum_{i=0}^{n-1}w_i^m}, $$which leads to an update of
$$ f_m(x) = f_{m-1}(x) +\beta_m G_m(x). $$This leads to the new weights
$$ w_i^{m+1} = w_i^m \exp{(-y_i\beta_m G_m(x_i))} $$The algorithm here is rather straightforward. Assume that our weak classifier is a decision tree and we consider a binary set of outputs with \( y_i \in \{-1,1\} \) and \( i=0,1,2,\dots,n-1 \) as our set of observations. Our design matrix is given in terms of the feature/predictor vectors \( \boldsymbol{X}=[\boldsymbol{x}_0\boldsymbol{x}_1\dots\boldsymbol{x}_{p-1}] \). Finally, we define also a classifier determined by our data via a function \( G(x) \). This function tells us how well we are able to classify our outputs/targets \( \boldsymbol{y} \).
We have already defined the misclassification error \( \mathrm{err} \) as
$$ \mathrm{err}=\frac{1}{n}\sum_{i=0}^{n-1}I(y_i\ne G(x_i)), $$where the function \( I() \) is one if we misclassify and zero if we classify correctly.
With the above definitions we are now ready to set up the algorithm for AdaBoost. The basic idea is to set up weights which will be used to scale the correctly classified and the misclassified cases.
For the iterations with \( m \le 2 \) the weights are modified individually at each steps. The observations which were misclassified at iteration \( m-1 \) have a weight which is larger than those which were classified properly. As this proceeds, the observations which were difficult to classifiy correctly are given a larger influence. Each new classification step \( m \) is then forced to concentrate on those observations that are missed in the previous iterations.
Using Scikit-Learn it is easy to apply the adaptive boosting algorithm, as done here.
from sklearn.ensemble import AdaBoostClassifier
ada_clf = AdaBoostClassifier(
DecisionTreeClassifier(max_depth=2), n_estimators=200,
algorithm="SAMME.R", learning_rate=0.01, random_state=42)
ada_clf.fit(X_train, y_train)
y_pred = ada_clf.predict(X_test)
skplt.metrics.plot_confusion_matrix(y_test, y_pred, normalize=True)
plt.show()
y_probas = ada_clf.predict_proba(X_test)
skplt.metrics.plot_roc(y_test, y_probas)
plt.show()
skplt.metrics.plot_cumulative_gain(y_test, y_probas)
plt.show()
Gradient boosting is again a similar technique to Adaptive boosting, it combines so-called weak classifiers or regressors into a strong method via a series of iterations.
In order to understand the method, let us illustrate its basics by bringing back the essential steps in linear regression, where our cost function was the least squares function.
We start again with our cost function \( {\cal C}(\boldsymbol{y}m\boldsymbol{f})=\sum_{i=0}^{n-1}{\cal L}(y_i, f(x_i)) \) where we want to minimize This means that for every iteration, we need to optimize
$$ (\hat{\boldsymbol{f}}) = \mathrm{argmin}_{\boldsymbol{f}}\hspace{0.1cm} \sum_{i=0}^{n-1}(y_i-f(x_i))^2. $$We define a real function \( h_m(x) \) that defines our final function \( f_M(x) \) as
$$ f_M(x) = \sum_{m=0}^M h_m(x). $$In the steepest decent approach we approximate \( h_m(x) = -\rho_m g_m(x) \), where \( \rho_m \) is a scalar and \( g_m(x) \) the gradient defined as
$$ g_m(x_i) = \left[ \frac{\partial {\cal L}(y_i, f(x_i))}{\partial f(x_i)}\right]_{f(x_i)=f_{m-1}(x_i)}. $$With the new gradient we can update \( f_m(x) = f_{m-1}(x) -\rho_m g_m(x) \). Using the above squared-error function we see that the gradient is \( g_m(x_i) = -2(y_i-f(x_i)) \).
Choosing \( f_0(x)=0 \) we obtain \( g_m(x) = -2y_i \) and inserting this into the minimization problem for the cost function we have
$$ (\rho_1) = \mathrm{argmin}_{\rho}\hspace{0.1cm} \sum_{i=0}^{n-1}(y_i+2\rho y_i)^2. $$Optimizing with respect to \( \rho \) we obtain (taking the derivative) that \( \rho_1 = -1/2 \). We have then that
$$ f_1(x) = f_{0}(x) -\rho_1 g_1(x)=-y_i. $$We can then proceed and compute
$$ g_2(x_i) = \left[ \frac{\partial {\cal L}(y_i, f(x_i))}{\partial f(x_i)}\right]_{f(x_i)=f_{1}(x_i)=y_i}=-4y_i, $$and find a new value for \( \rho_2=-1/2 \) and continue till we have reached \( m=M \). We can modify the steepest descent method, or steepest boosting, by introducing what is called gradient boosting.
Steepest descent is however not much used, since it only optimizes \( f \) at a fixed set of \( n \) points, so we do not learn a function that can generalize. However, we can modify the algorithm by fitting a weak learner to approximate the negative gradient signal.
Suppose we have a cost function \( C(f)=\sum_{i=0}^{n-1}L(y_i, f(x_i)) \) where \( y_i \) is our target and \( f(x_i) \) the function which is meant to model \( y_i \). The above cost function could be our standard squared-error function
$$ C(\boldsymbol{y},\boldsymbol{f})=\sum_{i=0}^{n-1}(y_i-f(x_i))^2. $$The way we proceed in an iterative fashion is to
import matplotlib.pyplot as plt
import numpy as np
from sklearn.model_selection import train_test_split
from sklearn.ensemble import GradientBoostingRegressor
import scikitplot as skplt
from sklearn.metrics import mean_squared_error
n = 100
maxdegree = 6
# Make data set.
x = np.linspace(-3, 3, n).reshape(-1, 1)
y = np.exp(-x**2) + 1.5 * np.exp(-(x-2)**2)+ np.random.normal(0, 0.1, x.shape)
error = np.zeros(maxdegree)
bias = np.zeros(maxdegree)
variance = np.zeros(maxdegree)
polydegree = np.zeros(maxdegree)
X_train, X_test, y_train, y_test = train_test_split(x, y, test_size=0.2)
for degree in range(1,maxdegree):
model = GradientBoostingRegressor(max_depth=degree, n_estimators=100, learning_rate=1.0)
model.fit(X_train,y_train)
y_pred = model.predict(X_test)
polydegree[degree] = degree
error[degree] = np.mean( np.mean((y_test - y_pred)**2) )
bias[degree] = np.mean( (y_test - np.mean(y_pred))**2 )
variance[degree] = np.mean( np.var(y_pred) )
print('Max depth:', degree)
print('Error:', error[degree])
print('Bias^2:', bias[degree])
print('Var:', variance[degree])
print('{} >= {} + {} = {}'.format(error[degree], bias[degree], variance[degree], bias[degree]+variance[degree]))
plt.xlim(1,maxdegree-1)
plt.plot(polydegree, error, label='Error')
plt.plot(polydegree, bias, label='bias')
plt.plot(polydegree, variance, label='Variance')
plt.legend()
save_fig("gdregression")
plt.show()
import matplotlib.pyplot as plt
import numpy as np
from sklearn.model_selection import train_test_split
from sklearn.datasets import load_breast_cancer
import scikitplot as skplt
from sklearn.ensemble import GradientBoostingClassifier
from sklearn.model_selection import cross_validate
# Load the data
cancer = load_breast_cancer()
X_train, X_test, y_train, y_test = train_test_split(cancer.data,cancer.target,random_state=0)
print(X_train.shape)
print(X_test.shape)
#now scale the data
from sklearn.preprocessing import StandardScaler
scaler = StandardScaler()
scaler.fit(X_train)
X_train_scaled = scaler.transform(X_train)
X_test_scaled = scaler.transform(X_test)
gd_clf = GradientBoostingClassifier(max_depth=3, n_estimators=100, learning_rate=1.0)
gd_clf.fit(X_train_scaled, y_train)
#Cross validation
accuracy = cross_validate(gd_clf,X_test_scaled,y_test,cv=10)['test_score']
print(accuracy)
print("Test set accuracy with Gradient boosting and scaled data: {:.2f}".format(gd_clf.score(X_test_scaled,y_test)))
import scikitplot as skplt
y_pred = gd_clf.predict(X_test_scaled)
skplt.metrics.plot_confusion_matrix(y_test, y_pred, normalize=True)
save_fig("gdclassiffierconfusion")
plt.show()
y_probas = gd_clf.predict_proba(X_test_scaled)
skplt.metrics.plot_roc(y_test, y_probas)
save_fig("gdclassiffierroc")
plt.show()
skplt.metrics.plot_cumulative_gain(y_test, y_probas)
save_fig("gdclassiffiercgain")
plt.show()
XGBoost or Extreme Gradient Boosting, is an optimized distributed gradient boosting library designed to be highly efficient, flexible and portable. It implements machine learning algorithms under the Gradient Boosting framework. XGBoost provides a parallel tree boosting that solve many data science problems in a fast and accurate way. See the article by Chen and Guestrin.
The authors design and build a highly scalable end-to-end tree boosting system. It has a theoretically justified weighted quantile sketch for efficient proposal calculation. It introduces a novel sparsity-aware algorithm for parallel tree learning and an effective cache-aware block structure for out-of-core tree learning.
It is now the algorithm which wins essentially all ML competitions!!!
import matplotlib.pyplot as plt
import numpy as np
from sklearn.model_selection import train_test_split
import xgboost as xgb
import scikitplot as skplt
from sklearn.metrics import mean_squared_error
n = 100
maxdegree = 6
# Make data set.
x = np.linspace(-3, 3, n).reshape(-1, 1)
y = np.exp(-x**2) + 1.5 * np.exp(-(x-2)**2)+ np.random.normal(0, 0.1, x.shape)
error = np.zeros(maxdegree)
bias = np.zeros(maxdegree)
variance = np.zeros(maxdegree)
polydegree = np.zeros(maxdegree)
X_train, X_test, y_train, y_test = train_test_split(x, y, test_size=0.2)
for degree in range(maxdegree):
model = xgb.XGBRegressor(objective ='reg:squarederror', colsaobjective ='reg:squarederror', colsample_bytree = 0.3, learning_rate = 0.1,max_depth = degree, alpha = 10, n_estimators = 200)
model.fit(X_train,y_train)
y_pred = model.predict(X_test)
polydegree[degree] = degree
error[degree] = np.mean( np.mean((y_test - y_pred)**2) )
bias[degree] = np.mean( (y_test - np.mean(y_pred))**2 )
variance[degree] = np.mean( np.var(y_pred) )
print('Max depth:', degree)
print('Error:', error[degree])
print('Bias^2:', bias[degree])
print('Var:', variance[degree])
print('{} >= {} + {} = {}'.format(error[degree], bias[degree], variance[degree], bias[degree]+variance[degree]))
plt.xlim(1,maxdegree-1)
plt.plot(polydegree, error, label='Error')
plt.plot(polydegree, bias, label='bias')
plt.plot(polydegree, variance, label='Variance')
plt.legend()
plt.show()
As you will see from the confusion matrix below, XGBoots does an excellent job on the Wisconsin cancer data and outperforms essentially all agorithms we have discussed till now.
import matplotlib.pyplot as plt
import numpy as np
from sklearn.model_selection import train_test_split
from sklearn.datasets import load_breast_cancer
from sklearn.preprocessing import LabelEncoder
from sklearn.model_selection import cross_validate
import scikitplot as skplt
import xgboost as xgb
# Load the data
cancer = load_breast_cancer()
X_train, X_test, y_train, y_test = train_test_split(cancer.data,cancer.target,random_state=0)
print(X_train.shape)
print(X_test.shape)
#now scale the data
from sklearn.preprocessing import StandardScaler
scaler = StandardScaler()
scaler.fit(X_train)
X_train_scaled = scaler.transform(X_train)
X_test_scaled = scaler.transform(X_test)
xg_clf = xgb.XGBClassifier()
xg_clf.fit(X_train_scaled,y_train)
y_test = xg_clf.predict(X_test_scaled)
print("Test set accuracy with Gradient Boosting and scaled data: {:.2f}".format(xg_clf.score(X_test_scaled,y_test)))
import scikitplot as skplt
y_pred = xg_clf.predict(X_test_scaled)
skplt.metrics.plot_confusion_matrix(y_test, y_pred, normalize=True)
save_fig("xdclassiffierconfusion")
plt.show()
y_probas = xg_clf.predict_proba(X_test_scaled)
skplt.metrics.plot_roc(y_test, y_probas)
save_fig("xdclassiffierroc")
plt.show()
skplt.metrics.plot_cumulative_gain(y_test, y_probas)
save_fig("gdclassiffiercgain")
plt.show()
xgb.plot_tree(xg_clf,num_trees=0)
plt.rcParams['figure.figsize'] = [50, 10]
save_fig("xgtree")
plt.show()
xgb.plot_importance(xg_clf)
plt.rcParams['figure.figsize'] = [5, 5]
save_fig("xgparams")
plt.show()
A Support Vector Machine (SVM) is a very powerful and versatile Machine Learning method, capable of performing linear or nonlinear classification, regression, and even outlier detection. It is one of the most popular models in Machine Learning, and anyone interested in Machine Learning should have it in their toolbox. SVMs are particularly well suited for classification of complex but small-sized or medium-sized datasets.
The case with two well-separated classes only can be understood in an intuitive way in terms of lines in a two-dimensional space separating the two classes (see figure below).
The basic mathematics behind the SVM is however less familiar to most of us. It relies on the definition of hyperplanes and the definition of a margin which separates classes (in case of classification problems) of variables. It is also used for regression problems.
With SVMs we distinguish between hard margin and soft margins. The latter introduces a so-called softening parameter to be discussed below. We distinguish also between linear and non-linear approaches. The latter are the most frequent ones since it is rather unlikely that we can separate classes easily by say straight lines.
The theory behind support vector machines (SVM hereafter) is based on the mathematical description of so-called hyperplanes. Let us start with a two-dimensional case. This will also allow us to introduce our first SVM examples. These will be tailored to the case of two specific classes, as displayed in the figure here based on the usage of the petal data.
We assume here that our data set can be well separated into two domains, where a straight line does the job in the separating the two classes. Here the two classes are represented by either squares or circles.
from sklearn import datasets
from sklearn.svm import SVC, LinearSVC
from sklearn.linear_model import SGDClassifier
from sklearn.preprocessing import StandardScaler
import matplotlib
import matplotlib.pyplot as plt
plt.rcParams['axes.labelsize'] = 14
plt.rcParams['xtick.labelsize'] = 12
plt.rcParams['ytick.labelsize'] = 12
iris = datasets.load_iris()
X = iris["data"][:, (2, 3)] # petal length, petal width
y = iris["target"]
setosa_or_versicolor = (y == 0) | (y == 1)
X = X[setosa_or_versicolor]
y = y[setosa_or_versicolor]
C = 5
alpha = 1 / (C * len(X))
lin_clf = LinearSVC(loss="hinge", C=C, random_state=42)
svm_clf = SVC(kernel="linear", C=C)
sgd_clf = SGDClassifier(loss="hinge", learning_rate="constant", eta0=0.001, alpha=alpha,
max_iter=100000, random_state=42)
scaler = StandardScaler()
X_scaled = scaler.fit_transform(X)
lin_clf.fit(X_scaled, y)
svm_clf.fit(X_scaled, y)
sgd_clf.fit(X_scaled, y)
print("LinearSVC: ", lin_clf.intercept_, lin_clf.coef_)
print("SVC: ", svm_clf.intercept_, svm_clf.coef_)
print("SGDClassifier(alpha={:.5f}):".format(sgd_clf.alpha), sgd_clf.intercept_, sgd_clf.coef_)
# Compute the slope and bias of each decision boundary
w1 = -lin_clf.coef_[0, 0]/lin_clf.coef_[0, 1]
b1 = -lin_clf.intercept_[0]/lin_clf.coef_[0, 1]
w2 = -svm_clf.coef_[0, 0]/svm_clf.coef_[0, 1]
b2 = -svm_clf.intercept_[0]/svm_clf.coef_[0, 1]
w3 = -sgd_clf.coef_[0, 0]/sgd_clf.coef_[0, 1]
b3 = -sgd_clf.intercept_[0]/sgd_clf.coef_[0, 1]
# Transform the decision boundary lines back to the original scale
line1 = scaler.inverse_transform([[-10, -10 * w1 + b1], [10, 10 * w1 + b1]])
line2 = scaler.inverse_transform([[-10, -10 * w2 + b2], [10, 10 * w2 + b2]])
line3 = scaler.inverse_transform([[-10, -10 * w3 + b3], [10, 10 * w3 + b3]])
# Plot all three decision boundaries
plt.figure(figsize=(11, 4))
plt.plot(line1[:, 0], line1[:, 1], "k:", label="LinearSVC")
plt.plot(line2[:, 0], line2[:, 1], "b--", linewidth=2, label="SVC")
plt.plot(line3[:, 0], line3[:, 1], "r-", label="SGDClassifier")
plt.plot(X[:, 0][y==1], X[:, 1][y==1], "bs") # label="Iris-Versicolor"
plt.plot(X[:, 0][y==0], X[:, 1][y==0], "yo") # label="Iris-Setosa"
plt.xlabel("Petal length", fontsize=14)
plt.ylabel("Petal width", fontsize=14)
plt.legend(loc="upper center", fontsize=14)
plt.axis([0, 5.5, 0, 2])
plt.show()
The aim of the SVM algorithm is to find a hyperplane in a \( p \)-dimensional space, where \( p \) is the number of features that distinctly classifies the data points.
In a \( p \)-dimensional space, a hyperplane is what we call an affine subspace of dimension of \( p-1 \). As an example, in two dimension, a hyperplane is simply as straight line while in three dimensions it is a two-dimensional subspace, or stated simply, a plane.
In two dimensions, with the variables \( x_1 \) and \( x_2 \), the hyperplane is defined as
$$ b+w_1x_1+w_2x_2=0, $$where \( b \) is the intercept and \( w_1 \) and \( w_2 \) define the elements of a vector orthogonal to the line \( b+w_1x_1+w_2x_2=0 \). In two dimensions we define the vectors \( \boldsymbol{x} =[x1,x2] \) and \( \boldsymbol{w}=[w1,w2] \). We can then rewrite the above equation as
$$ \boldsymbol{x}^T\boldsymbol{w}+b=0. $$We limit ourselves to two classes of outputs \( y_i \) and assign these classes the values \( y_i = \pm 1 \). In a \( p \)-dimensional space of say \( p \) features we have a hyperplane defines as
$$ b+wx_1+w_2x_2+\dots +w_px_p=0. $$If we define a matrix \( \boldsymbol{X}=\left[\boldsymbol{x}_1,\boldsymbol{x}_2,\dots, \boldsymbol{x}_p\right] \) of dimension \( n\times p \), where \( n \) represents the observations for each feature and each vector \( x_i \) is a column vector of the matrix \( \boldsymbol{X} \),
$$ \boldsymbol{x}_i = \begin{bmatrix} x_{i1} \\ x_{i2} \\ \dots \\ \dots \\ x_{ip} \end{bmatrix}. $$If the above condition is not met for a given vector \( \boldsymbol{x}_i \) we have
$$ b+w_1x_{i1}+w_2x_{i2}+\dots +w_px_{ip} >0, $$if our output \( y_i=1 \). In this case we say that \( \boldsymbol{x}_i \) lies on one of the sides of the hyperplane and if
$$ b+w_1x_{i1}+w_2x_{i2}+\dots +w_px_{ip} < 0, $$for the class of observations \( y_i=-1 \), then \( \boldsymbol{x}_i \) lies on the other side.
Equivalently, for the two classes of observations we have
$$ y_i\left(b+w_1x_{i1}+w_2x_{i2}+\dots +w_px_{ip}\right) > 0. $$When we try to separate hyperplanes, if it exists, we can use it to construct a natural classifier: a test observation is assigned a given class depending on which side of the hyperplane it is located.
Let us try to develop our intuition about SVMs by limiting ourselves to a two-dimensional plane. To separate the two classes of data points, there are many possible lines (hyperplanes if you prefer a more strict naming) that could be chosen. Our objective is to find a plane that has the maximum margin, i.e the maximum distance between data points of both classes. Maximizing the margin distance provides some reinforcement so that future data points can be classified with more confidence.
What a linear classifier attempts to accomplish is to split the feature space into two half spaces by placing a hyperplane between the data points. This hyperplane will be our decision boundary. All points on one side of the plane will belong to class one and all points on the other side of the plane will belong to the second class two.
Unfortunately there are many ways in which we can place a hyperplane to divide the data. Below is an example of two candidate hyperplanes for our data sample.
Let us define the function
$$ f(x) = \boldsymbol{w}^T\boldsymbol{x}+b = 0, $$as the function that determines the line \( L \) that separates two classes (our two features), see the figure here.
Any point defined by \( \boldsymbol{x}_i \) and \( \boldsymbol{x}_2 \) on the line \( L \) will satisfy \( \boldsymbol{w}^T(\boldsymbol{x}_1-\boldsymbol{x}_2)=0 \).
The signed distance \( \delta \) from any point defined by a vector \( \boldsymbol{x} \) and a point \( \boldsymbol{x}_0 \) on the line \( L \) is then
$$ \delta = \frac{1}{\vert\vert \boldsymbol{w}\vert\vert}(\boldsymbol{w}^T\boldsymbol{x}+b). $$How do we find the parameter \( b \) and the vector \( \boldsymbol{w} \)? What we could do is to define a cost function which now contains the set of all misclassified points \( M \) and attempt to minimize this function
$$ C(\boldsymbol{w},b) = -\sum_{i\in M} y_i(\boldsymbol{w}^T\boldsymbol{x}_i+b). $$We could now for example define all values \( y_i =1 \) as misclassified in case we have \( \boldsymbol{w}^T\boldsymbol{x}_i+b < 0 \) and the opposite if we have \( y_i=-1 \). Taking the derivatives gives us
$$ \frac{\partial C}{\partial b} = -\sum_{i\in M} y_i, $$and
$$ \frac{\partial C}{\partial \boldsymbol{w}} = -\sum_{i\in M} y_ix_i. $$We can now use the Newton-Raphson method or different variants of the gradient descent family (from plain gradient descent to various stochastic gradient descent approaches) to solve the equations
$$ b \leftarrow b +\eta \frac{\partial C}{\partial b}, $$and
$$ \boldsymbol{w} \leftarrow \boldsymbol{w} +\eta \frac{\partial C}{\partial \boldsymbol{w}}, $$where \( \eta \) is our by now well-known learning rate.
The equations we discussed above can be coded rather easily (the framework is similar to what we developed for logistic regression). We are going to set up a simple case with two classes only and we want to find a line which separates them the best possible way.
There are however problems with this approach, although it looks pretty straightforward to implement. When running the above code, we see that we can easily end up with many diffeent lines which separate the two classes.
For small gaps between the entries, we may also end up needing many iterations before the solutions converge and if the data cannot be separated properly into two distinct classes, we may not experience a converge at all.
A better approach is rather to try to define a large margin between the two classes (if they are well separated from the beginning).
Thus, we wish to find a margin \( M \) with \( \boldsymbol{w} \) normalized to \( \vert\vert \boldsymbol{w}\vert\vert =1 \) subject to the condition
$$ y_i(\boldsymbol{w}^T\boldsymbol{x}_i+b) \geq M \hspace{0.1cm}\forall i=1,2,\dots, p. $$All points are thus at a signed distance from the decision boundary defined by the line \( L \). The parameters \( b \) and \( w_1 \) and \( w_2 \) define this line.
We seek thus the largest value \( M \) defined by
$$ \frac{1}{\vert \vert \boldsymbol{w}\vert\vert}y_i(\boldsymbol{w}^T\boldsymbol{x}_i+b) \geq M \hspace{0.1cm}\forall i=1,2,\dots, n, $$or just
$$ y_i(\boldsymbol{w}^T\boldsymbol{x}_i+b) \geq M\vert \vert \boldsymbol{w}\vert\vert \hspace{0.1cm}\forall i. $$If we scale the equation so that \( \vert \vert \boldsymbol{w}\vert\vert = 1/M \), we have to find the minimum of \( \boldsymbol{w}^T\boldsymbol{w}=\vert \vert \boldsymbol{w}\vert\vert \) (the norm) subject to the condition
$$ y_i(\boldsymbol{w}^T\boldsymbol{x}_i+b) \geq 1 \hspace{0.1cm}\forall i. $$We have thus defined our margin as the invers of the norm of \( \boldsymbol{w} \). We want to minimize the norm in order to have a as large as possible margin \( M \). Before we proceed, we need to remind ourselves about Lagrangian multipliers.
Consider a function of three independent variables \( f(x,y,z) \) . For the function \( f \) to be an extreme we have
$$ df=0. $$A necessary and sufficient condition is
$$ \frac{\partial f}{\partial x} =\frac{\partial f}{\partial y}=\frac{\partial f}{\partial z}=0, $$due to
$$ df = \frac{\partial f}{\partial x}dx+\frac{\partial f}{\partial y}dy+\frac{\partial f}{\partial z}dz. $$In many problems the variables \( x,y,z \) are often subject to constraints (such as those above for the margin) so that they are no longer all independent. It is possible at least in principle to use each constraint to eliminate one variable and to proceed with a new and smaller set of independent varables.
The use of so-called Lagrangian multipliers is an alternative technique when the elimination of variables is incovenient or undesirable. Assume that we have an equation of constraint on the variables \( x,y,z \)
$$ \phi(x,y,z) = 0, $$resulting in
$$ d\phi = \frac{\partial \phi}{\partial x}dx+\frac{\partial \phi}{\partial y}dy+\frac{\partial \phi}{\partial z}dz =0. $$Now we cannot set anymore
$$ \frac{\partial f}{\partial x} =\frac{\partial f}{\partial y}=\frac{\partial f}{\partial z}=0, $$if \( df=0 \) is wanted because there are now only two independent variables! Assume \( x \) and \( y \) are the independent variables. Then \( dz \) is no longer arbitrary.
However, we can add to
$$ df = \frac{\partial f}{\partial x}dx+\frac{\partial f}{\partial y}dy+\frac{\partial f}{\partial z}dz, $$a multiplum of \( d\phi \), viz. \( \lambda d\phi \), resulting in
$$ df+\lambda d\phi = (\frac{\partial f}{\partial z}+\lambda \frac{\partial \phi}{\partial x})dx+(\frac{\partial f}{\partial y}+\lambda\frac{\partial \phi}{\partial y})dy+ (\frac{\partial f}{\partial z}+\lambda\frac{\partial \phi}{\partial z})dz =0. $$Our multiplier is chosen so that
$$ \frac{\partial f}{\partial z}+\lambda\frac{\partial \phi}{\partial z} =0. $$We need to remember that we took \( dx \) and \( dy \) to be arbitrary and thus we must have
$$ \frac{\partial f}{\partial x}+\lambda\frac{\partial \phi}{\partial x} =0, $$and
$$ \frac{\partial f}{\partial y}+\lambda\frac{\partial \phi}{\partial y} =0. $$When all these equations are satisfied, \( df=0 \). We have four unknowns, \( x,y,z \) and \( \lambda \). Actually we want only \( x,y,z \), \( \lambda \) needs not to be determined, it is therefore often called Lagrange's undetermined multiplier. If we have a set of constraints \( \phi_k \) we have the equations
$$ \frac{\partial f}{\partial x_i}+\sum_k\lambda_k\frac{\partial \phi_k}{\partial x_i} =0. $$In order to solve the above problem, we define the following Lagrangian function to be minimized
$$ {\cal L}(\lambda,b,\boldsymbol{w})=\frac{1}{2}\boldsymbol{w}^T\boldsymbol{w}-\sum_{i=1}^n\lambda_i\left[y_i(\boldsymbol{w}^T\boldsymbol{x}_i+b)-1\right], $$where \( \lambda_i \) is a so-called Lagrange multiplier subject to the condition \( \lambda_i \geq 0 \).
Taking the derivatives with respect to \( b \) and \( \boldsymbol{w} \) we obtain
$$ \frac{\partial {\cal L}}{\partial b} = -\sum_{i} \lambda_iy_i=0, $$and
$$ \frac{\partial {\cal L}}{\partial \boldsymbol{w}} = 0 = \boldsymbol{w}-\sum_{i} \lambda_iy_i\boldsymbol{x}_i. $$Inserting these constraints into the equation for \( {\cal L} \) we obtain
$$ {\cal L}=\sum_i\lambda_i-\frac{1}{2}\sum_{ij}^n\lambda_i\lambda_jy_iy_j\boldsymbol{x}_i^T\boldsymbol{x}_j, $$subject to the constraints \( \lambda_i\geq 0 \) and \( \sum_i\lambda_iy_i=0 \). We must in addition satisfy the Karush-Kuhn-Tucker (KKT) condition
$$ \lambda_i\left[y_i(\boldsymbol{w}^T\boldsymbol{x}_i+b) -1\right] \hspace{0.1cm}\forall i. $$When \( \lambda_i > 0 \), the vectors \( \boldsymbol{x}_i \) are called support vectors. They are the vectors closest to the line (or hyperplane) and define the margin \( M \).
We can rewrite
$$ {\cal L}=\sum_i\lambda_i-\frac{1}{2}\sum_{ij}^n\lambda_i\lambda_jy_iy_j\boldsymbol{x}_i^T\boldsymbol{x}_j, $$and its constraints in terms of a matrix-vector problem where we minimize w.r.t. \( \lambda \) the following problem
$$ \frac{1}{2} \boldsymbol{\lambda}^T\begin{bmatrix} y_1y_1\boldsymbol{x}_1^T\boldsymbol{x}_1 & y_1y_2\boldsymbol{x}_1^T\boldsymbol{x}_2 & \dots & \dots & y_1y_n\boldsymbol{x}_1^T\boldsymbol{x}_n \\ y_2y_1\boldsymbol{x}_2^T\boldsymbol{x}_1 & y_2y_2\boldsymbol{x}_2^T\boldsymbol{x}_2 & \dots & \dots & y_1y_n\boldsymbol{x}_2^T\boldsymbol{x}_n \\ \dots & \dots & \dots & \dots & \dots \\ \dots & \dots & \dots & \dots & \dots \\ y_ny_1\boldsymbol{x}_n^T\boldsymbol{x}_1 & y_ny_2\boldsymbol{x}_n^T\boldsymbol{x}_2 & \dots & \dots & y_ny_n\boldsymbol{x}_n^T\boldsymbol{x}_n \\ \end{bmatrix}\boldsymbol{\lambda}-\mathbb{1}\boldsymbol{\lambda}, $$subject to \( \boldsymbol{y}^T\boldsymbol{\lambda}=0 \). Here we defined the vectors \( \boldsymbol{\lambda} =[\lambda_1,\lambda_2,\dots,\lambda_n] \) and \( \boldsymbol{y}=[y_1,y_2,\dots,y_n] \).
Solving the above problem, yields the values of \( \lambda_i \). To find the coefficients of your hyperplane we need simply to compute
$$ \boldsymbol{w}=\sum_{i} \lambda_iy_i\boldsymbol{x}_i. $$With our vector \( \boldsymbol{w} \) we can in turn find the value of the intercept \( b \) (here in two dimensions) via
$$ y_i(\boldsymbol{w}^T\boldsymbol{x}_i+b)=1, $$resulting in
$$ b = \frac{1}{y_i}-\boldsymbol{w}^T\boldsymbol{x}_i, $$or if we write it out in terms of the support vectors only, with \( N_s \) being their number, we have
$$ b = \frac{1}{N_s}\sum_{j\in N_s}\left(y_j-\sum_{i=1}^n\lambda_iy_i\boldsymbol{x}_i^T\boldsymbol{x}_j\right). $$With our hyperplane coefficients we can use our classifier to assign any observation by simply using
$$ y_i = \mathrm{sign}(\boldsymbol{w}^T\boldsymbol{x}_i+b). $$Below we discuss how to find the optimal values of \( \lambda_i \). Before we proceed however, we discuss now the so-called soft classifier.
Till now, the margin is strictly defined by the support vectors. This defines what is called a hard classifier, that is the margins are well defined.
Suppose now that classes overlap in feature space, as shown in the figure here. One way to deal with this problem before we define the so-called kernel approach, is to allow a kind of slack in the sense that we allow some points to be on the wrong side of the margin.
We introduce thus the so-called slack variables \( \boldsymbol{\xi} =[\xi_1,x_2,\dots,x_n] \) and modify our previous equation
$$ y_i(\boldsymbol{w}^T\boldsymbol{x}_i+b)=1, $$to
$$ y_i(\boldsymbol{w}^T\boldsymbol{x}_i+b)=1-\xi_i, $$with the requirement \( \xi_i\geq 0 \). The total violation is now \( \sum_i\xi \). The value \( \xi_i \) in the constraint the last constraint corresponds to the amount by which the prediction \( y_i(\boldsymbol{w}^T\boldsymbol{x}_i+b)=1 \) is on the wrong side of its margin. Hence by bounding the sum \( \sum_i \xi_i \), we bound the total amount by which predictions fall on the wrong side of their margins.
Misclassifications occur when \( \xi_i > 1 \). Thus bounding the total sum by some value \( C \) bounds in turn the total number of misclassifications.
This has in turn the consequences that we change our optmization problem to finding the minimum of
$$ {\cal L}=\frac{1}{2}\boldsymbol{w}^T\boldsymbol{w}-\sum_{i=1}^n\lambda_i\left[y_i(\boldsymbol{w}^T\boldsymbol{x}_i+b)-(1-\xi_)\right]+C\sum_{i=1}^n\xi_i-\sum_{i=1}^n\gamma_i\xi_i, $$subject to
$$ y_i(\boldsymbol{w}^T\boldsymbol{x}_i+b)=1-\xi_i \hspace{0.1cm}\forall i, $$with the requirement \( \xi_i\geq 0 \).
Taking the derivatives with respect to \( b \) and \( \boldsymbol{w} \) we obtain
$$ \frac{\partial {\cal L}}{\partial b} = -\sum_{i} \lambda_iy_i=0, $$and
$$ \frac{\partial {\cal L}}{\partial \boldsymbol{w}} = 0 = \boldsymbol{w}-\sum_{i} \lambda_iy_i\boldsymbol{x}_i, $$and
$$ \lambda_i = C-\gamma_i \hspace{0.1cm}\forall i. $$Inserting these constraints into the equation for \( {\cal L} \) we obtain the same equation as before
$$ {\cal L}=\sum_i\lambda_i-\frac{1}{2}\sum_{ij}^n\lambda_i\lambda_jy_iy_j\boldsymbol{x}_i^T\boldsymbol{x}_j, $$but now subject to the constraints \( \lambda_i\geq 0 \), \( \sum_i\lambda_iy_i=0 \) and \( 0\leq\lambda_i \leq C \). We must in addition satisfy the Karush-Kuhn-Tucker condition which now reads
$$ \lambda_i\left[y_i(\boldsymbol{w}^T\boldsymbol{x}_i+b) -(1-\xi_)\right]=0 \hspace{0.1cm}\forall i, $$ $$ \gamma_i\xi_i = 0, $$and
$$ y_i(\boldsymbol{w}^T\boldsymbol{x}_i+b) -(1-\xi_) \geq 0 \hspace{0.1cm}\forall i. $$