Scott Doyle /<EMAIL_ADDRESS>A training set $\mathcal{D}$ is a limited random sampling of the feature space.
We’ve seen several approaches to classification using class labels:
$\mathcal{D}$ may come from an underlying parameterized distribution:
Each of these has its own form!
Choose the one that:
What if:
In these cases, we need non-parametric methods.
Classification using Parzen Windows. If $h$ is small, we risk over-training. If $h$ is large, we will provide greater generalizability at the possible expense of accuracy.
Classification with $k$NN is so simple, it essentially “skips” the consideration of densities and just labels samples by counting the neighbors that belong to each class.
In almost all situations, gathering class labels is hard.
We are often presented with datasets having many unlabeled samples:
For some kinds of problems, you can crowd-source labeling.
Without labels, can we learn about categories?
From Zhou, Z. “A Brief Introduction to Weakly Supervised Learning”:
As ever, we start with some basic assumptions:
We can start by modeling the observed density, $p(\mathbf{x}|\boldsymbol{\theta})$, as a mixture of $c$ different class densities:
$ p(\mathbf{x}|\boldsymbol{\theta})=\sum_{j=1}^{c}p(\mathbf{x}|\omega_{j},\boldsymbol{\theta}_{j})P(\omega_{j}) $
… where $\boldsymbol{\theta}=\left(\boldsymbol{\theta}_{1}, \cdots, \boldsymbol{\theta}_{c}\right)^{T}$.
In this form, $p(\mathbf{x}|\boldsymbol{\theta})$ is known as a mixture density.
Conditional densities $p(\mathbf{x}|\omega_{j},\boldsymbol{\theta}_{j})$ are the component densities.
Priors $P(\omega_{j})$ are the mixing parameters.
$ p(\mathbf{x}|\boldsymbol{\theta})=\sum_{j=1}^{c}p(\mathbf{x}|\omega_{j},\boldsymbol{\theta_{j}})P(\omega_{j}) $
Our unknown is our set of parameters $\boldsymbol{\theta}$, so that’s what we want to estimate.
Once we have our parameter sets, we can “un-mix” the density, figure out what classes are the largest contributors at each point of $\mathbf{x}$, and then classify new points accordingly.
One valid question: if we have an infinite number of samples, and we know the underlying form of $p(\mathbf{x}|\boldsymbol{\theta})$, then is $\boldsymbol{\theta}$ unique?
A density $p(\mathbf{x}|\boldsymbol{\theta})$ is identifiable if $\boldsymbol{\theta}\neq\boldsymbol{\theta}^{\prime}$ implies that there exists an $\mathbf{x}$ such that $p(\mathbf{x}|\boldsymbol{\theta})\neq p(\mathbf{x}|\boldsymbol{\theta}^{\prime})$.
Suppose $p(\mathbf{x}|\boldsymbol{\theta})$ gives us a set $\mathcal{D}=\{\mathbf{x}_{1},\cdots,\mathbf{x}_{n}\}$
The likelihood of observing a specific $\mathcal{D}$ is the joint density:
$ p(\mathcal{D}|\boldsymbol{\theta})=\prod_{k=1}^{n}p(\mathbf{x}_{k}|\boldsymbol{\theta}) $
If we’re looking to find $\hat{\boldsymbol{\theta}}$ which maximizes $p(\mathcal{D}|\boldsymbol{\theta})$, we have to do the whole log-likelihood / gradient thing:
$ l=\sum_{k=1}^{n}\ln{p(\mathbf{x}_{k}|\boldsymbol{\theta})} $
$\nabla_{\boldsymbol{\theta}_{i}}l=\sum_{k=1}^{n}\frac{1}{p(\mathbf{x}_{k}|\boldsymbol{\theta})}\nabla_{\boldsymbol{\theta}_{i}}\left[\sum_{j=1}^{c}p(\mathbf{x}_{k}|\omega_{j},\boldsymbol{\theta}_{j})P(\omega_{j})\right] $
If we introduce the posterior probability, we can write in terms of the component densities:
$ P(\omega_{i}|\mathbf{x}_{k},\boldsymbol{\theta})=\frac{p(\mathbf{x}_{k}|\omega_{i},\boldsymbol{\theta}_{i})P(\omega_{i})}{p(\mathbf{x}_{k}|\boldsymbol{\theta})} $
Then we can rewrite the previous derivative as:
$\nabla_{\boldsymbol{\theta}_{i}}l = \sum_{k=1}^{n}P(\omega_{i}|\mathbf{x}_{k},\boldsymbol{\theta}) \nabla_{\boldsymbol{\theta}_{i}}\ln{p(\mathbf{x}_{k}|\omega_{i},\boldsymbol{\theta}_{i})} $
As always, we set this to zero and solve for the class-specific parameters $\boldsymbol{\theta}_{i}$.
We’ve already assumed we know the form of the density (namely, that it’s Gaussian).
There are four parameters that we may not know:
Just like linear discriminants, for simplicity we start by assuming that we know three of these: $\boldsymbol{\Sigma}_{i}$, $P(\omega_{i})$, and $c$.
Once again, we take the log-likelihood of the Gaussian for simplicity:
$ \ln{p(\mathbf{x}|\omega_{i},\boldsymbol{\mu}_{i})}=-\ln{\left[(2)^{\frac{d}{2}} |\boldsymbol{\Sigma}_{i}|^{\frac{1}{2}} \right]} -\frac{1}{2}(\mathbf{x}-\boldsymbol{\mu}_{i})^{T}\boldsymbol{\Sigma}_{i}^{-1}(\mathbf{x}-\boldsymbol{\mu}_{i}) $
… take the derivative…
$ \nabla_{\boldsymbol{\mu}_{i}}\ln{p(\mathbf{x}|\omega_{i},\boldsymbol{\mu}_{i})}=\boldsymbol{\Sigma}_{i}^{-1}(\mathbf{x}-\boldsymbol{\mu}_{i}) $
… and drop it into the old MLE equation for finding $\hat{\boldsymbol{\theta}}$, which, to refresh your memory, is:
$ \sum_{k=1}^{n}P(\omega_{i}|\mathbf{x}_{k},\boldsymbol{\theta})\nabla_{\boldsymbol{\theta}_{i}}\ln{p(\mathbf{x}_{k}|\omega_{i},\boldsymbol{\theta}_{i})}=0 $
Thus we have:
$ \sum_{k=1}^{n}P(\omega_{i}|\mathbf{x}_{k},\hat{\boldsymbol{\mu}})\boldsymbol{\Sigma}_{i}^{-1}(\mathbf{x}_{k}-\hat{\boldsymbol{\mu}_{i}})=0 $
If we multiply by $\boldsymbol{\Sigma}_{i}$ and moving around terms, we are left with:
$ \hat{\boldsymbol{\mu}}_{i} = \frac{\sum_{k=1}^{n}P(\omega_{i}|\mathbf{x}_{k},\hat{\boldsymbol{\mu}})\mathbf{x}_{k}}{\sum_{k=1}^{n}P(\omega_{i}|\mathbf{x}_{k},\hat{\boldsymbol{\mu}})} $
In other words, $\hat{\boldsymbol{\mu}}_{i}$ is a weighted average of the samples.
The weight for the $k$th sample is an estimate of how likely it is that $\mathbf{x}_{k}$ belongs to the $i$th class.
Does this equation give us $\hat{\boldsymbol{\mu}}_{i}$?
Unfortunately, we can’t solve $\hat{\boldsymbol{\mu}}_{i}$ explicitly.
What we CAN do is perform an iterative search: Select an initial value for $\hat{\boldsymbol{\mu}}_{i}(0)$, then solve:
$ \hat{\boldsymbol{\mu}}_{i}(j+1)=\frac{\sum_{k=1}^{n}P(\omega_{i}|\mathbf{x}_{k},\hat{\boldsymbol{\mu}}(j))\mathbf{x}_{k}}{\sum_{k=1}^{n}P(\omega_{i}|\mathbf{x}_{k},\hat{\boldsymbol{\mu}}(j))} $
This is essentially a gradient ascent method, meaning that eventually we will converge to a point where $\hat{\boldsymbol{\mu}}_{i}(j+1)\approx\hat{\boldsymbol{\mu}}_{i}(j)$
As with all iterative approaches, once the gradient is zero, we are only ensured that we’ve reached a local maximum.
Let’s say we have the following two-component, one-dimensional normal mixture:
$ p(x|\mu_{1},\mu_{2})=\underbrace{\frac{1}{3\sqrt{2}}\exp\left[-\frac{1}{2}(x-\mu_{1})^{2}\right]}_{\omega_{1}} +\underbrace{\frac{2}{3\sqrt{2}}\exp\left[-\frac{1}{2}(x-\mu_{2})^{2}\right]}_{\omega_{2}} $
We pull a set of $k=25$ samples sequentially from this mixture.
These samples are used to calculate the log-likelihood function:
$ l(\mu_{1},\mu_{2})=\sum_{k=1}^{n}\ln{p(x_{k}|\mu_{1},\mu_{2})} $
As ever, it’s important to have a significant number of samples to estimate your parameters.
What would it look like if we had insufficient samples for this problem?
If no constraints are placed on $\boldsymbol{\Sigma}_{i}$, then we have a problem…
Let’s say our two-component mixture is given like so:
$ p(x|\mu,\sigma^{2}) = \underbrace{\frac{1}{2\sqrt{2}\sigma}\exp\left[-\frac{1}{2}\left(\frac{x-\mu}{\sigma}\right)^{2}\right]}_{\textrm{First Component}} + \underbrace{\frac{1}{2\sqrt{2}}\exp\left[-\frac{1}{2}x^{2}\right]}_{\textrm{Second Component}} $
In other words, the second component has $\mu=0$, $\sigma=1$.
Now let’s set the mean of the first component to $\mu=x_{1}$. For $p(x_{1}|\mu,\sigma^{2})$, the first term’s exponential goes to 0, and so we have:
$ p(x_{1} | \mu, \sigma^{2}) = \frac{1}{2\sqrt{2}\sigma} + \frac{1}{2\sqrt{2}}\exp\left[-\frac{1}{2}x_{1}^{2}\right] $
For $x_{k}\neq\mu$, the first term’s exponential remains. Thus:
$ p(x_{k} | \mu,\sigma^{2}) \geq \frac{1}{2\sqrt{2}}\exp\left[-\frac{1}{2}x_{k}^{2}\right] $
The joint probability for $x_{1},\cdots,x_{n}$ is the product of all of the above, so with some rearranging we have:
$ p(x_{1},\cdots,x_{n} | \mu,\sigma^{2}) \geq \left[\frac{1}{\sigma}+\exp\left[-\frac{1}{2}x_{1}^{2}\right]\right]\frac{1}{(2\sqrt{2})^{n}}\exp\left[-\frac{1}{2}\sum_{k=2}^{n}x_{k}^{2}\right] $
$ p(x_{1},\cdots,x_{n} | \mu,\sigma^{2}) \geq \left[\frac{1}{\sigma}+\exp\left[-\frac{1}{2}x_{1}^{2}\right]\right]\frac{1}{(2\sqrt{2})^{n}}\exp\left[-\frac{1}{2}\sum_{k=2}^{n}x_{k}^{2}\right] $
What’s wrong with this equation? Specifically, look at $\sigma$…
If $\sigma$ is small, the equation explodes. That is to say, the MLE solution is singular.
So how do we get around this? Put some constraints on $\boldsymbol{\Sigma}_{i}$!
Recall the likelihood equation when finding $\boldsymbol{\mu}_{i}$:
$ \ln{p(\mathbf{x}|\omega_{i},\boldsymbol{\mu}_{i})}=-\ln{\left[(2)^{\frac{d}{2}} |\boldsymbol{\Sigma}_{i}|^{\frac{1}{2}} \right]} -\frac{1}{2}(\mathbf{x}-\boldsymbol{\mu}_{i})^{T}\boldsymbol{\Sigma}_{i}^{-1}(\mathbf{x}-\boldsymbol{\mu}_{i}) $
Just like before, we want to differentiate and set to zero, but this time we have to differentiate with respect to both $\boldsymbol{\mu}_{i}$ and $\boldsymbol{\Sigma}_{i}$.
Remember the relation we had previously:
$ \sum_{k=1}^{n}\hat{P}(\omega_{i}|\mathbf{x}_{k},\hat{\boldsymbol{\theta}})\nabla_{\boldsymbol{\theta}_{i}}\ln{p(\mathbf{x}_{k}|\omega_{i},\hat{\boldsymbol{\theta}}_{i}) = 0} $
I am skipping a lot of the math, but we end up with the following:
\begin{align} \hat{P}(\omega_{i}) &= \frac{1}{n}\sum_{k=1}^{n}\hat{P}(\omega_{i}|\mathbf{x}_{k},\hat{\boldsymbol{\theta}}) \\ \hat{\boldsymbol{\mu}_{i}} &= \frac{\sum_{k=1}^{n}\hat{P}(\omega_{i}|\mathbf{x}_{k},\hat{\boldsymbol{\theta}})\mathbf{x}_{k}}{\sum_{k=1}^{n}\hat{P}(\omega_{i}|\mathbf{x}_{k},\hat{\boldsymbol{\theta}})} \\ \hat{\boldsymbol{\Sigma}_{i}} &= \frac{\sum_{k=1}^{n}\hat{P}(\omega_{i}|\mathbf{x}_{k},\hat{\boldsymbol{\theta}})(\mathbf{x}_{k}-\hat{\boldsymbol{\mu}}_{i})(\mathbf{x}_{k}-\hat{\boldsymbol{\mu}}_{i})^{t}}{\sum_{k=1}^{n}\hat{P}(\omega_{i}|\mathbf{x}_{k},\hat{\boldsymbol{\theta}})} \end{align}
Okay, so what do those mean?
| $\hat{P}(\omega_{i}) = \frac{1}{n}\sum_{k=1}^{n}\hat{P}(\omega_{i}|\mathbf{x}_{k},\hat{\boldsymbol{\theta}})$ | The likelihood of $\omega_{i}$ is the fraction of samples from $\omega_{i}$ |
| $\hat{\boldsymbol{\mu}_{i}} = \frac{\sum_{k=1}^{n}\hat{P}(\omega_{i}|\mathbf{x}_{k},\hat{\boldsymbol{\theta}})\mathbf{x}_{k}}{\sum_{k=1}^{n}\hat{P}(\omega_{i}|\mathbf{x}_{k},\hat{\boldsymbol{\theta}})}$ | $\hat{\boldsymbol{\mu}}_{i}$ is the mean of those samples in $\omega_{i}$ |
| $\hat{\boldsymbol{\Sigma}_{i}} = \frac{\sum_{k=1}^{n}\hat{P}(\omega_{i}|\mathbf{x}_{k},\hat{\boldsymbol{\theta}})(\mathbf{x}_{k}-\hat{\boldsymbol{\mu}}_{i})(\mathbf{x}_{k}-\hat{\boldsymbol{\mu}}_{i})^{T}}{\sum_{k=1}^{n}\hat{P}(\omega_{i}|\mathbf{x}_{k},\hat{\boldsymbol{\theta}})}$ | $\hat{\boldsymbol{\Sigma}}_{i}$ is the corresponding covariance matrix |
If $\hat{P}(\omega_{i} | \mathbf{x}_{k}, \hat{\boldsymbol{\theta}})$ is between $0.0$ and $1.0$, then all samples play a role in the estimates (not just those belonging to $\omega_{i}$).
One final point: Above we’ve written $\sum_{k=1}^{n}\hat{P}(\omega_{i}|\mathbf{x}_{k},\hat{\boldsymbol{\theta}})$ a lot.
For each of the above three equations, we can substitute Bayes equation here:
\begin{align} \hat{P}(\omega_{i}|\mathbf{x}_{k},\hat{\boldsymbol{\theta}}) &= \frac{ p(\mathbf{x}_{k}|\omega_{i},\hat{\boldsymbol{\theta}}_{i})\hat{P}(\omega_{i}) }{ \sum_{j=1}^{c}p(\mathbf{x}_{k}|\omega_{j},\hat{\boldsymbol{\theta}}_{j})\hat{P}(\omega_{j}) }\\ &= \frac{ |\boldsymbol{\hat{\Sigma}}_{i}| ^{-\frac{1}{2}} \exp\left[-\frac{1}{2}(\mathbf{x}_{k}-\hat{\boldsymbol{\mu}}_{i})^{T}\hat{\boldsymbol{\Sigma}}_{i}^{-1}(\mathbf{x}_{k}-\hat{\boldsymbol{\mu}}_{i})\right]\hat{P}(\omega_{i})}{ \sum_{j=1}^{c}|\hat{\boldsymbol{\Sigma}}_{j}|^{-\frac{1}{2}}\exp\left[-\frac{1}{2}(\mathbf{x}_{k}-\hat{\boldsymbol{\mu}}_{j})^{T}\hat{\boldsymbol{\Sigma}}_{j}^{-1}(\mathbf{x}_{k}-\hat{\boldsymbol{\mu}}_{j})\right] \hat{P}(\omega_{j})} \end{align}
$\hat{P}(\omega_{i}|\mathbf{x}_{k},\hat{\boldsymbol{\theta}}) = \frac{|\boldsymbol{\hat{\Sigma}}_{i}|^{-\frac{1}{2}}\exp\left[-\frac{1}{2}(\mathbf{x}_{k}-\hat{\boldsymbol{\mu}}_{i})^{T}\hat{\boldsymbol{\Sigma}}_{i}^{-1}(\mathbf{x}_{k}-\hat{\boldsymbol{\mu}}_{i})\right]\hat{P}(\omega_{i})}{\sum_{j=1}^{c}|\hat{\boldsymbol{\Sigma}}_{j}|^{-\frac{1}{2}}\exp\left[-\frac{1}{2}(\mathbf{x}_{k}-\hat{\boldsymbol{\mu}}_{j})^{T}\hat{\boldsymbol{\Sigma}}_{j}^{-1}(\mathbf{x}_{k}-\hat{\boldsymbol{\mu}}_{j})\right]\hat{P}(\omega_{j})}$
Much simpler, right?
This is just to illustrate that now, we DO have an explicit way to calculate each of our unknown parameters given our sample set.
Once again, we can take an iterative approach to solving for our parameter sets:
$ \hat{\boldsymbol{\mu}}_{i}(j+1)=\frac{\sum_{k=1}^{n}P(\omega_{i}|\mathbf{x}_{k},\hat{\boldsymbol{\mu}}(j))\mathbf{x}_{k}}{\sum_{k=1}^{n}P(\omega_{i}|\mathbf{x}_{k},\hat{\boldsymbol{\mu}}(j))} $
$\require{color} \hat{P}(\omega_{i}|\mathbf{x}_{k},\hat{\boldsymbol{\theta}}) = \frac{|\boldsymbol{\hat{\Sigma}}_{i}|^{-\frac{1}{2}}\exp\left[-\frac{1}{2} \colorbox{lightblue}{$(\mathbf{x}_{k}-\hat{\boldsymbol{\mu}}_{i})^{T}\hat{\boldsymbol{\Sigma}}_{i}^{-1}(\mathbf{x}_{k}-\hat{\boldsymbol{\mu}}_{i})$} \right]\hat{P}(\omega_{i})}{\sum_{j=1}^{c}|\hat{\boldsymbol{\Sigma}}_{j}|^{-\frac{1}{2}}\exp\left[-\frac{1}{2}(\mathbf{x}_{k}-\hat{\boldsymbol{\mu}}_{j})^{T}\hat{\boldsymbol{\Sigma}}_{j}^{-1}(\mathbf{x}_{k}-\hat{\boldsymbol{\mu}}_{j})\right]\hat{P}(\omega_{j})} $
The value of $\hat{P}(\omega_{i}|\mathbf{x}_{k},\hat{\boldsymbol{\theta}})$ is high if the squared Mahalanobis distance $\colorbox{lightblue}{$(\mathbf{x}_{k}-\hat{\boldsymbol{\mu}}_{i})^{T}\hat{\boldsymbol{\Sigma}}_{i}^{-1}(\mathbf{x}_{k}-\hat{\boldsymbol{\mu}}_{i})$}$ is small.
Intuitively, this just means that if a sample is close to the centroid of a class cluster, then the likelihood of it belonging to the associated class is high.
If we replace the Mahalanobis with the squared Euclidean distance $|\mathbf{x}_{k}-\hat{\boldsymbol{\mu}}_{i}|^{2}$, we can find the mean $\hat{\boldsymbol{\mu}}_{m}$ nearest to $\mathbf{x}_{k}$.
Thus we can approximate $\hat{P}(\omega_{i}|\mathbf{x}_{k},\hat{\boldsymbol{\theta}})$ as:
$ \hat{P}(\omega_{i}|\mathbf{x}_{k},\hat{\boldsymbol{\theta}}) \simeq \begin{cases} 1 & \quad \textrm{if }i=m \\ 0 & \quad \textrm{otherwise.} \\ \end{cases}$
Then we can plug this into the equation we got before and solve for $\hat{\boldsymbol{\mu}}_{1},\cdots,\hat{\boldsymbol{\mu}}_{c}$.
We can “initialize” by selecting $c$ class centroids at random from the unlabeled data, and then iterating.
\begin{algorithm} \caption{\$k\$-Means Clustering} \begin{algorithmic} \INPUT Initialize \$n\$, \$c\$, \$\\boldsymbol{\\mu}\_{1}, \\ldots, \\boldsymbol{\\mu}\_{c}\$ \REPEAT \STATE Classify \$n\$ samples according to nearest \$\\boldsymbol{\\mu}\_{i}\$ \STATE Recompute \$\\boldsymbol{\\mu}\_{i}\$ \UNTIL{no change in \$\\boldsymbol{\\mu}_{i}\$} \RETURN \$\\boldsymbol{\\mu}\_{1}, \\ldots, \\boldsymbol{\\mu}\_{c}\$ \end{algorithmic} \end{algorithm}
Comparison of MLE and $k$-Means. Since the overlap between the components is relatively small, we arrive at basically the same answers for $\mu_{1}$ and $\mu_{2}$.
$k$-Means is a staple of unsupervised clustering methods.
It is simple and relatively fast, operating in $O(ndcT)$ time where $n$ is the number of patterns, $d$ is the number of features, $c$ is the number of clusters and $T$ is the number of iterations.
It typically finishes in a small number of iterations.
When does it fail?
Remember: everything is dependent on your features!
So far, we’ve been talking about clusters and classes as being synonymous, but that’s clearly not the case.
An underlying density with multiple modes could represent the same class, while a unimodal density may represent multiple classes (i.e. it’s a bad feature for discrimination).
We make assumptions about the form of the data, but clearly we could estimate perfectly valid parameter sets and completely miss the underlying structure of the data.
If we are interested in finding subclasses in our unlabeled data, we have to define what we mean.
Finding “clusters” seems like a good place to start, but how do we define a cluster?
We need a way to say that samples in one cluster are more similar to each other than they are to samples in another cluster.
We also need a way to evaluate a given partitioning of the dataset, to say that Clustering Result A is better or worse than Clustering Result B.
We can even start by using a distance metric to define our clusters in the first place!
Clustering with a high threshold means that almost everything is put into the same cluster. In graph theory, this would be called a fully connected graph and is akin to under-training.
Clustering with an intermediate threshold can reveal structure in your data, but it is up to the designer to say whether this is the correct structure.
Clustering with a low threshold forces only the closest points to be associated with the same cluster. This is akin to over-training.
Generally speaking, we assume the correct distance metric is Euclidean (the shortest distance between two points is a straight line).
However, this is only justified if the space is isotropic (distances in one dimension are equivalent to those in another) and the data is evenly spread.
Euclidean distance is robust to rigid transformations of the feature space (rotation and translation).
It is NOT robust to arbitrary linear transformations which distort distance relationships.
We can try to counteract the effects of non-isotropy in our feature space through normalization, where we translate and scale the data to have a zero mean and unit variance.
We can also rotate the coordinate space so that the axes coincide with the eigenvectors of the covariance matrix; this is part of principal component analysis (PCA) which we will cover shortly.
However, normalization has its own problems…
Normalization can take a well-separated dataset with two processes (left) and smoosh it together so we end up with a single cloud (right).
Question: Will supervised methods always out-perform unsupervised methods?
As always, it depends! Having more information (like labels) is better, but over-fitting is always a danger.
Over-fitting isn’t just fine-tuning your decision hyperplanes, it is also when you choose $c$ to be too high, or if you pick too many mixtures for each of your densities.
Sometimes, clustering can give you insight above and beyond what you thought you knew!
In most situations, the choice of similarity metric / distance function will be arbitrary.
In the literature, Euclidean distance metric is usually assumed, unless you have reason to believe otherwise.
With non-linear datasets, this may NOT true… but that’s for next class.
We will also look at evaluating our clusters by calculating criterion functions: sum-of-squared-error, minimum variance, scatter matrices, trace / determinant criterions, and invariant criterions.
(We may not look at all of those, but enough to give you the idea.)


It may not be HQ2 but AOC declares victory and trolls Amazon over New York expansion
by Jessica Guynn, Usa TodayCan we call it HQ1/2?
Less than a year after walking away from plans to put its second headquarters in Queens, Amazon has signed a new giant lease in New York City. It didn't take long for Democratic congresswoman Alexandria Ocasio-Cortez to troll the company on Twitter.
"Won't you look at that: Amazon is coming to NYC anyway—*without* requiring the public to finance shady deals, helipad handouts for Jeff Bezos, & corporate giveaways," she wrote Friday.
In a separate tweet, showing her sitting on a yellow bench, she added: "Me waiting on the haters to apologize after we were proven right on Amazon and saved the public billions."
The Amazon lease, which will accommodate more than 1,500 employees, is a far smaller footprint than the giant retailer was pledging with its second headquarters on Long Island.
Amazon had promised to bring between 25,000 to 40,000 jobs to the city in exchange for up to $3 billion in tax breaks and financial incentives from the state and city governments. Ocasio-Cortez's vocal opposition, along with others, was blamed for scuttling the deal.
The Wall Street Journal reported Friday that Amazon.com is taking 335,000 square feet on the city's west side in the new Hudson Yards neighborhood. According to the newspaper, Amazon said it did not receive any tax credits or other inducements.
Amazon already has more than 3,500 employees in its New York City tech hub and told the newspaper it will continue to grow.
Public subsidies for the HQ2 project as well as concerns over the gentrification of Long Island City sparked significant political opposition, including Ocasio-Cortez, also known as AOC.
"Anything is possible: today was the day a group of dedicated, everyday New Yorkers & their neighbors defeated Amazon's corporate greed, its worker exploitation, and the power of the richest man in the world," she wrote on Twitter after the deal collapsed.


Dashboard
| Guests Attending / Possible |
{{ guests }} / {{ possible_guests }} |
| Pending Invitations |
{{ pending_invites }} ({{ pending_guests }} guests) |
| Unopened Invitations |
{{ unopened_invite_count }} / {{ total_invites }} |
| Not Coming |
{{ not_coming_guests }} |
Meal choices
{% for meal in meal_breakdown %}
| {{ meal.meal }} |
{% endfor %}
{% for meal in meal_breakdown %}
| {{ meal.count }} |
{% endfor %}
Unopened Invitations
{% with party_list=parties_with_unopen_invites %}
{% include "guests/partials/party_list.html" %}
{% endwith %}
Open Unresponded Invitations
{% with parties_with_open_unresponded_invites=party_list %}
{% include "guests/partials/party_list.html" %}
{% endwith %}
Attending Guests Without Meals
{% with guest_list=guests_without_meals %}
{% include "guests/guest_list.html" %}
{% endwith %}
{% endblock %}


Tutorials
Various tutorials for helping people do stuff.
Motorcycles
Front Suspension Upgrade
Identifying a Dead Reg/Rec
Honda Reg/Rec Replacement


This web application relies on Javascript and the jQuery library to request information from Twitter. Without Javascript enabled, no data can be requested (and therefor shown to you, the user).
Please, turn on Javascript; if you don't, then please leave.


Weekly Meeting: 2014-06-10
10th June 2014: Irmin, Conduit, Deployment and Fist pumping
Agenda
- Xen Hackathon summary
- Irmin update
- OCaml conduit progress
- Networking fuzz testing
- Unikernel deployment
- Call schedule and organisation
Attendees: Amir Chaudhry (chair), Thomas Gazagnaire, Thomas Leonard, Anil Madhavapeddy, Richard Mortier, Mindy Preston and David Scott
Notes
Xen Hackathon
The Xen Developer Hackathon took place a couple of weeks ago and Anil, ThomasL, Dave, Jon and Nik all attended. Apparently it did turn into a chatathon as previously predicted.
A group of people went through Amir’s Jekyll to Unikernel post to
get their sites building using TravisCI. They did get this working but the
post was written based on a Mac workflow and some parts didn’t work well on
other machines, specifically involving tap interfaces (e.g. sudo mirage run
didn’t work as expected). In general, Amir mentioned that the blog post
isn’t going to get updated but it should point to somewhere that current
information is available for people to follow (that can account for any
quirks on different systems). The Mirage website would be good but updating
this isn’t as easy as a wiki.
Other than the above, there were lots of conversations with others people including those trying lots guests on Xen and seeing lots of latency. Important for us as we want to be able to run lots of unikernels on top of a single Xen instance.
Irmin
Irmin 0.8.1 was released. ThomasG has been working on an filesystem backend implementation. You can take a normal filesystem and write at the block level. Next step is to expose Irmin as a filesystem. So you can use the filesystem and connect somewhere else to get the history back. Practically speaking, you build an Irmin filesystem on top of an existing filesystem (e.g FAT). It’ll be a combinator that takes the FAT implementation and make a new one that’s used/exposed (so the file system is like staging). [If these notes are confusing, it’s because I couldn’t keep up - AC :)]
ThomasG also learned that the DOS protocol doesn’t let you have two letter directories. Also had some discussion on FAT, blockstore and key value stores.
Conduit
The network stack has become a chimney, where the underlying choices are affecting things further up the stack. Anil has been thinking about different approaches and discussing things with people. One thought was around using functors but that leads to having a functorised stack and then the applications also have to deal with things this way and whole universe becomes functorised. This becomes difficult to use. Another approach is to use objects (or at least partially) and this seems like it would work much better. In any case, this is blocking other people’s cohttp fixes but looks like there is a workable solution to this. Anil will have to write this down to clarify it and the details are quite involved (but are to do with existential types – in case anyone feels like diving into it).
In general conduit will permit people to use either openSSL bindings or the new TLS library and doesn’t impose those choices on anyone.
Network fuzz testing
Mindy is in a cycle of finding bugs and fixing them. Been thinking about the platform question and doing research today. Also saw Balraj’s email, who has been bisecting to figure out where/when the problems arose in the stack. It turns out that it’s a set of commits over Christmas/New Year by Anil.
Overall, it’s a great sign that we’re able to go back and pin down which set of commits introduced the problems but it does show that we need to improve the type safety in the stack and add unit tests. Any thoughts on unit test frameworks for networking protocols would be useful so please do send them to the mailing list.
Unikernel deployment
We currently have two machines, configured with Xen that we could deploy our unikernels to. Amir would like to put his unikernels there and (when stable) start moving more of the static sites we have to those machines. It should be possible to get an account for Anil and at the moment users would need to write their own config file. There is the question of DNS servers.
In the long term the idea is that we each have our own unikernel DNS servers but in the short term to have a separate DNS server that would know about the mapping between the names and the IP addresses. We have a set of IP addresses on a wiki and can grab one of those and run with it. For example Mort has one but has not deployed yet as he’s blocked on filesystem issues. Essentially, his site is large enough (papers, media, etc) that he needs two FAT images, which isn’t handled very well. Can achieve it locally but it requires tweaking by hand and shell scripts, so not really appropriate for deployment. Is thinking of patching the Mirage tool but has not time to get to that. Should create an issue to track this as others are likely to follow in his wake as their sites grow (e.g Amir switched from Crunch to FAT some time ago). An issue might help surface others who are having similar kinds of problems.
Call schedule and organisation
Amir points out that we should improve the call organisation. Announcement of the call was sent about an hour beforehand and we should be giving others more time. Proposed that a notice is sent on the Monday beforehand to the list to collect Agenda items, and then the call is confirmed the following day. Regular people on the call can help by adding their agenda items in advance and Amir will structure them and help keep the call focused (which it usually is anyway).
AoB
Mindy and ThomasL have written some awesome blog posts recently. The posts made it to the front pages of news aggregators and generated a lot of interesting discussion. The posts were Python to OCaml: Retrospective (HN, Reddit) and The Minnesota Goodbye (HN).
OCaml-TLS is is gearing up to an alpha release and we will point people to the Mirage mailing list for communications.
Some discussion about Mirage 2.0 release and what the requirements would be for it. Would need ARM support, Irmin and a story on distributed computing. This all seems achievable but really would like to have this by the time of OSCON.
Carmack thinks we’re “really interesting”. Cue lots of fist pumping.
Next call will be 24th June. Amir will send an email to the list for Agenda items on the 23rd.


static
methods with a signature identical to a static
method of a superclass.
Such a method may look like an override when in fact it hides the method from the superclass because
static
methods in Java cannot be overridden.
Example:
class Parent {
static void method(){}
}
class Example extends Parent {
static void method(){} //warning
}


When you join the Advanced Reader Team you recieve a copy of my books before they're available on Amazon, absolutely free of charge. You also have the opportunity to email me back and tell me what you thought of the book, and point out anything I might have messed up, so that the final product that gets sent to Amazon is the best possible book.
Sign UpI really encourage (but don't require) my Advanced Reader Team to take a couple minutes and leave a review on Amazon. We indie writers are competing with all the big publishing companies on Amazon. By leaving an honest review you're helping to support independent writers like me, who are working hard to make a career for themselves. It doesn't have to be a 400 word essay, just whatever you're comfortable with.
Plus, I often run Amazon Gift Card giveaways for my reviewers!
Sign UpI consider my Advanced Readers my friends. I trade emails with them all the time, and really welcome anything they have to say, even if it's just 'hi.' By joining the Advanced Reader Team you're getting free books, but you're also becoming part of something I consider really special.
Sign Up


{{ define "main" }}
I like to beat the brush. Maybe there's a happy little bush that lives right there. Exercising the imagination, experimenting with talents, being creative; these things, to me, are truly the windows to your soul. We'll make some happy little bushes here.
Now let's put some happy little clouds in here. Let your imagination be your guide. See. We take the corner of the brush and let it play back-and-forth. We can always carry this a step further. There's really no end to this. Anytime you learn something your time and energy are not wasted
Let's do that again. You can create the world you want to see and be a part of. You have that power. If you didn't have baby clouds, you wouldn't have big clouds. We can always carry this a step further. There's really no end to this. Isn't it fantastic that you can change your mind and create all these happy things? Talk to trees, look at the birds. Whatever it takes.


hello we are the BUSMALL we want you to choose between those three images that you would be most likely to purchase


Hello to your MongoDB playground
Add new tag:
Tag name:
Add new Image:
Image url:
Title:
Description:
Tags:
Search:
Tag name:
Posted after:
Posted before:
Limit:


\usecounter{counter}
The \usecounter command is used in the second argument of the list environment to allow the counter specified to be used to number the list items.
See also Counters, list environment Go to LaTeX Table of Contents


Create various types of {@link net.java.html.charts.Chart charts}. Display them using HTML/Java API and other presenters provided by the DukeScript project.
Click the chart to increase the values. Hold Shift to decrease, hold Ctrl to modify the value by ten.
To use this library from your
DukeScript application follow
the getting started tutorial
and then add dependency on the Charts API to your
client/pom.xml
:
<dependency> <groupId>com.dukescript.api</groupId> <artifactId>charts</artifactId> <version>1.0</version> <-- or any newer --> <type>jar</type> </dependency>
If you are also using client-web/pom.xml
consider to include
pre-built Bck2Brwsr Library
as
<dependency> <groupId>com.dukescript.api</groupId> <artifactId>charts</artifactId> <version>1.0</version> <-- the same as above --> <classifier>bck2brwsr</classifier> </dependency>
Including this pre-built library will speed your compilation time up.


A long-term forecast model called "Jubilee-Tectonic model" is implemented to forecast future returns of the U.S. stock market, Treasury yield, and gold price. The five-factor model can forecast the 10-year and 20-year future equity returns with high R-squared above 80 percent. It is based on linear growth and mean reversion characteristics in the U.S. stock market. In addition, this model enhances the CAPE model by introducing the hypothesis that there are fault lines in the historical CAPE, which can be calibrated and corrected through statistical learning.
| Version: | 0.2.5 |
| Depends: | R (≥ 3.3.0) |
| Imports: | stats, yaml, utils, xts, zoo, splines, parallel, graphics, methods, readxl, data.table |
| Suggests: | knitr, tinytex, R.rsp, testthat, roxygen2, scales, shape |
| Published: | 2018-09-11 |
| Author: | Stephen H-T. Lihn [aut, cre] |
| Maintainer: | Stephen H-T. Lihn <stevelihn at gmail.com> |
| License: | Artistic-2.0 |
| URL: | https://ssrn.com/abstract=3156574 |
| NeedsCompilation: | no |
| Materials: | NEWS |
| CRAN checks: | jubilee results |
| Reference manual: | jubilee.pdf |
| Vignettes: |
Jubilee: Forecasting Long-Term Growth of S&P 500 Index |
| Package source: | jubilee_0.2.5.tar.gz |
| Windows binaries: | r-devel: jubilee_0.2.5.zip, r-release: jubilee_0.2.5.zip, r-oldrel: jubilee_0.2.5.zip |
| OS X binaries: | r-release: jubilee_0.2.5.tgz, r-oldrel: jubilee_0.2.5.tgz |
| Old sources: | jubilee archive |
Please use the canonical form https://CRAN.R-project.org/package=jubilee to link to this page.


After my first couple of weeks of coding the question I keep asking myself is, "Why didn't start this much, much sooner?" Coding is far more accessible than I ever imagined. Yet, what I like most about it, is that there is so much to learn. The basics are quiet accessible and user friendly, but the depth of complexity and possibility are so incredible that I'm excited to get started and learn more every day.


{{ code }}
Oops! There seems to be a problem.
{{ error|safe }}
{% if install_key %}
Please execute this command as Root user to install Mom agent automatically:
-
cd /tmp && wget -q https://cdn.monit.dip.lenaic.me/mom-agent.sh && bash mom-agent.sh {{ install_key }}


Create
Find inspiration and keep it
Read More
Hi.
My name is Olya Sarafin
and I am a front-end developer.
In the my past life I was a bank worker. I changed my way,
because I always want to make something new and interesting.
Do you want to see what I can do?


SavedModel builder.
Builds a SavedModel that can be saved to storage, is language neutral, and enables systems to produce, consume, or transform TensorFlow Models.
class SavedModelBuilder
: Builds the SavedModel
protocol buffer and saves variables and assets.


We provide some apis to play with :
https://api.gravitee.io/echo
(Gravitee.io API Echo)
https://api.gravitee.io/whattimeisit
(Gravitee.io API What Time Is It)
https://api.gravitee.io/whoami
(Gravitee.io API Who Am I)
see
https://github.com/gravitee-io/gravitee-sample-apis/blob/master/README.md


WKTPoint Class Reference
| Inherits from | WKTGeometry : NSObject |
|---|---|
| Declared in | WKTPoint.h |
Overview
This class represent Point Geometry on WKT Format.
Its String representation are:
2D Representation: POINT (0 0)
3D Representation: POINTZ (0 0 0)
At 3D representation may be POINT Z too.
Example of use:
WKTPoint *p1 = [[WKTPoint alloc] initWithDimensionX:5.0 andDimensionY:10.0];
WKTPoint *p2 = [[WKTPoint alloc] initWithDimensionX:10.0 andDimensionY:10.0 andDimensionZ:5.0];
Example of MapKit Annotation creation:
MKPointAnnotation *mapPoint = [p1 toMapPointAnnotation];
[mapPoint setTitle: @"Annotation title"];
[mapPoint setSubtitle: @"Annotation subtitle"];
MKMapView *map = [[MKMapView alloc] init];
[map addAnnotation: mapPoint];
Example of MapKit Point creation:
MKMapPoint coordinateMPoint = [p1 toMapPoint];
MKPointAnnotation *a1 = [[MKPointAnnotation alloc] init];
[a1 setCoordinate:MKCoordinateForMapPoint(coordinateMPoint)];
[a1 setTitle: @"Annotation title"];
[a1 setSubtitle: @"Annotation subtitle"];
[map addAnnotation: a1];
Example of Distance between Locations:
CLLocation *loc1 = [p1 toLocation];
CLLocation *loc2 = [p2 toLocation];
CLLocationDistance distance = [loc1 distanceFromLocation:loc2];
Example of Coordinate creation:
CLLocationCoordinate2D coord1 = [p1 toMapCoordinate];
MKPointAnnotation *a2 = [[MKPointAnnotation alloc] init];
[a2 setCoordinate:coordinatePoint];
[a2 setTitle: @"Annotation title"];
[a2 setSubtitle: @"Annotation subtitle"];
[map addAnnotation: a2];
Example of WKT representation:
NSString *wktString = [p1 toWKT];
NSLog(@"WKT: %@", wktString);
> WKT: POINT (5.000000 10.000000)
Properties
dimensionX
Dimension X property same as Longitude
@property (nonatomic, readwrite) float dimensionX
Declared In
WKTPoint.h
dimensionY
Dimension Y property same as Latitude
@property (nonatomic, readwrite) float dimensionY
Declared In
WKTPoint.h
dimensionZ
Dimension Z property same as Altitude
@property (nonatomic, readwrite) float dimensionZ
Declared In
WKTPoint.h
Constructor
– init
Basic Constructor that set default values to new instance.
- (instancetype)init
Return Value
WKTPoint instance
Declared In
WKTPoint.h
– initWithDimensionX:andDimensionY:
Complex Constructor that set default values and dimensions to new instance.
- (instancetype)initWithDimensionX:(float)dimX andDimensionY:(float)dimY
Parameters
dimX |
longitude value (float format) |
|---|---|
dimY |
latitude value (float format) |
Return Value
WKTPoint instance
Discussion
This method is the same that init join setDimensionX: and setDimensionY:
Declared In
WKTPoint.h
– initWithDimensionX:andDimensionY:andDimensionZ:
Complex Constructor that set default values and dimensions to new instance.
- (instancetype)initWithDimensionX:(float)dimX andDimensionY:(float)dimY andDimensionZ:(float)dimZ
Parameters
dimX |
longitude value (float format) |
|---|---|
dimY |
latitude value (float format) |
dimZ |
altitude value (float format) |
Return Value
WKTPoint instance
Discussion
This method is the same that init join setDimensionX:, setDimensionY: and setDimensionZ:
Declared In
WKTPoint.h
WKTPoint operations
– isEqual:
Check if two WKTPoint are equal.
- (BOOL)isEqual:(WKTPoint *)otherPoint
Parameters
otherPoint |
WKTPoint to compare with itself |
|---|
Return Value
Boolean
Declared In
WKTPoint.h
– copyTo:
Pass all properties from itself to other WKTPoint reference.
- (void)copyTo:(WKTPoint *)otherPoint
Parameters
otherPoint |
WKTPoint to pass properties |
|---|
Exceptions
WKTPoint |
Parameter is nil. |
|---|
Declared In
WKTPoint.h
– toWKT
Return WKT representation.
- (NSString *)toWKT
Return Value
String like POINT ( …. ) or POINT EMPTY
Declared In
WKTPoint.h
MapKit operations
– toMapPointAnnotation
Return MKPointAnnotation representation.
- (MKPointAnnotation *)toMapPointAnnotation
Return Value
MapKit Annotation for add to map
Exceptions
WKTPoint |
coordinate is not valid WGS84 format. |
|---|
Declared In
WKTPoint.h
– toMapCoordinate
Return Coordinate representation.
- (CLLocationCoordinate2D)toMapCoordinate
Return Value
CLLocationCoordinate2D to use
Exceptions
WKTPoint |
coordinate is not valid WGS84 format. |
|---|
Declared In
WKTPoint.h
– toLocation
Return Location representation.
- (CLLocation *)toLocation
Return Value
CLLocation to use
Exceptions
WKTPoint |
coordinate is not valid WGS84 format. |
|---|
Declared In
WKTPoint.h
– toMapPoint
Return Point representation.
- (MKMapPoint)toMapPoint
Return Value
MKMapPoint to use
Exceptions
WKTPoint |
coordinate is not valid WGS84 format. |
|---|
Declared In
WKTPoint.h


Used dictionaries
Chrome and Firefox browsers block pop-up windows. When you open dictionaries for the first time, all or some windows are blocked. You have to add exception/allow opening for this website, and reload the page.


Resources
Here are a few resources to help you on your path to freedom.
Phone Numbers
This page has numbers for multiple national and state hotlines to help with immediate violence needs. They may help with other needs as well.
Cost of Living Calculator
This page will let you calculate the difference in the average cost of living between where you are now and other places in the US. Please note you may need to find the nearest metropolitan area to get an estimate.
https://www.nerdwallet.com/cost-of-living-calculator
Safety Plans
Having a plan will help you be able to not only stop
the cycle of violence, it will help you have a plan to thrive
once you are out. These two pages show ways to make that plan.
https://www.safehorizon.org/our-services/safety-plan/
https://www.thehotline.org/help/path-to-safety/
Social Media Safety
Social Media, including email, is often used to find a survivor after
they have started their new life. It can also be used to derail a plan to
escape. This page has some tips to remain safe while planning your way out and
beyond.
https://www.thehotline.org/help/tech-social-media-safety/
Other Misc Resources
This page has tabs for state specific, national, teen and LGBTQ
resources for those expierencing domestice violence and survivors.


{% extends 'layout.html' %} {% block content %}
{{ site_settings.instance_description }}
More about this site
With a BookWyrm account, you can track and share your reading activity with friends here and on any other federated server, like Mastodon and PixelFed.


Autolab
500 - Internal error
Oops!
Autolab seems to have done something wrong.
We have NOT been notified about this.


{% extends "layout.html" %} {% block content %}
Try going back to the homepage, then we'll try to get things rolling again!