Multi-Label Classifier in Tensorflow
<p>I want to develop a multi-label classifier with TensorFlow, i try to mean there exists multiple label which contains multiple classes. To illustrate you can image the situation like: </p>

<ul>
<li>label-1 classes: lighting raining, raining, partial raining, no raining </li>
<li>label-2 classes: sunny, partly cloudy, cloudy, very cloudy. </li>
</ul>

<p>I want to classify these two labels with neural network. For now, i used different class label for every (label-1, label-2) pair classes. Which means i have 4 x 4 = 16 different label. </p>

<p>By training my model with </p>

<h1>current loss</h1>

<pre><code>cross_entropy = tf.reduce_mean(-tf.reduce_sum(ys * tf.log(prediction), reduction_indices=[1])) 

# prediction is sofmaxed
loss = cross_entropy + regul * schema['regul_ratio'] # regul things is for regularization 
train_step = tf.train.GradientDescentOptimizer(0.1).minimize(loss)
</code></pre>

<p>However i think that a multi-label training will work nicer in such a condition. </p>

<ul>
<li>My features will be [n_samples, n_features]</li>
<li>My labels will be [n_samples, n_classes, 2]</li>
</ul>

<p>n_samples of [x1, x2, x3, x4 ...]  # features</p>

<p>n_samples of [[0, 0, 0, 1],  [0, 0, 1, 0]] # no raining and cloudy </p>

<p>How can i make a softmax probability distribution predictor with tensorflow. Is there any working example for multilabel problem like this one. How will be my loss tensor like?</p>

<p>Why not just have your network produce two different outputs?</p>

<p>network -> prediction1 and prediction2</p>

<p>where prediction1 and prediction2 are both [#,#,#,#] but what I describe below works even if they're different sizes.</p>

<p>Then just run </p>

<pre><code>loss1 = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(prediction1, labels_1))
loss2 = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(prediction2, labels_2))

loss = loss1 + loss2
</code></pre>

python machine-learning tensorflow classification multilabel-classification