How do you get a probability of all classes to predict without building a classifier for each single class?
<p>Given a classification problem, sometimes we do not just predict a class, but need to return the probability that it is a class.</p>

<p>i.e. P(y=0|x), P(y=1|x), P(y=2|x), ..., P(y=C|x)</p>

<p>Without building a new classifier to predict y=0, y=1, y=2... y=C respectively. Since training C classifiers (let's say C=100) can be quite slow.</p>

<p>What can be done to do this? What classifiers naturally can give all probabilities easily (one I know is using neural network with 100 out nodes)? But if I use traditional random forests, I can't do that, right? I use the Python Scikit-Learn library.</p>

<p>Random forests do indeed give P(Y/x) for multiple classes. In most cases
P(Y/x) can be taken as:</p>

<p>P(Y/x)= the number of trees which vote for the class/Total Number of trees.</p>

<p>However you can play around with this, for example in one case if the highest class has 260 votes, 2nd class 230 votes and other 5 classes 10 votes, and in another case class 1 has 260 votes, and other classes have 40 votes each, you migth feel more confident in your prediction in 2nd case as compared to 1st case, so you come up with a confidence metric according to your use case.</p>

python machine-learning scikit-learn