Obtaining the most informative features after dimensionality reduction
<p>I basically have a python script that tries a variety of dimensionality reduction techniques combined with a variety of classifiers.
I attempted to collect the most informative features for each classifier:</p>

<pre><code>if 'forest' in type(classifier).__name__.lower():
            importances = classifier.feature_importances_
            coefs_with_fns = numpy.argsort(importances)[::-1]
        else:
            coefs_with_fns = sorted(zip(classifier.coef_, reduced_training.columns))
</code></pre>

<p>While this works in principal, the output is just a series of integers, which (i assume) correspond to the column numbers in the feature array before the classifier. Which brings me to the problem: This array is the direct result of one dimensionality reduction method, which throws away all the previously attached column labels. </p>

<p>So my question is: is there a way to trace back the result of the dimensionality reduction to the actual columns/labels in the original dataset?</p>

<p>You can't.</p>

<p>When you do dimensionality reduction (like PCA), what you get is some new vectors and not a subset of the original feature set. And in the process you lose information. You project the the features of the original feature set, from a high dimensional space to a new (lower) space. You can't go back.</p>

<p>Note that <code>Dimensionality Reduction</code> or <code>Feature Extraction</code> is different from <code>Feature Selection</code>. In <code>Feature Selection</code> you select a subset of the original feature set.</p>

<p>Edit: in case you decide to use a Feature Selection technique, look at <a href="http://stackoverflow.com/questions/11116697/how-to-get-most-informative-features-for-scikit-learn-classifiers">this answer</a> in order to do what you want.</p>

python machine-learning scikit-learn