Processing data with different number of features
<p>I have this classification/regression task but the most interesting thing is that the number of features for each record is different. Features are already extracted and already prepared thus the context of the data is unknown, and the values of the features fluctuate from -10 to 10. There are records with more than 200 features, likewise there are records with the amount of features lower than 20.</p>

<p>The dataframe <code>df</code> has two columns: <code>ID</code> and <code>ATTRIBUTES</code>and the output looks like this:</p>

<pre><code>      ID                                          ATTRIBUTES                
   0  1  1.1 2.1 3.3 4.4 5.5 6.6 ... 99.9  100.0 101.1 102.2
   1  2  1.1 2.1 3.3 4.4 5.5 6.6 ... 45.0  46.0  47.0  49.0 
   2  3  1.1 2.1 3.3 4.4 5.5 6.6 ... 9.0   10.0  11.0  12.0
   3  4  1.1 2.1 3.3 4.4 5.5 6.6 ... 70.0  71.0  72.0  73.0
   4  5  1.1 2.1 3.3 4.4 5.5 6.6 ... 131.0 132.0 134.0 135.0
</code></pre>

<p>I have split column <code>ATTRIBUTES</code> into separate columns:</p>

<p><code>df['ATTRIBUTES'].str.split(' ', expand=True).astype(float)</code></p>

<p>Now <code>df</code> looks like this:</p>

<pre><code>       0   1    2   3   4    5    6    7    8   9  ...   131   132   133   134   135 
       0  1.1  2.1 3.3 4.4. 5.5. 6.6. 7.7  8.8 9.9 ...   NaN   NaN   NaN   NaN   NaN 
       1  1.1  2.1 3.3 4.4. 5.5. 6.6. 7.7  8.8 9.9 ...   NaN   NaN   NaN   NaN   NaN
       2  1.1  2.1 3.3 4.4. 5.5. 6.6. 7.7  8.8 9.9 ...   NaN   NaN   NaN   NaN   NaN
       3  1.1  2.1 3.3 4.4. 5.5. 6.6. 7.7  8.8 9.9 ...   NaN   NaN   NaN   NaN   NaN
       4  1.1  2.1 3.3 4.4. 5.5. 6.6. 7.7  8.8 9.9 ...  131.0 132.0 133.0 134.0 135.0 
</code></pre>

<p>Let's say record1 has 102 features, rec2 - 49, rec3- 12, rec4-73, rec5 - 135. After a split operation records <code>rec1, rec2, rec3, rec4</code> were populated with <code>NaN</code> values to fill dataframe.</p>

<p>After some googling I've came up with following ideas: </p>

<ol>
<li>First thought was to change <code>NaN</code> values with meaningful features using 
<a href="http://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.Imputer.html" rel="nofollow">Imputer</a>; </li>
<li>Discard records that have less than 20 (40, 60 etc.) features. </li>
</ol>

<p>For classification I had chosen <a href="http://scikit-learn.org/stable/modules/generated/sklearn.ensemble.RandomForestClassifier.html" rel="nofollow">RandomForest</a>. </p>

<p>The baseline performance is approximately 0.4117 whilst validating 10% of training set (using <a href="http://scikit-learn.org/stable/modules/generated/sklearn.cross_validation.train_test_split.html" rel="nofollow">train_test_split</a>). </p>

<p>Despite everything I had tried:</p>

<ol>
<li><a href="http://sebastianraschka.com/Articles/2014_about_feature_scaling.html" rel="nofollow">Feature scaling - standardisation</a></li>
<li>Dimensionality reduction via Principal Component Analysis (PCA)</li>
<li>Tree-based feature selection using <a href="http://scikit-learn.org/stable/modules/feature_selection.html#tree-based-feature-selection" rel="nofollow">ExtraTreesClassifier</a></li>
</ol>

<p>The baseline performance had not raised higher than 0.4â¦ 
So my question is - how would one should proceed with the lack of features for particular records?  </p>

<p>Data imputation <em>might</em> work when </p>

<ul>
<li>the datapoints with missing attributes represent a small percentage of your datapoints</li>
<li>the amount of missing attributes per datapoint is not big.</li>
</ul>

<p>The reason you are getting poor results - at least as far as the data completeness is concerned - is that you are really distorting the data variance in your data with imputation and in effect you are disguising any underlying problem-specific meta-information hidden in your original data. The results of your work in a classification problem are very much relying on the quality of your data and the 'correctness' of your model.</p>

<p>I would start trying a couple different ways to deal with this sort of data issue. By any means these are not the only ways to go forward and Which one is best for you depends on your problem and the specifics of your data.</p>

<ol>
<li><p>See if some of the 'missing' or 'extra' attributes are possible to aggregate. Could it be the case that you have a fixed amount of features per datapoint and then 'optional' features that could be aggregated in some way?</p></li>
<li><p>See if you can segment your data in a way that - within the each segment - you have datapoints that hold the same amount of features. Then try to fit your classifier within these segments.</p></li>
<li><p>Check whether there is some pattern between feature amount and datapoint class.</p></li>
<li><p>Drop all features that have <code>NaN</code> values from your dataset.</p></li>
</ol>

<p>I hope this helps a bit!</p>

python pandas machine-learning scikit-learn