predict external dataset with models from random forest
<p>I used <code>joblib.dump</code> in python to save models from 5 fold cross validation modelling using random forest. As a result I have 5 models for each dataset saved as: <code>MDL_1.pkl, MDL_2.pkl, MDL_3.pkl, MDL_4.pkl, MDL_5.pkl</code>. Now I want to use these models for prediction of external dataset using <code>predict_proba</code> when the final prediction for each line in my external dataset is an average of 5 models. What is the best way to proceed?
Thank you for your help   </p>

<p>First of all, you should not save the result of cross validation. Cross validation <strong>is not</strong> a training method, it is <strong>an evaluation scheme</strong>. You should build <strong>a single model</strong> on your whole dataset and use it to predict.</p>

<p>If, for some reason, you can no longer train your model, you can still use this 5 predictions by averaging them (as random forest itself is a simple averagin ensemble of trees), however going back and retraining should give you <strong>bettter</strong> results.</p>

python