Creating more rows based on condition in pandas
<p>I have a dataframe like following:</p>

<pre><code>id, index, val_1, val_2
1, 1, 0.2, 0
1, 2, 0.4, 0.2
2,2, 0.1, 0.5
2,4, 0.7, 0.0
....
</code></pre>

<p>and so on</p>

<p>Now, the complete range of index values that are allowed for each id is </p>

<pre><code>[1,2,3,4]
</code></pre>

<p>So, if any of this index is missing for each id, I want to add those rows.
So for the above example, the desired output is</p>

<pre><code>id, index, val_1, val_2
1, 1, 0.2, 0
1, 2, 0.4, 0.2
1, 3, 0, 0 # added because index 3 was missing for id 1
1, 4, 0, 0  # added because index 4 was missing for id 1
2, 1,0,0  # added because index 1 was missing for id 2
2,2, 0.1, 0.5
2, 3, 0, 0
2,4, 0.7, 0.0
....
</code></pre>

<p>How do i perform this in operation in pandas?</p>

<p>Try this:</p>

<p>Starting with this a df: </p>

<pre><code>id  index  val_1  val_2
0   1      1    0.2    0.0
1   1      2    0.4    0.2
2   2      2    0.1    0.5
3   2      4    0.7    0.0
</code></pre>

<p>Build New DataFrame:</p>

<pre><code>df2 = pd.DataFrame({'id': np.repeat(df.id.unique(),4),'index': np.asarray([1,2,3,4]*len(df.id.unique()))}, columns = [u'id', u'index', u'val_1', u'val_2']).fillna(0)
</code></pre>

<p>Append, drop dups and sort DataFrame: </p>

<pre><code>dfx = df.append(df2).drop_duplicates(subset=['id', 'index'], keep="first")
dfx.sort_values(['id','index']).reset_index(drop=True)

       id  index  val_1  val_2
    0   1      1    0.2    0.0
    1   1      2    0.4    0.2
    2   1      3    0.0    0.0
    3   1      4    0.0    0.0
    4   2      1    0.0    0.0
    5   2      2    0.1    0.5
    6   2      3    0.0    0.0
    7   2      4    0.7    0.0
</code></pre>

<p>df2 looks like this: </p>

<pre><code>   id  index  val_1  val_2
0   1      1      0      0
1   1      2      0      0
2   1      3      0      0
3   1      4      0      0
4   2      1      0      0
5   2      2      0      0
6   2      3      0      0
7   2      4      0      0
</code></pre>

python pandas dataframe