Pandas applied function erasing data
<pre><code>In  [1]: df = pd.DataFrame([[pubA, linkA,None], [pubB, linkB,textB], [pubC, linkC,textC]], columns=['pub', 'link','text])

In  [2]: df
Out [2]: 
    pub   link   text
 0  pubA  linkA  None
 1  pubB  linkB  textB
 2  pubC  linkC  textC
</code></pre>

<p>I have code I'm using to pull text from the web. My function iterates through the <code>df</code> and checks the content of <code>'text'</code> to make sure it is empty first. If <code>'text'</code> already has content it will <code>pass</code>. If <code>'text'</code> is empty it checks  <code>'pub'</code> to see if I've got an appropriate BeautifulSoup template for that publication and if so returns clean text. If there is not yet a template the function will <code>pass</code>.</p>

<pre><code>def pull_text(row):
    try:
        if(pd.isnull(row['text'])):
            if row['publication' ] == 'PubA':
                print('Now serving row',row.name,'of',len(df),'Template:',row['publication'])
                sys.stdout.flush()

                #Do Template A
                time.sleep(rand)
                return article.strip()

            elif row['publication' ] == 'PubB':
                #Do Template B
                time.sleep(rand)
                return article.strip()

            elif row['publication' ] == 'PubC':
                # Do Template C
                rand = randint(2,10)
                print('Waiting', rand, 'seconds')
                sys.stdout.flush()

                time.sleep(rand)
                return result.strip()
            else:
                pass
                print('No template set for', row['publication'],':row', row.name)
        else:
            pass
    except AttributeError:
        print('error at',row.name)
        sys.stdout.flush()
        return 'error'

df['text'] = df.apply (lambda row: pull_text (row),axis=1)
</code></pre>

<p>Each template works fine and has pulled text from each publication. However whenever I run the function (such as after adding a new template) it seems to erase all pre-existing text data and fills in the pre-existing blanks (where it can).</p>

<pre><code>In  [3] df['text'] = df.apply (lambda row: pull_text (row),axis=1)
In  [4] df
Out [4] pub   link   text
     0  pubA  linkA  textA
     1  pubB  linkB  None
     2  pubC  linkC  None
</code></pre>

<p>Whilst what I desire is</p>

<pre><code>Out [4] pub   link   text
     0  pubA  linkA  textA
     1  pubB  linkB  textB
     2  pubC  linkC  textC
</code></pre>

<p>All I can think of is that I am somehow setting the value of <code>'text'</code> as a null value if it is not already, but I'm not sure how I'm doing it.</p>

<p>What is going on is that <code>pass</code> makes your function return <code>None</code>. </p>

python pandas