Mapping each element in list to different column in pandas dataframe
<p><strong>Background</strong>: I have a dataframe with individuals' names and addresses. I'm trying to catalog people associated with each person in my dataframe, so I'm running each row/record in the dataframe through an external API that returns a list of people associated with the individual. The idea is to write a series of functions that calls the API, returns the list of relatives, and appends each name in the list to a distinct column in the original dataframe. The code will eventually be parallelized.</p>

<p>The dataframe:</p>

<pre><code>import pandas as pd

df = pd.DataFrame({
'first_name': ['Kyle', 'Ted', 'Mary', 'Ron'],
'last_name': ['Smith', 'Jones', 'Johnson', 'Reagan'],
'address': ['123 Main Street', '456 Maple Street', '987 Tudor Place', '1600 Pennsylvania Avenue']},
columns = ['first_name', 'last_name', 'address'])
</code></pre>

<p>The first function, which calls the API and returns a list of names:</p>

<pre><code>import requests
import json
import numpy as np
from multiprocessing import Pool

def API_call(row):
    api_key = '123samplekey'
    first_name = str(row['First_Name'])
    last_name = str(row['Last_Name'])
    address = str(row['Street_Address'])
    url = 'https://apiaddress.com/' + '?first_name=' + first_name + '?last_name=' + last_name + '?address' = address + '?api_key' + api_key
    response = requests.get(url)
    JSON = response.json()
    name_list = []
    for index, person in enumerate(JSON['people']):
        name = JSON['people'].get('name')
        name_list.append(name)
    return name_list
</code></pre>

<p>This function works well. For each person in the dataframe, a list of family/friends is returned. So, for Kyle Smith, the function returns <code>[Heather Smith, Dan Smith]</code>, for Ted Jones the function returns <code>[Al Jones, Karen Jones, Tiffany Jones, Natalie Jones]</code>, and so on for each row/record in the dataframe.</p>

<p><strong>Problem:</strong> I'm struggling to write a subsequent function that will iterate through the returned list and append each value to a unique column that corresponds to the searched name in the dataframe. I want the function to return a database that looks like this:</p>

<pre><code>First_Name | Last_Name  | Street_Address           | relative1_name  | relative2_name  | relative3_name   | relative4_name
-----------------------------------------------------------------------------------------------------------------------------
Kyle       | Smith      | 123 Main Street          | Heather Smith   | Dan Smith       |                  |
Ted        | Jones      | 456 Maple Street         | Al Jones        | Karen Jones     | Tiffany Jones    | Natalie Jones
Mary       | Johnson    | 987 Tudor Place          | Kevin Johnson   |                 |                  |
Ron        | Reagan     | 1600 Pennsylvania Avenue | Nancy Reagan    | Patti Davis     | Michael Reagan   | Christine Reagan
</code></pre>

<p>NOTE: The goal is to vectorize everything, so that I can use the <code>apply</code> method and eventually run the whole thing in parallel. Something along the lines of the following code has worked for me in the past, when the "API_call" function was returning a single object instead of a list that needed to be iterated/mapped:</p>

<pre><code>def API_call(row):
    # all API parameters
    url = 'https//api.com/parameters'
    response = request.get(url)
    JSON = response.json()
    single_object = JSON['key1']['key2'].get('key3')
    return single_object

def second_function(data):
    data['single_object'] = data.apply(API_call, axis =1)
    return data

def parallelize(dataframe, function):
    df_splits = np.array_split(dataframe, 10)
    pool = Pool(4)
    df_whole = pd.concat(pool.map(function, df_splits))
    pool.close()
    pool.join()
    return df_whole

parallelize(df, second_function)
</code></pre>

<p>The problem is I just can't write a vectorizable function (second_function) that maps names from the list returned by the API to unique columns in the original dataframe. Thanks in advance for any help! </p>

<pre><code>import pandas as pd


def make_relatives_frame(relatives):

    return pd.DataFrame(data=[relatives],
                        columns=["relative%i_name" % x for x in range(1, len(relatives) + 1)])

# example output from an API call
df_names = pd.DataFrame(data=[["Kyle", "Smith"]], columns=["First_Name", "Last_Name"])
relatives = ["Heather Smith", "Dan Smith"]
df_relatives = make_relatives_frame(relatives)
df_names[df_relatives.columns] = df_relatives

# example output from another API Call with more relatives
df_names2 = pd.DataFrame(data=[["John", "Smith"]], columns=["First_Name", "Last_Name"])
relatives2 = ["Heath Smith", "Daryl Smith", "Scott Smith"]
df_relatives2 = make_relatives_frame(relatives2)
df_names2[df_relatives2.columns] = df_relatives2

# example of stacking the outputs
total_df = df_names.append(df_names2)

print total_df
</code></pre>

<p>The above code should get you started. Obviously it is just a representative example, but you should be able to refactor it to fit your specific use case.  </p>

python list function pandas vectorization