validation of python script for extracting values based on regex
<p>I have data of the form: </p>

<pre><code>submission #,scores
882,"Overall evaluation: 1
Invite to interview: 1
Strength or novelty of the idea (1): 4
Strength or novelty of the idea (2): 4
Strength or novelty of the idea (3): 3
Use or provision of open data (1): 3
Use or provision of open data (2): 3
""Open by default"" (1): 4
""Open by default"" (2): 4
Value proposition and potential scale (1): 2
Value proposition and potential scale (2): 1
Market opportunity and timing (1): 3
Market opportunity and timing (2): 1
Triple bottom line impact (1): 2
Triple bottom line impact (2): 2
Triple bottom line impact (3): 4
Knowledge and skills of the team (1): 1
Knowledge and skills of the team (2): 2
Capacity to realise the idea (1): 1
Capacity to realise the idea (2): 3
Capacity to realise the idea (3): 1
Appropriateness of the budget to realise the idea: 3"
882,"Overall evaluation: 2
Invite to interview: 3
Strength or novelty of the idea (1): 4
Strength or novelty of the idea (2): 1
Strength or novelty of the idea (3): 4
Use or provision of open data (1): 4
Use or provision of open data (2): 3
""Open by default"" (1): 3
""Open by default"" (2): 4
Value proposition and potential scale (1): 4
Value proposition and potential scale (2): 4
Market opportunity and timing (1): 4
Market opportunity and timing (2): 4
Triple bottom line impact (1): 4
Triple bottom line impact (2): 1
Triple bottom line impact (3): 3
Knowledge and skills of the team (1): 3
Knowledge and skills of the team (2): 2
Capacity to realise the idea (1): 2
Capacity to realise the idea (2): 3
Capacity to realise the idea (3): 3
Appropriateness of the budget to realise the idea: 3"
883,"Overall evaluation: 1
Invite to interview: 1
Strength or novelty of the idea (1): 4
Strength or novelty of the idea (2): 3
Strength or novelty of the idea (3): 4
Use or provision of open data (1): 2
Use or provision of open data (2): 3
""Open by default"" (1): 3
""Open by default"" (2): 3
Value proposition and potential scale (1): 2
Value proposition and potential scale (2): 1
Market opportunity and timing (1): 3
Market opportunity and timing (2): 1
Triple bottom line impact (1): 1
Triple bottom line impact (2): 4
Triple bottom line impact (3): 2
Knowledge and skills of the team (1): 3
Knowledge and skills of the team (2): 3
Capacity to realise the idea (1): 1
Capacity to realise the idea (2): 3
Capacity to realise the idea (3): 4
Appropriateness of the budget to realise the idea: 3"
883,"Overall evaluation: 1
Invite to interview: 1
Strength or novelty of the idea (1): 2
Strength or novelty of the idea (2): 2
Strength or novelty of the idea (3): 1
Use or provision of open data (1): 2
Use or provision of open data (2): 1
""Open by default"" (1): 3
""Open by default"" (2): 2
Value proposition and potential scale (1): 2
Value proposition and potential scale (2): 2
Market opportunity and timing (1): 2
Market opportunity and timing (2): 2
Triple bottom line impact (1): 1
Triple bottom line impact (2): 2
Triple bottom line impact (3): 2
Knowledge and skills of the team (1): 4
Knowledge and skills of the team (2): 2
Capacity to realise the idea (1): 2
Capacity to realise the idea (2): 2
Capacity to realise the idea (3): 3
Appropriateness of the budget to realise the idea: 3"
885,"Overall evaluation: 2
Invite to interview: 1
Strength or novelty of the idea (1): 2
Strength or novelty of the idea (2): 2
Strength or novelty of the idea (3): 2
Use or provision of open data (1): 2
Use or provision of open data (2): 2
""Open by default"" (1): 2
""Open by default"" (2): 2
Value proposition and potential scale (1): 1
Value proposition and potential scale (2): 2
Market opportunity and timing (1): 2
Market opportunity and timing (2): 1
Triple bottom line impact (1): 2
Triple bottom line impact (2): 1
Triple bottom line impact (3): 1
Knowledge and skills of the team (1): 4
Knowledge and skills of the team (2): 2
Capacity to realise the idea (1): 1
Capacity to realise the idea (2): 3
Capacity to realise the idea (3): 2
Appropriateness of the budget to realise the idea: 3"
</code></pre>

<p>and the following python script: </p>

<pre><code>map = {}
lines=open("new_data.csv",'r').read().splitlines()
for l in lines:
    data = l.split('"Overall evaluation:')
    if len(data) == 2:
        if data[0] not in map.keys():
            map[data[0]] = (0,0)
        map[data[0]] = (map[data[0]][0]+int(data[1]) , map[data[0]][1]+1)
for x, y in map.items():
    print(str(x) + ", " + str(y[0]/y[1]))
</code></pre>

<p>what I think is happening is that it takes the average of the two <code>Overall evaluation:</code> numbers and outputs it next to the submission number, is that correct? </p>

<p>Your <code>map</code> values are each tuples corresponding with the number of items seen and the total of all values seen for the same item.</p>

<p>Dividing the two does indeed return the average (though since they're integers, that result is rounded -- consider casting one or both to floating-point if you want a floating-point result rather than an integer result).</p>

python regex