How do I find the rates TP, TN, FP, FN and measure the quality of segmented algorithm?
<p>Currently I am implementing a system that counts the number of passengers crossing a line of interest in a subway station. To measure the quality of the segmentation algorithm, I installed a video camera on the ceiling of the subway station and I recorded a video of 13 seconds.</p>

<p>The video of 13 seconds contains 412 frames.</p>

<p>Below is my questions:</p>

<ol>
<li><p>To obtain the TPR (True Positive Rate) is necessary to analyze the
frames manually, ie check every one of the 412 frames and count the
times that were labeled correctly? While the FPR, is counted only
the "false alarms"?</p></li>
<li><p>To test each algorithm, should I use the correction filters? Or just
analyze them raw form (no filters)?</p></li>
<li><p>Detection of these rates cover when you're crossing the line or across your screen extension?</p></li>
</ol>

<p><a href="http://i.stack.imgur.com/oasmN.jpg" rel="nofollow">Here is a portion of the 412 frames</a></p>

<p>I ask your help to solve this problem.</p>

<p>You definitely have to manually tag each time a man is crossing the line. This part is crucial in order to be able to evaluate your algorithm correctly. </p>

<p>I suggest you create a groundtruth file which list all frame indices when someone cross the line.
Your algorithm output should be of the same type - frame indices where it detects line crossing (if you are using segmentation or other technique it shouldn't affect the evaluation process).</p>

<p>The last thing you should do is to define a match criteria : if there is a real line crossing in frame_i and your algorithm detects a crossing in frame_j, then it will be count as true positive if abs(frame_i-frame_j) &lt; threshold, otherwise it will be count as false positive.</p>

python opencv computer-vision video-processing precision-recall