Passing captured video frame (numpy array object) from webcam feed to caffe model
<p>I am a beginner in Caffe and I am trying to use the Imagenet model for object classification. My requirement is that I want to use it from a webcam feed and detect the objects from the webcam feed.For this, I use the following code</p>

<pre><code>    cap = cv2.VideoCapture(0)
    while(True):    
        ret, frame = cap.read() #frame is of type numpy array       
        #frame = caffe.io.array_to_datum(frame) 
        img = caffe.io.load_image(frame)
</code></pre>

<p>Obviously this does not work since <code>caffe.io.load_image</code> expects an image path. 
As you can see, I also tried using <code>caffe io.py</code>'s  <code>array_to_datum</code> function (got it from <a href="http://stackoverflow.com/questions/30815035/how-to-convert-mat-from-opencv-to-caffe-format">this stackoverflow question </a>) and passed the frame to caffe io load_image but this too does not work.
How can I pass the captured video frames from the webcam directly to <code>caffe io load_image</code> ?
and If that is not possible then what is the way to load the frame into <code>caffe io</code>? Please help. Thanks in advance.</p>

<p>caffe.io.load_image does not do much. It only does the following :</p>

<ol>
<li>Read image from disk (given the path)</li>
<li>Make sure that the returned image has 3 dimensions (HxWx1 or HxWx3)</li>
</ol>

<p>(see source <a href="https://github.com/BVLC/caffe/blob/master/python/caffe/io.py#L279" rel="nofollow">caffe.io.load_image</a>)</p>

<p>So it does <strong>not</strong> load the image <strong>into your model</strong>, it's just a helper function that loads an image from disk. To load an image into memory, you can load it however you like (from disk, from webcam..etc). To load the network, feed the image into it and do inference, you can do something like the following :</p>

<pre><code># Load pre-trained network
net=caffe.Net(deployprototxt_path, caffemodel_path, caffe.TEST)
# Feed network with input
net.blobs['data'].data[0,...] = frame
# Do inference
net.forward()
# Access prediction
probabilities = net.blobs['prob'].data
</code></pre>

<p>Make sure the frame dimensions match the expected input dimensions as specified in the deploy.prototxt (see <a href="https://github.com/BVLC/caffe/blob/master/models/bvlc_reference_caffenet/deploy.prototxt#L6" rel="nofollow">example for CaffeNet</a>)</p>

python opencv video caffe pycaffe