Extracting hand writing text out in shape with OpenCV
<p>I am very new to OpenCV Python and I really need some help here.</p>

<p>So what I am trying to do here is to extract out these words in the image below.</p>

<p><a href="http://i.stack.imgur.com/XsSOP.jpg" rel="nofollow"><img src="http://i.stack.imgur.com/XsSOP.jpg" alt="hand drawn image"></a></p>

<p>The words and shapes are all hand drawn, so they are not perfect. I have did some coding below. </p>

<p>First of all, I grayscale the image</p>

<pre><code>img_final = cv2.imread(file_name)
img2gray = cv2.cvtColor(img,cv2.COLOR_BGR2GRAY)
</code></pre>

<p>Then I use THRESH_INV to show the content</p>

<pre><code>ret, new_img = cv2.threshold(image_final, 100 , 255, cv2.THRESH_BINARY_INV)
</code></pre>

<p>After which, I dilate the content</p>

<pre><code>kernel = cv2.getStructuringElement(cv2.MORPH_CROSS,(3 , 3)) 
dilated = cv2.dilate(new_img,kernel,iterations = 3)
</code></pre>

<p>I dilate the image is because I can identify text as one cluster</p>

<p>After that, I apply boundingRect around the contour and draw around the rectangle</p>

<pre><code>contours, hierarchy = cv2.findContours(dilated,cv2.RETR_EXTERNAL,cv2.CHAIN_APPROX_NONE) # get contours
index = 0
for contour in contours:

    # get rectangle bounding contour
    [x,y,w,h] = cv2.boundingRect(contour)

    #Don't plot small false positives that aren't text
    if w &lt; 10 or h &lt; 10:
        continue

    # draw rectangle around contour on original image
    cv2.rectangle(img,(x,y),(x+w,y+h),(255,0,255),2)
</code></pre>

<p>This is what I got after that.</p>

<p><a href="http://i.stack.imgur.com/vxKYi.png" rel="nofollow"><img src="http://i.stack.imgur.com/vxKYi.png" alt="result image"></a></p>

<p>I am only able to detect one of the text. I have tried many other methods but this is the closet results I have got and it does not fulfill the requirement.</p>

<p>The reason for me to identify the text is so that I can get the X and Y coordinate of each of the text in this image by putting a bounding Rectangle "boundingRect()".</p>

<p>Please help me out. Thank you so much</p>

<p>You can use the fact that the connected component of the letters are much smaller than the large strokes of the rest of the diagram.</p>

<p>I used opencv3 connected components in the code but you can do the same things using findContours.</p>

<p>The code:</p>

<pre><code>import cv2
import numpy as np

# Params
maxArea = 150
minArea = 10

# Read image
I = cv2.imread('i.jpg')

# Convert to gray
Igray = cv2.cvtColor(I,cv2.COLOR_RGB2GRAY)

# Threshold
ret, Ithresh = cv2.threshold(Igray,0,255,cv2.THRESH_BINARY_INV+cv2.THRESH_OTSU)

# Keep only small components but not to small
comp = cv2.connectedComponentsWithStats(Ithresh)

labels = comp[1]
labelStats = comp[2]
labelAreas = labelStats[:,4]

for compLabel in range(1,comp[0],1):

    if labelAreas[compLabel] &gt; maxArea or labelAreas[compLabel] &lt; minArea:
        labels[labels==compLabel] = 0

labels[labels&gt;0] =  1

# Do dilation
se = cv2.getStructuringElement(cv2.MORPH_ELLIPSE,(25,25))
IdilateText = cv2.morphologyEx(labels.astype(np.uint8),cv2.MORPH_DILATE,se)

# Find connected component again
comp = cv2.connectedComponentsWithStats(IdilateText)

# Draw a rectangle around the text
labels = comp[1]
labelStats = comp[2]
#labelAreas = labelStats[:,4]

for compLabel in range(1,comp[0],1):

    cv2.rectangle(I,(labelStats[compLabel,0],labelStats[compLabel,1]),(labelStats[compLabel,0]+labelStats[compLabel,2],labelStats[compLabel,1]+labelStats[compLabel,3]),(0,0,255),2)
</code></pre>

<p><a href="http://i.stack.imgur.com/k0UdG.jpg" rel="nofollow"><img src="http://i.stack.imgur.com/k0UdG.jpg" alt="enter image description here"></a></p>

python opencv