Find all the occurences of a string in an imperfect text
<p>I am trying to find a string within a long text extracted from a PDF file, and get the string's position in the text, and then return 100 words before the string and 100 after.
The problem is that the extraction is not perfect, so I am having a problem like this:</p>

<p>The query string is "test text"</p>

<p>The text may look like:</p>

<blockquote>
  <p>This is atest textwith a problem</p>
</blockquote>

<p>as you can see the word "test" is joined with the letter "a" and the word "text" is joined with the word "with"</p>

<p>So the only function is working with me is __contains __ which doesn't return the position of the word.</p>

<p>Any ideas to find all the occurences of a word in such a text with their postions?</p>

<p>Thank you very much</p>

<p>You could take the following kind of approach. This first attempts to split the whole text into words, and keeps note of the index of each word. </p>

<p>Next it iterates through the text looking for <code>test text</code> with possible 0 or more spaces between. For each match it notes the start and then creates a list of words found before and after that point using Python's <code>bisect</code> library to locate the required entries in the <code>words</code> list.</p>

<pre><code>import bisect
import re

test = "aa bb cc dd test text ee ff gg testtextwith hh ii jj"

words = [(w.start(), w.group(0)) for w in re.finditer(r'(\b\w+?\b)', test)]

adjacent_words = 2

for match in re.finditer(r'(test\s*?text)', test):
    start, end = match.span()

    words_start = bisect.bisect_left(words, (start, ''))
    words_end = bisect.bisect_right(words, (end, ''))

    words_before = [w for i, w in words[words_start-adjacent_words : words_start]]
    words_after = [w for i, w in words[words_end : words_end + adjacent_words]]

    #  Adjacent words as a list
    print words_before, match.group(0), words_after

    # Or, surrounding text as is.
    print test[words[words_start-adjacent_words][0] : words[words_end+adjacent_words][0]]

    print
</code></pre>

<p>So for this example with 2 adjacent words, you would get the following output:</p>

<pre><code>['cc', 'dd'] test text ['ee', 'ff']
cc dd test text ee ff 

['ff', 'gg'] testtext ['hh', 'ii']
ff gg testtextwith hh ii
</code></pre>

python