How to print only printable charcters in binary file (equvalent to strings under Linux)?
<p>I am undertaking conversion of my python application from python 2 to python 3. One of the functions which I use is to get the printable character out of binary file. I earlier used following function in python 2 and it worked great:</p>

<pre><code>import string

def strings(filename, min=4):
    with open(filename, "rb") as f:
        result = ""
        for c in f.read():
            if c in string.printable:
                result += c
                continue
            if len(result) &gt;= min:
                yield result
            result = ""
        if len(result) &gt;= min:  # catch result at EOF
            yield result
</code></pre>

<p>Code is actually from <a href="http://stackoverflow.com/questions/17195924/python-equivalent-of-unix-strings-utility">Python equivalent of unix &quot;strings&quot; utility</a>. When I run the above code with python 2 it produces the output like this which is absolutely ok for me:</p>

<pre><code> +s
^!1^
i*Q(
}"~ 
%lh!ghY
#dh!
!`,!
mL#H
o!&lt;XXT0
'   &lt; 
z !Uk
%
 wS
n`  !wl
*ty

(Q  6
!XPLO$
E#kF
</code></pre>

<p>However, the function gives weird results under python 3. It produces the error:</p>

<pre><code>TypeError: 'in &lt;string&gt;' requires string as left operand, not int
</code></pre>

<p>So I converted the 'int' to 'str' by replacing this </p>

<pre><code>if c in string.printable:
</code></pre>

<p>with this</p>

<pre><code>if str(c) in string.printable:
</code></pre>

<p>(I also converted all the places where the same error message is thrown)</p>

<p>Now the python 3 gives the following output:</p>

<pre><code>56700
0000000000000000000000000000000000000000
1236
60000
400234
00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000
2340
0000
5010
5000
17889
2348
23400000000
5600
</code></pre>

<p>I cant see any characters when I use python 3. Any help to get the code working or pointer to the solution is appreciated. All I require is to extract the strings from binary file (very small with few kb) and store it in a variable.</p>

<p>In Python 3, opening a file in binary mode gives you <code>bytes</code> results. Iterating over a <code>bytes</code> object gives you <em>integers</em>, not characters, in the range 0 to 255 (inclusive). From the <a href="https://docs.python.org/3/library/stdtypes.html#bytes" rel="nofollow"><code>bytes</code> documentation</a>:</p>

<blockquote>
  <p>While bytes literals and representations are based on ASCII text, <code>bytes</code> objects actually behave like immutable sequences of integers, with each value in the sequence restricted such that <code>0 &lt;= x &lt; 256</code></p>
</blockquote>

<p>Convert <code>string.printable</code> to a set and test against that:</p>

<pre><code>printable = {ord(c) for c in string.printable}
</code></pre>

<p>and</p>

<pre><code>if c in printable:
</code></pre>

<p>Next, you want to append to a <code>bytesarray()</code> object to keep things reasonably performant, and decode from ASCII to produce a <code>str</code> result:</p>

<pre><code>printable = {ord(c) for c in string.printable}

with open(filename, "rb") as f:
    result = bytearray()
    for c in f.read():
        if c in printable:
            result.append(c)
            continue
        if len(result) &gt;= min:
            yield result.decode('ASCII')
            result.clear()
    if len(result) &gt;= min:  # catch result at EOF
        yield result
</code></pre>

<p>Rather than iterate over the bytes one by one, you could instead split on anything that is <em>not</em> printable:</p>

<pre><code>import re

nonprintable = re.compile(b'[^%s]+' % re.escape(string.printable.encode('ascii')))

with open(filename, "rb") as f:
    for result in nonprintable.split(f.read()):
        if result:
            yield result.decode('ASCII')
</code></pre>

<p>I'd explore reading the file in <em>chunks</em> rather than in one go; don't try to fit a large file into memory in one go here:</p>

<pre><code>with open(filename, "rb") as f:
    buffer = b''
    for chunk in iter(lambda: f.read(2048), b''):
        splitresult = nonprintable.split(buffer + chunk)            
        buffer = splitresult.pop()
        for string in splitresult:
            if string:
                yield string.decode('ascii')
    if buffer:
        yield buffer.decode('ascii')
</code></pre>

<p>The buffer carries over any incomplete word from one chunk to the next; <code>re.split()</code> produces empty values at the start and end if the input started or ended with non-printable characters, respectively.</p>

python string python-3.x