How to write python code that finishes a read from stdin even though the read buffer isn't full
<p>Python question:</p>

<pre><code>$ python -V
Python 2.4.3
</code></pre>

<p>Searched for the answer to this and maybe didn't know what search to use.</p>

<p>Basically the question is simple.
I have perl code like this and it works perfect.</p>

<pre><code>while ($count)
{
    $count = sysread(STDIN,$data,2000);
    if ($count)
    {
        #print "Read $data\n";
        showHex($data);
        $s-&gt;mcast_send($data,"$MulticastIP:$MulticastPort");
    }
}
</code></pre>

<p>What this does, which is exactly what I want, is that sysread, even though given a max read of 2000, will return early if it has read at least one byte and there is a delay, in time, in the incoming data. </p>

<p><strong>Important: My incoming data is in raw-binary, not newline delimited text.</strong></p>

<p>I need this in python because the IO::Socket::Multicast in perl doesn't come standard in the perl libraries by default.</p>

<p>But I can't figure out how to do this in python. <strong>Python annoyingly waits until all 2000 bytes are read before returning.</strong></p>

<p>My python code: go.py (for this example I only read 10 bytes at at time)</p>

<pre><code>#!/bin/env python

import sys

while True:
    buf = sys.stdin.read(10)
    if len(buf) &gt; 0:
        print "Read %d" % (len(buf))
</code></pre>

<p>Proof of the issue: go.sh</p>

<pre><code>while (( 1 ))
do
    echo Sending ab 1&gt;&amp;2
    echo ab
    sleep 1
done | ./go.py
Sending ab
Sending ab
Sending ab
Sending ab
Read 10
Sending ab
Sending ab
Sending ab
Read 10
</code></pre>

<p>I know from experience that a read() on any file descriptor returns early if it has read at least one byte and comes to a point where there is no more data immediately available to read, like from a pipe from the stdout output of another program.</p>

<pre><code>READ(2)                    Linux Programmerâs Manual                   READ(2)

NAME
       read - read from a file descriptor

SYNOPSIS
       #include &lt;unistd.h&gt;

       ssize_t read(int fd, void *buf, size_t count);

DESCRIPTION
       read() attempts to read up to count bytes from file descriptor fd into the buffer starting at buf.

       If count is zero, read() returns zero and has no other results. If count is greater than SSIZE_MAX, the result is unspecified.

RETURN VALUE
       On  success,  the  number  of  bytes  read is returned (zero indicates end of file), and the file position is advanced by this number.  It is not an error if this number is smaller than the number of bytes
       requested; this may happen for example because fewer bytes are actually available right now (maybe because we were close to end-of-file, or because we are reading from a  pipe,  or  from  a  terminal),  or
       because read() was interrupted by a signal.  On error, -1 is returned, and errno is set appropriately. In this case it is left unspecified whether the file position (if any) changes.
</code></pre>

<p>So how do I accomplish this?</p>

<p>In the event you are able to use Python 3 instead:</p>

<pre><code>while True:
    buf = sys.stdin.buffer.raw.read(10)
    if buf::
        print("Read", len(buf))
</code></pre>

<p>Then:</p>

<pre><code>while (( 1 ))
do
    echo Sending ab 1&gt;&amp;2
    echo -n ab
    sleep 1
done | ./go.py
</code></pre>

<p>gives:</p>

<pre><code>Sending ab
Read 2
Sending ab
Read 2
Sending ab
Read 2
...
</code></pre>

python stdin