Python server is sending truncated images over TCP
<p>I have a simple server on my Windows PC written in python that reads files from a directory and then sends the file to the client via TCP.</p>

<p>Files like HTML and Javascript are received by the client correctly (sent and original file match).<br>
The issue is that <strong>image data is truncated</strong>.  </p>

<p>Oddly, different images are truncated at different lengths, but it's consistent per image.<br>
For example, a specific 1MB JPG is always received as 95 bytes. Another image which should be 7KB, is received as 120 bytes.</p>

<p>Opening the truncated image files in notepad++, the data that <em>is</em> there is correct. (The only issue is that the file ends too soon).<br>
I do not see a pattern for where the files end. The chars/bytes immediately before and after truncation are different per image. </p>

<p>I've tried three different ways for the server to read the files, but they all have the same result.</p>

<p>Here is a snippet of the reading and sending of files:</p>

<pre><code>print ("Cache size=" + str(os.stat(filename).st_size))

#1st attempt, using readlines
fileobj = open(filename, "r") 
cacheBuffer = fileobj.readlines() 
for i in range(0, len(cacheBuffer)):
    tcpCliSock.send(cacheBuffer[i])
</code></pre>

<p>&#13;</p>

<pre><code>#2nd attempt, using line, same result
with open(filename) as f:
    for line in f:
        tcpCliSock.send(f)
</code></pre>

<p>&#13;</p>

<pre><code>#3rd attempt, using f.read(), same result
with open(filename) as f:
    tcpCliSock.send(f.read())
</code></pre>

<p>The script prints to the console the size of the file read, and <em>the number of bytes matches the original image</em>. So this proves the problem is in sending, right?<br>
If the issue is with sending, what can I change to have the whole image sent properly?</p>

<p>Since you're dealing with images, which are binary files, you need to open the files in <em>binary</em> mode.</p>

<pre><code>open(filename, 'rb')
</code></pre>

<p>From the Python documentation for <a href="https://docs.python.org/2/library/functions.html#open" rel="nofollow"><code>open()</code></a>:</p>

<blockquote>
  <p>The default is to use text mode, which may convert <code>'\n'</code> characters to a platform-specific representation on writing and back on reading. Thus, when opening a binary file, you should append <code>'b'</code> to the mode value to open the file in binary mode, which will improve portability. (Appending <code>'b'</code> is useful even on systems that donât treat binary and text files differently, where it serves as documentation.)</p>
</blockquote>

<p>Since your server is running on Windows, as you <code>read</code> the file, Python is converting every <code>\r\n</code> it sees to <code>\n</code>. For text files, this is nice: You can write platform-independent code that only deals with <code>\n</code> characters.  For binary files, this <em>completely corrupts your data</em>. That's why it's important to use <code>'b'</code> when dealing with binary files, but also important to leave it off when dealing with text files.</p>

<hr>

<p>Also, as TCP is a stream protocol, it's better to <em>stream</em> the data into the socket in smaller pieces. This avoids the need to read an entire file into memory, which will keep your memory usage down. Like this:</p>

<pre><code>with open(filename, 'rb') as f:
    while True:
        data = f.read(4096)
        if len(data) == 0:
            break
        tcpCliSock.send(data)
</code></pre>

python tcp server