Python socket recv() doesn't get every message if send too fast
<p>I send mouse coordinates from python server to python client via socket. Mouse coordinates are send every time when mouse movement event is catch on the server which means quite often (dozen or so per second).</p>

<p>Problem is when I use python server and python client on different hosts. Then only part of messages are delivered to the client. 
e.g. 3 first messages are delivered, 4 messages aren't delivered, 4 messages are delivered etc...</p>

<p>Everything is fine when server and client are on the same host (localhost).</p>

<p>Everything is fine when server and client are on different hosts but instead of python client I use standard windows Telnet client to read messages from the server.</p>

<p>I noticed that when I use time.sleep(0.4) break between each message that is send then all messages are delivered. Problem is I need that information in real time not with such delay. Is it possible to achieve that in Python using sockets?</p>

<p>Below python client code that I use:</p>

<pre><code>import pickle
import socket
import sys


host = '192.168.1.222'
port = 8888
try:
    s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
except socket.error, msg:
    print "Faile. Error:" + str(msg[0]), "Error message : " + msg[1]
    sys.exit()

mySocket = socket.socket()
mySocket.connect((host,port))

while 1:
    data = mySocket.recv(1024)
    if not data: break
    load_data = pickle.loads(data)

    print 'parametr x: ' + str(load_data[0])
    print 'parametr y : ' + str(load_data[1])

mySocket.close()
</code></pre>

<p>You are using TCP (SOCK_STREAM) which is a reliable protocol which (contrary to UDP) does not loose any messages, even if the recipient is not reading the data fast enough. Instead TCP will reduce the sending speed.<br>
This means that the problem must be somewhere in your application code. </p>

<p>One possibility is that the problem is in your sender, i.e. that you use <code>socket.send</code> and do not check if all the bytes you've intended to send are really got send. But this check needs to be done since <code>socket.send</code> might only send part of the data if the socket buffer of the OS is full which can happen if the client does not read the data fast enough.</p>

<p>Another possibility is that your <code>socket.recv</code> call receives more data than your <code>pickle.loads</code> needs and that the rest of the data gets discarded (not sure if <code>pickle.loads</code> will throw an exception if too much data are provided). Note that TCP is not a message but a stream protocol so it might be that you have more that <code>socket.recv</code> will return a buffer which contains more than one pickled object but you only read the first. The chance that this will happen on a network is higher than on localhost because by default the TCP layer will try to concatenate multiple <code>send</code> buffers into a single TCP packet for better use of the connection (i.e. less overhead). And the chance is high that these will then be received within the same <code>recv</code> call. By putting a <code>sleep(0.4)</code> on the sender side you've effectively switch off this optimization of TCP, see <a href="https://en.wikipedia.org/wiki/Nagle%27s_algorithm" rel="nofollow">NAGLE algorithm</a> for details.</p>

<p>Thus the correct way to implement what you want would be:</p>

<ul>
<li>Make sure that all data are delivered at the server, i.e. check the return of <code>socket.send</code>.</li>
<li>Make sure that you unpack all messages you receive. To do this you probable need to add some message layer on top of the TCP stream to find out where the message boundary is.</li>
</ul>

python sockets