tcp python socket hold connection forever
<p>I have a client-server model where the client will constantly checking a log file and as soon as a new line comes in the log file it sends that line to the server. </p>

<p>Somehow I managed to work this thing using the following code.</p>

<p><strong>server.py</strong></p>

<pre><code>import SocketServer


class MyTCPSocketHandler(SocketServer.BaseRequestHandler):

    def handle(self):
        # self.request is the TCP socket connected to the client
        data = self.request.recv(1024).strip()
        print data
        # process the data..


if __name__ == "__main__":

    HOST, PORT = "localhost", 9999
    server = SocketServer.TCPServer((HOST, PORT), MyTCPSocketHandler)
    server.serve_forever()
</code></pre>

<p><strong>client.py</strong></p>

<pre><code>import time
import socket


def follow(thefile):
    thefile.seek(0, 2)
    while True:
        line = thefile.readline()
        if not line:
            time.sleep(0.1)
            continue
        yield line


def connect_socket():
    HOST, PORT = "localhost", 9999
    sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)

    sock.connect((HOST, PORT))
    return sock

if __name__ == '__main__':

    logfile = open("my_log.log")
    loglines = follow(logfile)
    for line in loglines:
        sock = connect_socket()
        # send data
        sock.sendall(bytes(line))
</code></pre>

<p>the problem is every time I need to call the <code>connect_socket()</code> method to send a new line.</p>

<p>I'm quite new to this topic so somebody please let me know is there any workaround for this to work in a such a way that once the connection is established between client and server I need to send data continuously to the server without making a new connection again and again. </p>

<p>If I'm connecting only one time and using the same socket object to send data it was throwing </p>

<blockquote>
  <p>socket.error: [Errno 32] Broken pipe</p>
</blockquote>

<p>Some StackOverflow links which I have followed are given below,</p>

<p><a href="http://stackoverflow.com/questions/28021928/tcp-server-client-errno-32-broken-pipe">1</a>, <a href="http://stackoverflow.com/questions/180095/how-to-handle-a-broken-pipe-sigpipe-in-python">2</a>, <a href="http://stackoverflow.com/questions/6237569/python-socket-send-can-only-send-once-then-socket-error-errno-32-broken-pi">3</a></p>

<p>One thing I found is </p>

<blockquote>
  <p>Broken Pipe occurs when one end of the connection tries sending data while the other end has already closed the connection.</p>
</blockquote>

<p>How can I keep the connection open on both ends?
For this use case should I go for an asynchronous method and if so  which framework will be the best match <code>tornado</code> or <code>twisted</code>?</p>

<p>After a line is transmitted, you close the connection on the client, but don't close it on the server.</p>

<p>From <a href="https://docs.python.org/3.4/library/socketserver.html#socketserver.RequestHandler.finish" rel="nofollow">the docs</a>:</p>

<blockquote>
  <p>RequestHandler.finish()</p>
  
  <p>Called after the handle() method to perform any clean-up actions required. <strong>The default implementation does nothing</strong>.</p>
</blockquote>

<p>So you should implement <code>finish</code> as well and close the socket (<code>self.request</code>) there.</p>

<hr>

<p>Another option is not to close the connection on the client:</p>

<pre><code>sock = connect_socket()
for line in loglines:
    # send data
    sock.sendall(bytes(line))
sock.sendall(b'bye')  # EOF
sock.close()
</code></pre>

<p>However in this case you should modify the server code and make sure it can serve multiple clients and it understands when to close the socket. With all these difficulties it is nevertheless the preferable way of doing things. Ideally, a client has a single TCP connection per session for they're costly to establish.</p>

python sockets tcp