Skip to content
All Posts

Notes on Python Sockets

Python socket notes covering connection identity, listen queues, SO_REUSEADDR, TCP framing, and related API behavior.

507 wordsView all posts with this tag pythonView all posts with this tag networking

Basics

A socket is identified by a five-tuple: protocol, local address, local port, remote address, and remote port.

Sockets are full duplex.

The listen function

listen is only used with TCP. Its signature is def listen(self, backlog). The default backlog is commonly 5, although it can be adjusted, and web servers often use a higher value. listen does two things:

  1. It converts an unconnected socket into a passive socket, telling the kernel to accept connection requests for it.
  2. It specifies the maximum number of connections the kernel may queue for the socket.

For a given socket, the kernel maintains both an incomplete connection queue and a completed connection queue. Connections that finish the three-way handshake move from the first queue to the second. accept removes one connection from the completed queue for the current process. If the queue is empty, the process sleeps, reflecting the event-driven nature of the operation.

backlog is the maximum combined size of the two queues. Once they exceed that value, the kernel ignores new connections without sending an RST, allowing the client to retransmit its SYN through TCP’s retry mechanism.

A backlog of 0 means that the number of connections is unlimited. This may enable SYN flooding, so the value is not recommended.

socket.SO_REUSEADDR

socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)

This option can be used in four situations:

  • socket1 with the same local address and port is in TIME_WAIT, while the newly started program needs socket2 to use that address and port.
  • SO_REUSEADDR allows multiple instances of the same server—multiple processes—to start on one port, as long as each instance binds to a different IP address. This can be tested on a machine with multiple network interfaces or IP aliases.
  • SO_REUSEADDR allows one process to bind the same port to multiple sockets, provided that every socket binds to a different IP address. This resembles the previous case; see UNPv1 for the distinction.
  • SO_REUSEADDR allows duplicate bindings of exactly the same address and port. This applies only to UDP multicast, not TCP.

Making a listening address reusable has several practical uses:

  1. It prevents a bind error when restarting a listener.
  2. It can be used for TCP NAT traversal.

Handling TCP packet coalescing

Unlike UDP, TCP is a stream protocol. A stream is continuous and has no inherent boundaries, so applications must frame their own messages. Network and sending conditions may also cause received chunks to differ from the chunks originally sent, requiring explicit handling.

For example, to avoid buffering delays, use sendall to send data immediately instead of waiting for a buffer to fill, and enable NODELAY on the socket:

sock.setsockopt(socket.IPPROTO_TCP, socket.TCP_NODELAY, 1)

The underlying problem is how to define message boundaries in TCP data. There are three general solutions:

  1. Send fixed-length messages.
  2. Send each message’s size together with the message.
  3. Use a special delimiter between messages.

The second approach is probably the most common. It is easy to implement and has little performance impact.

HTTP uses all three ideas, as illustrated below.

HTTP message framing

HTTP headers use CRLF, or carriage return plus line feed, to separate fields. The message-body length determines where the body ends, and EOF (ASCII 255) closes the connection. Two consecutive CRLF sequences separate the headers from the body.

Originally published on SegmentFault.