Asyncio Echo Unix Socket Server

March 31, 2024 Python Asyncio

We can develop an echo client and server with Unix sockets in asyncio.

An echo client and server is a simple protocol where the server expects to receive one message, and then sends it back to the client.

Unix sockets are a simple and efficient way of connecting processes using a file-based socket. As their name suggests, they are only supported on Unix and Unix-like platforms such as Linux and macOS.

In this tutorial, you will discover how to develop an asyncio echo client and server with Unix sockets.

After completing this tutorial, you will know:

About Unix sockets in asyncio and echo client and server applications.
How to develop an echo server in asyncio with Unix sockets.
How to develop an echo client in asyncio with Unix sockets.

Let's get started.

What Are Unix Sockets

UNIX sockets, also known as IPC (Inter-Process Communication) sockets or Unix Domain sockets, provide a communication mechanism between processes on the same Unix-like operating system.

A Unix domain socket aka UDS or IPC socket (inter-process communication socket) is a data communications endpoint for exchanging data between processes executing on the same host operating system. It is also referred to by its address family AF_UNIX.
-- Unix domain socket, Wikipedia.

Unlike TCP sockets, UNIX sockets operate locally within a single machine, offering a fast and efficient way for processes to exchange data without the overhead associated with network communication.

UNIX sockets leverage the file system to establish communication channels between processes. They are represented as special files in the file system hierarchy, residing in the file namespace. Processes can communicate through these sockets using standard file I/O operations, such as reading and writing.

One notable advantage of UNIX sockets is their low overhead and high speed, as they operate entirely within the kernel and avoid the complexities of network protocols. This makes them ideal for communication between processes on the same machine, providing a lightweight and efficient alternative to network-based communication.

UNIX sockets are commonly used for communication between different components of a single application or between separate applications running on the same host. They are well-suited for scenarios where low latency and high throughput are crucial, such as communication between a web server and a database server on the same machine.

Security is another notable aspect of UNIX sockets. Since they operate locally, there is no exposure to network-based attacks. Access to UNIX sockets is governed by file permissions, providing a level of control over which processes can communicate through a particular socket.

You can learn more about Unix sockets in the article:

Unix domain socket

Now that we know about Unix sockets, let's look at how to create a Unix socket server in asyncio.

How to Create a Unix Socket Server

We can create an asyncio Unix socket server using the asyncio.start_unix_server() function.

This function takes many arguments, although critically these include a path to the file that represents the socket.

path is the name of a Unix domain socket, and is required, unless a sock argument is provided.
-- Event Loop, Asyncio API Documentation.

For example:

...
# create a unix socket server
server = await asyncio.start_unix_server('./unix.socket')

The server will accept socket.AF_UNIX family connections that are the stream type, e.g. socket.SOCK_STREAM.

This server can only be created on Unix and Unix-like platforms (e.g. Linux and macOS, not Windows).

Now that we know how to create a Unix socket server, let's look at an echo client and server.

What is An Echo Client and Server

An echo client and server is a common first example when learning network programming.

It is designed to illustrate bidirectional communication between two endpoints: a client and a server.

Developing an echo client and server program provides an excellent way to learn the basics of how to use streams in asyncio.

Server: The echo server listens for incoming connections, and upon receiving data from a client, it sends the same data back.
Client: The echo client connects to the server, sends a message, and waits for the reply from the server before disconnecting.

The client connects once and disconnects, whereas the server accepts multiple concurrent connections.

Developing an echo client and server in asyncio provides a practical introduction to asynchronous programming concepts, showcasing how tasks can be scheduled efficiently, I/O operations can be non-blocking, and multiple connections can be managed concurrently.

Now that we know what an echo client and server are, let's look at how to develop an example using Unix sockets.

Example of Echo Unix Socket Client and Server

We can develop an echo client and server using Unix sockets.

Note, that this example will only work on those platforms that support Unix sockets (e.g. Linux and macOS) and may not work on Windows.

This section is divided into three parts:

Develop an echo server using Unix sockets.
Develop an echo client using Unix sockets.
Example interaction between client and server.

Let's dive in.

Example of Unix Socket Server

We can explore how to develop an echo server with Unix sockets.

In this case, the server is divided into two main parts, they are the handler coroutine for handling client socket connections and the main coroutine for accepting client connections.

Firstly, we can develop the handler for client connections.

This is the client callback coroutine called when a new client connection is made.

The protocol for the echo server is simple: read a message and write it back.

The first step is to report progress to indicate that a client is connected, then to read a message from the client and report it. Messages are read in bytes and can be converted to a Python string via the decode() method.

For example:

...
# report progress
print('Client connected')
# read the message from the client
msg_bytes = await reader.readline()
# report the message
print(f'Got: {msg_bytes.decode().strip()}')

Next, we can sleep for a moment to slow down the interaction.

...
# wait a moment
await asyncio.sleep(1)

We can then echo back the message from the client and wait for the message to be transmitted.

...
# report progress
print('Echoing message...')
# send the message back
writer.write(msg_bytes)
# wait for the buffer to empty
await writer.drain()

We can then close the connection with the client and wait for the connection to close.

...
# close the connection
writer.close()
await writer.wait_closed()
# report progress
print('Closing connection')

Tying this together, the handle_echo_client() coroutine below implements the handling of the client connection.

# handle an echo client connection
async def handle_echo_client(reader, writer):
    # report progress
    print('Client connected')
    # read the message from the client
    msg_bytes = await reader.readline()
    # report the message
    print(f'Got: {msg_bytes.decode().strip()}')
    # wait a moment
    await asyncio.sleep(1)
    # report progress
    print('Echoing message...')
    # send the message back
    writer.write(msg_bytes)
    # wait for the buffer to empty
    await writer.drain()
    # close the connection
    writer.close()
    await writer.wait_closed()
    # report progress
    print('Closing connection')

For more on how to use asyncio streams, see the tutorial:

Asyncio Streams in Python

Next, we can develop the main coroutine to manage the server.

This involves first creating the server and specifying the client callback coroutine to run for each new connection and the local file used to represent the socket connections.

...
# create the server
server = await asyncio.start_unix_server(handle_echo_client, 'unix.socket')

Next, we can accept client connections forever. This involves calling the serve_forever() method on the server.

This can be performed within the context manager interface of the server instance, ensuring that the server is closed safely on error or when the program is terminated.

...
# run the server
async with server:
    # report message
    print('Server running...')
    # accept connections
    await server.serve_forever()

You can learn more about the asyncio server context manager in the tutorial:

Asyncio Server Context Manager

Tying this together, the main() coroutine used to manage the server is listed below.

# echo server
async def main():
    # create the server
    server = await asyncio.start_unix_server(handle_echo_client, 'unix.socket')
    # run the server
    async with server:
        # report message
        print('Server running...')
        # accept connections
        await server.serve_forever()

Finally, we can start the event loop and run the main() coroutine.

...
# start the event loop
asyncio.run(main())

Tying this together, the complete example is listed below.

# SuperFastPython.com
# example of an echo server using unix sockets
import asyncio

# handle an echo client connection
async def handle_echo_client(reader, writer):
    # report progress
    print('Client connected')
    # read the message from the client
    msg_bytes = await reader.readline()
    # report the message
    print(f'Got: {msg_bytes.decode().strip()}')
    # wait a moment
    await asyncio.sleep(1)
    # report progress
    print('Echoing message...')
    # send the message back
    writer.write(msg_bytes)
    # wait for the buffer to empty
    await writer.drain()
    # close the connection
    writer.close()
    await writer.wait_closed()
    # report progress
    print('Closing connection')

# echo server
async def main():
    # create the server
    server = await asyncio.start_unix_server(handle_echo_client, 'unix.socket')
    # run the server
    async with server:
        # report message
        print('Server running...')
        # accept connections
        await server.serve_forever()

# start the event loop
asyncio.run(main())

Next, let's explore how we might develop an echo client using Unix sockets.

Example of Unix Socket Client

We can develop an asyncio echo server using Unix sockets.

In this case, we will split the client into three parts: a coroutine to send a message to the server, a coroutine to read the echoed message from the server and a coroutine to drive the overall process.

Firstly, We can develop the coroutine to send the message to the server.

The coroutine takes a string as an argument and the StreamWriter is connected to the server.

Progress is reported, then the provided message is encoded, sent, and we wait for the buffer to empty and the transmission to complete.

The send_message() coroutine below implements this.

# send message to server
async def send_message(writer, message):
    # report progress
    print(f'Sending message: {message.strip()}')
    # encode message to bytes
    msg_bytes = message.encode()
    # send the message
    writer.write(msg_bytes)
    # wait for buffer to empty
    await writer.drain()

Next, we can develop the coroutine for reading the echoed response from the server.

This coroutine takes a StreamReader connected to the server as an argument.

It reports progress, reads one line of data from the server, decodes the bytes, and reports the response.

The read_response() coroutine below implements this.

# read response from the server
async def read_response(reader):
    # report progress
    print('Waiting for response...')
    # read the response
    result_bytes = await reader.readline()
    # decode response
    response = result_bytes.decode()
    # report the response
    print(f'Received response: {response.strip()}')

Next, we can develop the main coroutine to drive the client program.

Firstly, we must open a connection to the Unix socket server, specifying the known socket file used by the server.

We can call the asyncio.open_unix_connection() function to open a connection.

...
# define the socket file
socket_file = 'unix.socket'
# report progress
print(f'Connecting to {socket_file}...')
# open a connection to the server
reader, writer = await asyncio.open_unix_connection(socket_file)
# report message
print('Connected')

Next, we can write a message by awaiting our send_message() coroutine, ensuring that the message ends with a new line, expected by both client and server.

...
# send the message
await send_message(writer, 'Hello There\n')

We can then read the echoed response by awaiting our read_response() coroutine.

...
# read response
await read_response(reader)

Finally, we can close the connection to the server and wait for the socket to be disconnected.

...
# close the connection
print('Shutting down...')
writer.close()
await writer.wait_closed()

Tying this together, the main() coroutine implements the client connection manager.

# echo client
async def main():
    # define the socket file
    socket_file = 'unix.socket'
    # report progress
    print(f'Connecting to {socket_file}...')
    # open a connection to the server
    reader, writer = await asyncio.open_unix_connection(socket_file)
    # report message
    print('Connected')
    # send the message
    await send_message(writer, 'Hello There\n')
    # read response
    await read_response(reader)
    # close the connection
    print('Shutting down...')
    writer.close()
    await writer.wait_closed()

We can then start the asyncio event loop and run the main() coroutine.

...
# run the event loop
asyncio.run(main())

Tying this together, the complete example is listed below.

# SuperFastPython.com
# example of an echo client using unix sockets
import asyncio

# send message to server
async def send_message(writer, message):
    # report progress
    print(f'Sending message: {message.strip()}')
    # encode message to bytes
    msg_bytes = message.encode()
    # send the message
    writer.write(msg_bytes)
    # wait for buffer to empty
    await writer.drain()

# read response from the server
async def read_response(reader):
    # report progress
    print('Waiting for response...')
    # read the response
    result_bytes = await reader.readline()
    # decode response
    response = result_bytes.decode()
    # report the response
    print(f'Received response: {response.strip()}')

# echo client
async def main():
    # define the socket file
    socket_file = 'unix.socket'
    # report progress
    print(f'Connecting to {socket_file}...')
    # open a connection to the server
    reader, writer = await asyncio.open_unix_connection(socket_file)
    # report message
    print('Connected')
    # send the message
    await send_message(writer, 'Hello There\n')
    # read response
    await read_response(reader)
    # close the connection
    print('Shutting down...')
    writer.close()
    await writer.wait_closed()

# run the event loop
asyncio.run(main())

Next, let's look at an exchange between the client and the server.

Example of Client-Server Exchange

We can explore an exchange between the echo client and server.

Firstly, we can save the server to a new Python file, such as server.py.

We can then run it from the command line using the Python interpreter.

For example:

python server.py

This will start the server and begin listening for connections from clients.

We can then save the client program to a new Python file, such as client.py.

We can then run it from the command line in a new terminal window using the Python interpreter.

python client.py

The client runs and opens a connection to the server.

The server accepts the connection and runs our client callback coroutine.

The client sends a string message to the server and the server reads and reports the message.

The server then echos back the message from the client and the client reads and reports it.

Both client and server then disconnect.

The client program exits.

The server remains running and can be manually terminated by the Control-C key combination that sends the interrupt signal (SIGINT) to the process, forcing it to terminate.

Below is a sample output from the server during the exchange.

Server running...
Client connected
Got: Hello There
Echoing message...
Closing connection

Below is a sample output from the client during the exchange.

Connecting to unix.socket...
Connected
Sending message: Hello There
Waiting for response...
Received response: Hello There
Shutting down...

This highlights how we can run the Unix socket echo server and client.

Takeaways

You now know how to develop an asyncio echo client and server in Python.

If you enjoyed this tutorial, you will love my book: Python Asyncio Jump-Start. It covers everything you need to master the topic with hands-on examples and clear explanations.