You are running multiple instances of the Python interpreter, possibly on different ma‐chines, and you would like to exchange data between interpreters using messages.
It is easy to communicate between interpreters if you use the multiprocessing.connection module. Here is a simple example of writing an echo server:
from multiprocessing.connection import Listenerimport traceback
def echo_client(conn):try:while True:msg = conn.recv()conn.send(msg)except EOFError:print(‘Connection closed')def echo_server(address, authkey):
serv = Listener(address, authkey=authkey)while True:
try:> client = serv.accept()
echo_client(client)
except Exception:traceback.print_exc()
echo_server((‘', 25000), authkey=b'peekaboo')
Here is a simple example of a client connecting to the server and sending variousmessages:
>>> from multiprocessing.connection import Client
>>> c = Client(('localhost', 25000), authkey=b'peekaboo')
>>> c.send('hello')
>>> c.recv()
'hello'
>>> c.send(42)
>>> c.recv()
42
>>> c.send([1, 2, 3, 4, 5])
>>> c.recv()
[1, 2, 3, 4, 5]
>>>Unlike a low-level socket, messages are kept intact (each object sent using send() isreceived in its entirety with recv()). In addition, objects are serialized using pickle.So, any object compatible with pickle can be sent or received over the connection.
There are many packages and libraries related to implementing various forms of mes‐sage passing, such as ZeroMQ, Celery, and so forth. As an alternative, you might alsobe inclined to implement a message layer on top of low-level sockets. However, some‐times you just want a simple solution. The multiprocessing.connection library is justthat—using a few simple primitives, you can easily connect interpreters together andhave them exchange messages.If you know that the interpreters are going to be running on the same machine, you canuse alternative forms of networking, such as UNIX domain sockets or Windows namedpipes. To create a connection using a UNIX domain socket, simply change the addressto a filename such as this:
s = Listener(‘/tmp/myconn', authkey=b'peekaboo')
To create a connection using a Windows named pipe, use a filename such as this:
s = Listener(r'.pipemyconn', authkey=b'peekaboo')
As a general rule, you would not be using multiprocessing to implement public-facingservices. The authkey parameter to Client() and Listener() is there to help authen‐ticate the end points of the connection. Connection attempts with a bad key raise anexception. In addition, the module is probably best suited for long-running connections
(not a large number of short connections). For example, two interpreters might establisha connection at startup and keep the connection active for the entire duration of aproblem.Don’t use multiprocessing if you need more low-level control over aspects of the con‐nection. For example, if you needed to support timeouts, nonblocking I/O, or anythingsimilar, you’re probably better off using a different library or implementing such featureson top of sockets instead.