TopolearnUnderstanding the protocol explains kernel behavior (busy/idle, interrupts, stray output) and lets you build tools that run code in kernels.
Jupyter separates the user interface from code execution. A kernel is a standalone process that speaks the Jupyter messaging protocol over ZeroMQ sockets. A connection file (JSON with IP, ports, transport and an HMAC key) tells clients where to connect; every message is signed with that key.
Each kernel exposes several channels:
- shell: requests and replies, such as
execute_request,complete_request(tab completion),inspect_request(Shift+Tab help) andkernel_info_request; - iopub: broadcasts to all clients:
status(busy/idle),execute_input,stream(stdout/stderr),display_data,execute_result,error; - stdin: lets the kernel ask the client for input, as
input()does; - control: high-priority messages such as shutdown, interrupt and debug requests;
- heartbeat: a ping to detect a dead kernel.
A message has a header (with msg_id and msg_type), a parent_header (linking replies and outputs to the request that caused them), metadata and content. Front ends route output to the right cell by parent_header.msg_id, which is why output from a background thread can appear under a different cell than you expect.
In the browser, the Jupyter Server bridges ZeroMQ to WebSockets. Any language can provide a kernel by implementing the protocol, directly or with libraries such as xeus.
jupyter_client lets Python start and drive kernels: KernelManager starts the process and BlockingKernelClient sends requests and reads messages. Messages are signed but, over the default TCP transport, not encrypted; recent ipykernel versions warn about this, so keep kernels on localhost or use IPC transport.
Example
from jupyter_client.manager import start_new_kernel
km, kc = start_new_kernel(kernel_name="python3")
try:
msg_id = kc.execute("import os\nprint('pid', os.getpid() > 0)\n6 * 7")
while True:
msg = kc.get_iopub_msg(timeout=10)
if msg["parent_header"].get("msg_id") != msg_id:
continue # output from another request
kind, content = msg["msg_type"], msg["content"]
if kind == "stream":
print("stream:", content["text"].strip()) # stream: pid True
elif kind == "execute_result":
print("result:", content["data"]["text/plain"]) # result: 42
elif kind == "status" and content["execution_state"] == "idle":
break # request finished
reply = kc.get_shell_msg(timeout=10)
print(reply["msg_type"], reply["content"]["status"]) # execute_reply ok
finally:
kc.stop_channels()
km.shutdown_kernel()Remember
- Kernels speak a JSON messaging protocol over ZeroMQ channels: shell, iopub, stdin, control, heartbeat
- Outputs are linked to requests through
parent_header.msg_id jupyter_clientstarts kernels and sends requests from Python
Watch out for
- Assuming outputs arrive on the shell channel; results and prints are broadcast on iopub.