LMDeploy has Remote Code Execution by Pickle Deserialization via handle_zmq_recv in lmdeploy/lmdeploy/pytorch/disagg/conn/engine_conn.py

Summary

LMDeploy's PyTorch DistServe/PD-disaggregation control plane used
recv_pyobj() to deserialize messages received through a ZeroMQ PULL
socket. PyZMQ implements recv_pyobj() using Python pickle
deserialization, which can execute arbitrary code while reconstructing
an object.

The peer address used by the receiver was supplied through the
POST /distserve/p2p_connect HTTP endpoint. An attacker who could reach
an affected DistServe API server could cause the server to connect to an
attacker-controlled ZeroMQ endpoint and deserialize a crafted pickle
payload.

API-key authentication is not enabled unless the operator explicitly
configures it. As a result, affected DistServe deployments without API
keys allowed unauthenticated remote code execution with the privileges
of the LMDeploy serving process.

This issue affects the PyTorch backend when PD-disaggregation/DistServe
is enabled. Ordinary deployments that do not use the affected
disaggregated-serving path do not expose this data flow.

Affected components

• HTTP entry point:
  lmdeploy/serve/openai/endpoints/distserve.py,
  POST /distserve/p2p_connect
• Attacker-controlled peer address:
  DistServeConnectionRequest.remote_engine_endpoint_info.zmq_address
• Vulnerable receiver:
  lmdeploy/pytorch/disagg/conn/engine_conn.py,
  EngineP2PConnection.handle_zmq_recv()
• Unsafe operation: recv_pyobj(), which performs pickle deserialization

Vulnerable data flow

1. A caller submits a DistServe P2P connection request containing a
   ZeroMQ address.
2. The LMDeploy engine connects its ZeroMQ PULL socket to that address.
3. handle_zmq_recv() receives messages using recv_pyobj().
4. A malicious peer sends a crafted pickle object.
5. Python code executes during deserialization, before LMDeploy can
   perform any type or field validation.

A type check performed after recv_pyobj() cannot mitigate this issue
because pickle payload execution occurs during deserialization.

Impact

Successful exploitation allows arbitrary code execution as the
LMDeploy serving process. This can expose model weights, prompts,
credentials, attached storage, cluster-network services, and host or
GPU resources. An attacker may also modify or terminate the serving
process.

Affected versions

Affected versions:

• lmdeploy >= 0.9.2, < 0.16.0

The vulnerable P2P receiver was introduced in commit b0b705f7.

Remediation

The issue was fixed by replacing the pickle-based ZeroMQ protocol with
JSON serialization:

• send_pyobj() was replaced with send_json().
• recv_pyobj() was replaced with recv_json().
• Received objects are validated using the
  DistServeCacheFreeRequest Pydantic schema before use.
• Invalid or off-schema messages are rejected without terminating the
  receive loop.

Fix commit:

https://github.com/InternLM/lmdeploy/commit/f05b4ad8bf2e2d84101a1d63b3c44fadd99223b2

The fix was released in LMDeploy 0.16.0.

Workarounds

Users who cannot upgrade immediately should:

• Prevent untrusted clients from reaching /distserve/* endpoints.
• Restrict the DistServe HTTP and ZeroMQ control planes to trusted
  cluster networks.
• Configure API-key authentication.
• Block arbitrary outbound ZeroMQ connections from serving nodes.

These measures reduce exposure but do not make pickle deserialization
safe. Upgrading to LMDeploy 0.16.0 or later is recommended.