Collaborative LLM Inference over LEO Satellite Networks: Model Splitting and Pipeline Parallelism

In this paper, we propose a collaborative large language model (LLM) inference scheme for low Earth orbit (LEO) satellite networks. Specifically, an entire LLM is split into multiple submodels deployed on each satellite, thereby enabling collaborative LLM inference via exchanging intermediate activations between satellites. In addition, the proposed scheme takes advantage of the pipeline parallelism mechanism that overlaps submodel inference with intermediate activation transmission, which can reduce LLM inference delay. Furthermore, we formulate an LLM inference delay minimization problem, which is transformed into a shortest-path search problem. To solve it, a directed acyclic graph (DAG)-based model splitting algorithm is developed, which restructures the DAG to identify the optimal model splitting point via an <tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">$A^{\star}$</tex>-based algorithm. Extensive simulation results demonstrate that the proposed scheme can reduce inference delay by up to 42 %, as compared to the state-of-the-art benchmarks.

Fetched live from OpenAlex and de-inverted. Abstracts are not stored in this database: the inverted indexes are 8.6 GB of the frame’s 9.3 GB of text, and the host has 13 GB free.