In the dynamic realm of AI, large language models (LLMs) are driving groundbreaking advancements, from conversational chatbots to agentic workflows. We stand at a pivotal moment, grounded by two axioms that increasingly define the trajectory of AI systems:

Humans will always crave more intelligence.
More tokens equal more intelligence.

Those axioms echo the heartbeat of our species. From the first tools we carved to the neural networks we train today, our hunger for knowledge and capability knows no bounds. In the age of Agentic AI, this craving manifests as an insatiable demand for tokens—the fuel of digital intelligence.

Yet, the systems we’ve relied on are crumbling under this pressure: Chip manufacturers can’t keep up with the demand, but packing more transistors into silicon takes astronomical capital; Data centers guzzle power at a rate our grids can’t sustain, and our energy supply won’t renew at a proportionate pace. Enterprise-grade GPUs are required to host the most advanced models, yet achieving personal intelligence sovereignty is nearly impossible due to the high cost. The status quo isn’t just lagging, it’s a bottleneck strangling the future of AI.

That’s why we’re building Parallax—the world’s first fully distributed inference engine to ignite a revolution. We believe decentralized serving is the key to lifting the existing ceiling in AI development, particularly in terms of scalability, sovereignty, privacy, and cost-effectiveness.

Parallax reimagines model inference as a global, collaborative process—one where models are no longer chained to centralized infrastructure, but are instead composed, executed, and verified across a distributed machine mesh.

Why should I care?

Parallax isn’t just a framework—it’s a redefinition of who gets to access and participate in intelligence.

It enables three foundational paradigm shifts:

Intelligence Sovereignty and Democratization for All: Today, access to advanced intelligence is gated by centralized APIs, corporate clouds, and expensive compute. Parallax breaks this monopoly. It empowers individuals to run state-of-the-art models on their own terms—at home, on their own machines, with no centralized dependency. True equality begins not with money, but with access to intelligence. The right to think, to know, to act—with tools as powerful as the world itself.
Composable, Collaborative Inference: Parallax enables large models to be decomposed and executed across many devices, not just within a single household, but among a global mesh of participants. Consumer GPUs, Apple Silicon, and desktops can be stitched together into a composable inference pipeline, turning fragmented hardware into a coherent execution engine.
Unlocking the World’s Latent Compute: The world is filled with untapped compute: gaming GPUs, laptops, workstations. Parallax brings them into the network. By making model execution modular and verifiable, we unlock the ability to serve large models not just with H100s—but with a global coalition of latent devices. This dramatically expands the viable GPU supply and slashes the cost of inference, further supporting the agentic world.

These shifts don’t just lower costs—they rewrite who gets to access, control, and contribute to the future of intelligence.

How Parallax Works

A production-level decentralized serving system aims to balance compute and communication across heterogeneous devices. Despite its importance, decentralized systems are inherently more complex to set up and manage compared to centralized frameworks. It requires coordination across fragmented environments and deep expertise in both machine learning (ML) and distributed systems. Currently, there is a lack of standardized tools and frameworks for developer adoption.

Parallax is grounded in our belief that a purely decentralized serving framework should be:

Device-agnostic: Hosts should not just be enterprise GPUs, but also Apple Silicons.
P2P connected: Users should be able to participate permissionlessly, despite their network environments and system settings.
Self-discoverable: The system should rebalance to ensure optimal workload partitions.
Performance-optimized.

Parallax is a system specifically engineered for high-performance structured generation on decentralized machine networks. The system consists of three main layers: Runtime, Communication, and Worker.

Runtime Layer

The Parallax Runtime Layer is the core orchestration engine for high-throughput, server-side LLM serving on distributed, heterogeneous networks. To handle the primary challenge of massive concurrency, it is architecturally composed of an Executor (control loop), Model Shard Holder, Request Manager, Scheduler, and Paged KV Cache Manager.

This integrated design enables critical server-grade optimizations:

Continuous Batching: Our Scheduler dynamically groups incoming requests to maximize hardware utilization and overall throughput.
Paged KV-Cache Management: The Cache Manager uses a block-based system, inspired by virtual memory, to prevent memory fragmentation, handle thousands of concurrent requests, and enable high-performance Paged Attention kernels.

As the first framework of its kind for the MLX ecosystem, the Parallax runtime pioneers professional-grade serving on Apple Silicon. Its unique architecture seamlessly integrates both NVIDIA GPUs and Apple devices into a single, cohesive serving fabric, representing a major step forward in building powerful and accessible decentralized AI systems.

Communication Layer

Manages gRPC and tensor streaming between peers, ensuring forward and backward passes succeed even if some nodes fail. Built on top of Hivemind’s Distributed Hash Table (DHT), a decentralized system that ensures fast, reliable information sharing without a central coordinator, to distribute values across peers. We aim to provide future support for NAT traversal, custom UDP, and dynamic routing for network robustness.

Worker Layer

Executes inference tasks across heterogeneous hardware platforms, ensuring optimal performance and scalability via a dual-platform approach:

GPU Workers: Use a modified version of SGLang (a fast serving framework for LLMs, which utilizes PyTorch with CUDA kernels to harness the full computational power of NVIDIA GPUs) with asynchronous batching for heterogeneous compute adaptations.
Apple Workers: We have engineered a state-of-the-art serving engine built upon our pioneering MLX-compatible runtime, with the integration of a highly optimized Metals kernel, including a Paged Flash Attention kernel, unlocking unprecedented inference efficiency and throughput on Apple Silicon.

The Swarm Architecture

Our decentralized inference engine is built atop a distributed architecture we call the Swarm: a dynamic network of nodes that collaboratively serve a large language model (LLM). Each node is responsible for specific segments of the model, executing a defined portion of the inference process.

When a user submits a request, the client software tokenizes the input into numerical IDs, generates attention masks, and, when needed, encodes routing metadata to optimize execution across the network. The client then identifies the most suitable nodes in the Swarm, selected based on availability, compute capacity, and network latency, that collectively span the full model.

Inference proceeds sequentially: the first node processes its designated layers and passes the resulting hidden states to the next, continuing layer by layer until the model completes its forward pass. All nodes communicate peer-to-peer via a decentralized hash table (DHT), eliminating single points of failure and enabling self-healing. The Swarm also supports dynamic node joining and automatic rebalancing, ensuring that model weights are redistributed efficiently as the network evolves.

Benchmark

We have evaluated Parallax’s performance and compared it with a baseline distributed inference system, like Petals. The evaluation focuses on latency, throughput, and scalability using real-world workloads. The evaluation is conducted on a distributed setup consisting of two nodes, each equipped with Nvidia RTX 5090 GPUs connected via networking.

Performance comparison between Parallax and baseline Petals framework on 1x4K input configuration. All experiments use Qwen2.5-72B-Instruct-GPTQ-Int4 model with 1,024 output tokens.

Performance comparison for Parallax across different input configurations (Single request: 1x4K, 1x8K, 1x16K, and Multi-request: 4x1k, 8x1K). All experiments use Qwen2.5-72B-Instruct-GPTQ-Int4 model with 1,024 output tokens.

We achieved 3.1x lower on end-to-end latency, 5.3x lower on Inter-token latency, and improved Time-to-first token, Input Throughput, and Output throughput by 2.9x, 3.1x, and 3.1x, respectively. Performance was stable as input length increased from 4K to 16K tokens with higher output throughput, while inter-token latency stayed within a narrow range across all configurations. The total output token throughput increases significantly as we increase batch size, indicating great scalability potential as we progress forward.

Experience Decentralized Intelligence, Real-Time

You can now try our decentralized AI Chatbot, powered by Parallax.

https://youtu.be/6FZ0LaFjA_E

Designed with decentralization at its core, Parallax offers a new paradigm for decentralized AI. While it functions like any other AI chatbot on the surface, each response is served from a real-time inference running on a swarm of personal machines, and not from a centralized server.

We’re currently in the closed beta phase of our Edge Host program, so you may encounter some traffic as host availability is limited. As Parallax continues to evolve, our goal is a fully decentralized serving network—an always-on intelligence owned and operated by the community.

If you’re interested in contributing your compute, apply to become an Edge Host

The Road Forward

Parallax is built for a world where AI is ambient, interactive, and user-owned.

This distributed model serving engine is more than a technical breakthrough; It’s a meaningful step toward democratizing intelligence. Frameworks like vLLM and SGLang have paved the way for optimized inference at scale, and their open-source ethos has been vital to the field’s progress. Inspired by their work, we plan to open-source Parallax once it’s production-ready, and we can’t wait to see how the community builds on it to shape the future of AI.

With Lattica powering data communication and Parallax enabling decentralized inference, the pieces are coming together at Gradient Network to form the foundation of a truly open, decentralized AI stack.

Stay tuned, for the decentralized future knows no bounds.

Read the full research paper here.