Feature
A grad student in Cairo queues up code on a gaming laptop. In Reykjavík, a professor’s workstation settles into its overnight rhythm. Across an ocean, a rack-mounted GPU rig in a co-working space joins the task. These machines share no location, only a common intention: training a large language model.
This training occurs without centralized contracts or traditional data center infrastructure. Cooling towers, server aisles and cloud leases remain untouched. Coordination flows only through protocol — software handles the timing, organizes updates, gathers fragments and reshapes them into something coherent.
Instead of stacking machines inside a single climate-controlled hall, this AI training method stretches them across continents. Each participant contributes what it can, and out of these scattered fragments, a new training architecture takes form: distributed LLM training.
Table of Contents
- How Distributed LLM Training Works
- Why Distributed AI Training Matters
- Companies Shaping the Future of Distributed AI
- What Still Holds Distributed AI Back
- What Comes Next for AI Training
How Distributed LLM Training Works
Large language models (LLMs) usually train inside specialized buildings filled with identical machines, and every step depends on proximity. This newer approach, distributed training, uses distance. Machines contribute from homes, offices, labs and schools. Some connect over broadband. Others rely on Wi-Fi. Their only shared requirement is the will to participate.
DisTrO: Training AI Over the Public Internet
DisTrO, short for Distributed Training Over the Internet, is a system created by research collective Nous that allows large language models to train across machines with modest connections. Each participant works at its own pace and the software handles the coordination.
Photon: Connecting Devices Without Sharing Data
Another group, Flower AI — in partnership with data platform Vana — has a toolkit called Photon. This system helps machines exchange information without direct access to each other. With it, the team trained an AI model called Collective-1. The model learned from contributions across the world, where each device offered processing power and some contributed data.
These tools borrow from a research method called federated learning. In that method, data stays close to its origin. The model travels instead. The system learns without moving the raw information, protecting privacy and reducing bandwidth.
Pam Didner, author of the "Modern AI Marketer" series, captured the shift clearly: “The future of enterprise AI isn’t about one mega data center, it’s a federation of distributed networks trained where the data lives.”
Related Article: The Billion-Dollar Data Center Boom No One Can Ignore
Why Distributed AI Training Matters
Distributed LLM training invites a wider circle into the work. Individuals can join with consumer-grade machines, organizations can participate without major infrastructure. Research collectives can grow without waiting for hardware grants or cloud credits.
Distributed training:
- Shifts costs from centralized investment to shared intent
- Reduces dependence on high-cost facilities
- Opens access to new kinds of data
In the Collective-1 project, participants contributed personal messages, forum posts and social chats. The system allowed each user to shape what data they shared and how it would be used.
Flexibility increases as the system expands. Participants can join when they want or step away without disrupting the LLM training process. Workflows can adapt to changing conditions. This gives the architecture a sense of resilience.
Models trained via distributed training reflect the world they learned from, and artificial intelligence begins to form without relying on the usual gatekeepers. While the machines stay separate, the outcomes become shared.
Companies Shaping the Future of Distributed AI
Nous
The open-source collective Nous focuses on distributed training at scale. Its system, DisTrO, lowers the demands of communication between machines, allowing participants with slow connections or modest hardware to contribute meaningfully. To track those contributions, Nous built a layer called Psyche, which records participation across time without central control.
“The entire open-source ecosystem for models depends on a benevolent third party releasing a foundation model…" said Jeffrey Quesnelle, co-founder of Nous Research.
"We asked ourselves, what happens if we don’t get Llama 4? The open community needs a credible way to stay in the race. That means finding a way to train our own foundation models, over the internet, with the GPUs people already have.”
FlowerAI
While Nous refines infrastructure, Flower AI focuses on orchestration. Its toolkit, Photon, handles the delicate timing of model updates. The software ensures that devices, regardless of location or uptime, can collaborate smoothly.
Flower AI, in partnership with personal data platform Vana — which asserts it treats information as something users can own, control and contribute — trained a model called Collective-1. The data came from Discord, Reddit, Telegram and other personal sources. Every submission came with user intent attached.
NodeGo
On the economic side, NodeGo offers a network for exchanging compute. Users share GPU time, set terms and build collective value. The marketplace helps distribute access and rewards consistency and availability.
Hivenet
In Switzerland, Hivenet moves in a parallel direction. The company builds around energy and scale. It connects dormant machines and guides them into shared training tasks. The focus sits on long-term stability, graceful participation and minimal overhead.
What Still Holds Distributed AI Back
Despite moves in the right direction, challenges still remain with distributed training. For example:
- Training times shift due to differences between fiber and Wi-Fi links
- Larger AI models demand more synchronization between machines and greater bandwidth
- Data travels among many contributors, potentially implicating security and trust
Each node adds complexity. Researchers tackle these challenges with compression, asynchronous updates and resilient protocols. Recent surveys on federated learning in IoT environments reflect these realities, reporting persistent constraints like:
- Latency issues
- Communication bottlenecks
- Device heterogeneity
- Privacy risks
As Didner explained, “Sensitive data in health, finance and industrial sectors cannot freely move. Distributed training lets organizations learn from it without centralizing raw data.”
Related Article: Open-Source vs Closed-Source AI: Which Model Should Your Enterprise Trust?
Webinar
Nov
6
Fix the Content Bottleneck: Build a Better WebOps Strategy
Content stalled? Dev overloaded? You’re not the only one. Learn how streamlined WebOps bridges the publishing gap.
Register
Webinar
Nov
20
How to Build a Solid Knowledge Foundation for AI Success
See how leading brands keep their AI honest, compliant and actually helpful.
Register
Webinar
Dec
2
From Manual to Magical: How AI Transforms CX Teams
Learn how to replace manual support processes with automation that actually delivers.
Register
Webinar
On demand
Beyond Storage: Smarter Content, Bigger Impact with DAM + AI
Discover how the DAM + AI duo makes content smarter, stronger and more accessible.
Watch Now
Webinar
On demand
Agentic AI Playbook: Real-World Customer Service Use Cases You Can Deploy Now
Boost self-service by 30% and slash call volume by 63% with agentic AI.
Watch Now
Webinar
On demand
CMS Briefing: A Live Look at What’s Next in AI-Driven Platforms
Learn how leading organizations are using AI‑driven tools to publish faster, personalize smarter and stay secure.
Watch Now
What Comes Next for AI Training
The architecture opens new directions for AI training. Models can begin at the edge, shaped by local data, then expand through global connection. A cluster of machines in Nairobi might train with a different rhythm than one in Toronto. Context enters the process early, location informs behavior and each contribution carries a trace of where it came from.
Researchers continue to develop new strategies for distributed training, helping updates grow lighter and synchronization becoming more flexible. Tools like Photon and DisTrO adapt with each iteration and, as more people join, the system improves.
This shift to distributed training methods changes who gets to build. It expands the circle, bringing new voices into AI.
