An Honest Assessment of a Federated Learning Pilot

A client had a genuine reason to need federated learning rather than reaching for it as a buzzword — their data lived across multiple separate entities, each with real contractual and regulatory restrictions preventing the raw data from leaving its own environment. Federated learning, training a shared model without centralizing the underlying data, was the right-shaped tool for the constraint. It still wasn't close to the plug-and-play experience the pitch decks imply.

What worked

The core mechanism held up — a central model aggregated weight updates from each entity's local training without ever seeing their raw data, satisfying the actual legal constraint that made federated learning necessary in the first place. The resulting model's accuracy was competitive with what we estimated a centralized model trained on pooled data would have achieved, which was the real test of whether the approach was viable at all.

Federated learning can solve a genuine data-sovereignty constraint. It cannot make the coordination overhead of training across separate organizations disappear.

What was harder than expected

Coordinating training rounds across entities with different infrastructure, different availability windows, and different local data quality was a real and ongoing operational cost, not a one-time setup task. One participating entity's local data distribution was meaningfully different from the others, and without careful aggregation weighting, its updates periodically pulled the shared model in a direction that hurt performance for the rest. Debugging an issue in a federated setup is also inherently harder, because you can't just look at the data causing a problem — you're one step removed from it by design.

• Coordination overhead across participating entities is an ongoing operational cost, not a setup-phase cost.
• Non-IID data distributions across entities need deliberate aggregation weighting, or one entity's data can quietly degrade the shared model.
• Debugging is structurally harder since you can't directly inspect the data behind an anomalous update.
• Communication overhead between rounds needs real infrastructure planning, especially with unreliable connectivity on some participants' end.

I'd recommend federated learning again for the same kind of constraint — a genuine inability to centralize data, not a vague privacy-adjacent preference. But I'd budget the operational coordination cost as a real, ongoing line item from the start, not as a footnote under 'infrastructure setup,' because that's where most of the actual project effort ended up going.