Thesis defence: Ali Beikmohammadi

DISSERTATION
Date: Friday 16 October 2026
Time: 13:00 - 16:00
Location: L50, DSV, Borgarfjordsgatan 12, Kista

Welcome to a thesis defence at DSV! Ali Beikmohammadi explores autonomous decision-making. The PhD thesis contributes to the research subject AI and Data Science.

On 16 October 2026, Ali Beikmohammadi will present his PhD thesis at the Department of Computer and Systems Sciences (DSV), Stockholm University. The thesis defence takes place at DSV in Kista, starting at 1:00 pm.

The title of the thesis is “Toward Scalable, Robust, and Efficient Reinforcement Learning: Theoretical Foundations and Algorithms for Sample-Efficient, Communication-Aware, and Heterogeneity-Robust RL”.

Download the thesis from DiVA

PhD student: Ali Beikmohammadi, DSV
External reviewer: Joni Pajarinen, Aalto University, Finland
Main supervisor: Sindri Magnússon, DSV
Supervisor: Panagiotis Papapetrou, DSV

Contact Ali Beikmohammadi

Find your way to DSV

Abstract

Reinforcement learning (RL) holds the promise of autonomous decision-making in robotics, energy management, and personalized healthcare, yet three practical obstacles curb its impact. A solitary agent typically needs an impractically large number of interactions before useful behavior emerges; distributing the workload across many learners shortens the wall-clock time but burdens the network with costly model exchanges, and once experience is gathered on diverse devices, differences in local data and dynamics erode the guarantees enjoyed by standard federated algorithms.

This thesis offers a unified response that advances sample efficiency, communication awareness, and heterogeneity robustness in tandem. For single-agent settings, it introduces two complementary forms of lightweight guidance: an automatically annealed curriculum of auxiliary goals that accelerates exploration in sparse-or-composite-reward tasks, and a decaying surrogate loss that soft-injects imperfect heuristic controllers during early training. In distributed scenarios, a new convergence analysis proves that momentum-based optimization remains stable even when updates are aggressively compressed, leading to a federated Q-learning framework that preserves linear progress while transmitting an order-of-magnitude fewer bits.

Finally, the work establishes convergence bounds for federated optimization without tying step-size choices to explicit data similarity constants and extends them to collaborative value estimation when each learner operates in a perturbed environment, revealing precisely how mismatch and communication frequency shape residual error. Experiments across tabular and high-dimensional decision-making domains confirm that interaction demands plummet, bandwidth usage drops sharply, and learning degrades gracefully under severe non-IID conditions. Collectively, the six papers assembled here chart a coherent route toward sample-efficient, communication-constrained, and heterogeneity-robust RL, advancing the feasibility of real-world deployment on resource-limited, privacy-sensitive platforms.

Keywords

Artificial Intelligence, Machine Learning, Reinforcement Learning, Federated Learning, Federated Reinforcement Learning

Last updated: 2026-09-30

Source: Department of Computer and Systems Sciences