Improving LC Method Development Using Machine Learning

Vrije Universiteit Brussel researchers led efforts to improve deep reinforcement learning (RL) for liquid chromatography (LC) method development. Their findings were published in the Journal of Chromatography A (1).

Liquid chromatography (LC) is a powerful technique that is commonly used for analyzing non-volatile, soluble samples. However, despite its separation power, method development can be costly, as this process requires dedicated staff to carry out lengthy processes, to achieve suitable separation in a reasonable amount of time to enable a high sample throughput (1). New samples require tedious searches for the right combination of the carrier solvent (mobile phase) and chemistry of the ligands bonded to the particles (stationary phase) with which a sample can be resolved into all its individual constituents. This process often involves optimizing column and particle dimensions. Further, common separation strategies are typically based on a gradual variation of the mobile-phase composition during the experiment.

One potential way to simplify method development is through the use of reinforcement learning (RL), a machine learning (ML) paradigm that focuses on decision making by autonomous agents, which are systems that can make decisions and act in response to their environment, independent of direct instruction by human users (2). This approach has been used for performing complex optimization tasks, such as controlling large-scale epidemics (3).

robot with glass laptop | Image Credit: © phonlamaiphoto - stock.adobe.com

In this study, the scientists tested RL’s potential for developing LC methods. Due to the need for large training budgets, they used an in silico approach to train several proximal policy optimization (PPO) algorithms. High-performing PPO agents were trained using sparse rewards, only being rewarded when all sample components were fully separated. The trained agents were benchmarked against a Bayesian Optimization (BO) algorithm, which involves a covariance function that determines how information provided by a new sample influences the model distribution around the sample (4). The algorithm was tested using a set of 1000 randomly composed samples, with both algorithms being tasked with finding gradient programs that fully resolved all compounds in the samples while using a minimal number of experiments.

When the number of parameters to tune was limited, PPO required one to two fewer experiments than BO, though it did not outperform BO regarding the number of solutions found. PPO and BO solved 17% and 19% of the most challenging samples, respectively. That said, PPO excelled at more complex tasks that requires higher numbers of parameters; for example, when optimizing a five-segment gradient, PPO solved 31%of the samples, while BO only solved 24%.

Further improvements could be obtained by optimizing the most relevant hyperparameters further, using either a finely spaced grid search or a BO approach. Additionally, shaping a more frequent reward properly can potentially allow for more efficient training of these automated process.

References

(1) Niezen, L. E.; Libin, P. J. K.; Cabooter, D.; Desmet, G. Reinforcement Learning for Automated Method Development in Liquid Chromatography: Insights in the Reward Scheme and Experimental Budget Selection. J. Chromatogr. A 2025, 1748, 465845. DOI: 10.1016/j.chroma.2025.465845

(2) Murel, J.; Kavlakoglu, E. What is Reinforcement Learning? IBM 2024. https://www.ibm.com/think/topics/reinforcement-learning (accessed 2025-3-16)

(3) Libin, P. J. K.; Moonens, A.; Verstraeten, T.; et al. Deep Reinforcement Learning for Large-Scale Epidemic Control. Machine Learning and Knowledge Discovery in Databases. Applied Data Science and Demo Track 2020, 12461. DOI: 10.1007/978-3-030-67670-4_10

(4) Bayesian Optimization. ScienceDirect 2019. https://www.sciencedirect.com/topics/mathematics/bayesian-optimization (accessed 2025-3-16)