LAPSE:2021.0497
Published Article
LAPSE:2021.0497
Comparing Reinforcement Learning Methods for Real-Time Optimization of a Chemical Process
Titus Quah, Derek Machalek, Kody M. Powell
June 2, 2021
One popular method for optimizing systems, referred to as ANN-PSO, uses an artificial neural network (ANN) to approximate the system and an optimization method like particle swarm optimization (PSO) to select inputs. However, with reinforcement learning developments, it is important to compare ANN-PSO to newer algorithms, like Proximal Policy Optimization (PPO). To investigate ANN-PSO’s and PPO’s performance and applicability, we compare their methodologies, apply them on steady-state economic optimization of a chemical process, and compare their results to a conventional first principles modeling with nonlinear programming (FP-NLP). Our results show that ANN-PSO and PPO achieve profits nearly as high as FP-NLP, but PPO achieves slightly higher profits compared to ANN-PSO. We also find PPO has the fastest computational times, 10 and 10,000 times faster than FP-NLP and ANN-PSO, respectively. However, PPO requires more training data than ANN-PSO to converge to an optimal policy. This case study suggests PPO has better performance as it achieves higher profits and faster online computational times. ANN-PSO shows better applicability with its capability to train on historical operational data and higher training efficiency.
Keywords
artificial neural networks, Particle Swarm Optimization, process optimization, Proximal Policy Optimization, real-time optimization, reinforcement learning
Suggested Citation
Quah T, Machalek D, Powell KM. Comparing Reinforcement Learning Methods for Real-Time Optimization of a Chemical Process. (2021). LAPSE:2021.0497
Author Affiliations
Quah T: Department of Chemical Engineering, University of Utah, 50 Central Campus Dr, Salt Lake City, UT 84112, USA [ORCID]
Machalek D: Department of Chemical Engineering, University of Utah, 50 Central Campus Dr, Salt Lake City, UT 84112, USA
Powell KM: Department of Chemical Engineering, University of Utah, 50 Central Campus Dr, Salt Lake City, UT 84112, USA; Department of Mechanical Engineering, University of Utah, 1495 E 100 S, Salt Lake City, UT 84112, USA
Journal Name
Processes
Volume
8
Issue
11
Article Number
E1497
Year
2020
Publication Date
2020-11-19
Published Version
ISSN
2227-9717
Version Comments
Original Submission
Other Meta
PII: pr8111497, Publication Type: Journal Article
Record Map
Published Article

LAPSE:2021.0497
This Record
External Link

doi:10.3390/pr8111497
Publisher Version
Download
Files
[Download 1v1.pdf] (1016 kB)
Jun 2, 2021
Main Article
License
CC BY 4.0
Meta
Record Statistics
Record Views
413
Version History
[v1] (Original Submission)
Jun 2, 2021
 
Verified by curator on
Jun 2, 2021
This Version Number
v1
Citations
Most Recent
This Version
URL Here
https://psecommunity.org/LAPSE:2021.0497
 
Original Submitter
Calvin Tsay
Links to Related Works
Directly Related to This Work
Publisher Version