LAPSE:2025.0444
Published Article
LAPSE:2025.0444
Optimization of Shell and Tube Heat Exchangers Using Reinforcement Learning
June 27, 2025
Abstract
This work presents a model for optimizing shell-and-tube heat exchanger design using Q-learning, a reinforcement learning technique. An agent is trained to interact with a simulated environment of a heat exchange model, iteratively refining design configurations to maximize a reward function. This reward function balances heat exchanger effectiveness and pressure drop, emphasizing designs that minimize pressure drop. Results showed that simpler configurations consistently achieved higher rewards, despite complex designs offering better heat transfer efficiency.
This work presents a model for optimizing shell-and-tube heat exchanger design using Q-learning, a reinforcement learning technique. An agent is trained to interact with a simulated environment of a heat exchange model, iteratively refining design configurations to maximize a reward function. This reward function balances heat exchanger effectiveness and pressure drop, emphasizing designs that minimize pressure drop. Results showed that simpler configurations consistently achieved higher rewards, despite complex designs offering better heat transfer efficiency.
Record ID
Keywords
design optimization, heat exchanger, Machine Learning, reinforcement learning
Subject
Suggested Citation
Queiroz LP, Bruaset OR, Ribeiro AM, Nogueira IBR. Optimization of Shell and Tube Heat Exchangers Using Reinforcement Learning. Systems and Control Transactions 4:1818-1823 (2025) https://doi.org/10.69997/sct.192011
Author Affiliations
Queiroz LP:
Bruaset OR:
Ribeiro AM:
Nogueira IBR:
Bruaset OR:
Ribeiro AM:
Nogueira IBR:
Journal Name
Systems and Control Transactions
Volume
4
First Page
1818
Last Page
1823
Year
2025
Publication Date
2025-07-01
Version Comments
Original Submission
Other Meta
PII: 1818-1823-1517-SCT-4-2025, Publication Type: Journal Article
Record Map
Published Article
LAPSE:2025.0444
This Record
External Link
https://doi.org/10.69997/sct.192011
Article DOI
Download
Meta
Record Statistics
Record Views
585
Version History
[v1] (Original Submission)
Jun 27, 2025
Verified by curator on
Jun 27, 2025
This Version Number
v1
Citations
Most Recent
This Version
URL Here
http://psecommunity.org/LAPSE:2025.0444
Record Owner
PSE Press
Links to Related Works
References Cited
- C. Balaji, Balaji Srinivasan, and Sateesh Gedupudi. "Chapter 7 - Heat exchangers". In: Heat Transfer Engineering: Fundamentals and Techniques. Academic Press, 2021, pp. 199-231. ISBN: 978-0-12-818503-2. https://doi.org/10.1016/B978-0-12-818503-2.00007-1
- Louis Theodore. Heat transfer applications for the practicing engineer. John Wiley & Sons, 2011, pp. 3-5. ISBN: 978-0-47-064372-3
- Antonio C. Caputo, Pacifico M. Pelagagge, and Paolo Salini. "Heat exchanger design based on economic optimisation". In: Applied Thermal Engineering 28.10 (2008), pp. 1151-1159. ISSN: 1359-4311. DOI: https://doi.org/10.1016/j.applthermaleng.2007.08.010
- Reetika Saxena and Sanjay Yadav. "Designing Steps for a Heat Exchanger ". In: International Journal of Engineering Research & Technology 2 (9 Sept.2013), pp. 943-959
- Joost Iwema. "Scientific Machine Learning". In: Wageningen University & Research (Jan. 2023). URL: https://sciml.wur.nl/reviews/sciml/sciml.html
- Yifei Wang, Yifan Ma, and Haojie Chao. "Machine learning and computational fluid dynamics-based optimization of finned heat pipe radiator performance". In: Journal of Building Engineering 78 (2023), p. 107612. ISSN: 2352-7102. DOI: https://doi.org/10.1016/j.jobe.2023.107612
- Nevin Celik et al. "Performance optimization of a heat exchanger with coiled-wire turbulator insert by using various machine learning methods". In: International Journal of Thermal Sciences 192 (2023). DOI: https://doi.org/10.1016/j.ijthermalsci.2023.108439
- Hadi Keramati, Feridun Hamdullahpur, and Mojtaba Barzegari. "Deep reinforcement learning for heat exchanger shape optimization". In: International Journal of Heat and Mass Transfer 194 (2022), p. 123112. ISSN: 0017-9310. DOI: https://doi.org/10.1016/j.ijheatmasstransfer.2022.123112
- Handibag, R., Potdar, D. U., & Jadhav, A. (2020). Thermal design of tube and shell heat exchanger and verification by HTRI software. International Journal of Engineering Research & Technology (IJERT), 9(12), 525-530
- Janaun, J., Kamin, N. H., Wong, K. H., Tham, H. J., Kong, V. V., & Farajpourlar, M. (2016, June). Design and simulation of heat exchangers using Aspen HYSYS, and Aspen exchanger design and rating for paddy drying application. In IOP Conference Series: Earth and Environmental Science (Vol. 36, No. 1, p. 012056). IOP Publishing https://doi.org/10.1088/1755-1315/36/1/012056
- F.P. Incropera et al. Fundamentals of Heat and Mass Transfer. 6th ed. John Wiley & Sons, 2007, pp. 686-699. ISBN: 978-0-471-45728-2
- Mohammed Rabeeh V. and Vysakh S. "Design of Shell and Tube Heat Exchanger Using MATLAB and Finding the Steady State Time Using Energy Balance Equation". In: International Journal of Advanced Mechanical Engineering 4 (2014), pp. 95-100. ISSN: 2250-3234. URL: https://www.ripublication.com/ijame-spl/ijamev4n1spl_12.pdf
- G Papaevangelou, C Evangelides, and C Tzimopoulos. "A new explicit relation for friction coefficient f in the Darcy-Weisbach equation". In: Proceedings of the Tenth Conference on Protection and Restoration of the Environment. Vol. 166. 2010, pp. 6-09
- Keshavan Niranjan. "Elements of Fluid Flow". In: Engineering Principles for Food Process and Product Realization. Cham: Springer International Publishing, 2022, pp. 23-50. ISBN: 978-3-031-07570-4. DOI:
- Yamin Kao et al. "A dynamic approach to support outbreak management using reinforcement learning and semi-connected SEIQR models". In: BMC Public Health 194 (2024). DOI: https://doi.org/10.1186/s12889-024-18251-0
- Eugene A Feinberg and Adam Shwartz. Handbook of Markov decision processes: methods and applications. Vol. 40. Springer Science & Business Media, 2012
- Richard S Sutton and Andrew G Barto. Reinforcement learning: An introduction. 2nd ed. MIT press, 2018, pp. 47-68, 119-138. ISBN: 978-02-62039-24-6
- Christopher John Cornish Hellaby Watkins. "Learning from delayed rewards". (May 1989)
- Jesse Clifton and Eric Laber. "Q-learning: Theory and applications". In: Annual Review of Statistics and Its Application 7 (2020), pp. 279-301 https://doi.org/10.1146/annurev-statistics-031219-041220