LAPSE:2026.0420
Published Article

LAPSE:2026.0420
From P&ID Drawings to Process Graphs: A Multimodal Language Model Approach
June 12, 2026
Abstract
Piping and instrumentation diagrams (P&IDs) encode the functional structure of process plants and are a critical yet underutilised source of engineering knowledge for digital twins and intelligent decision support. However, digitising legacy P&IDs remains challenging due to heterogeneous drawing standards and the reliance of existing methods on brittle symbol recognition and rule-based connectivity reconstruction. This work reframes P&ID digitization as the extraction of equipment tags and inference of process topology, rather than graphical reproduction. We propose a two-stage workflow based on multimodal large language models, in which visual extraction and topology reconstruction are treated as distinct reasoning stages guided by chemical engineering process knowledge. The approach is evaluated on two ANSI-standard P&ID case studies of increasing complexity. Results show that decomposing visual extraction and topology reasoning yields more accurate and structurally consistent process representations than end-to-end digitization, highlighting the potential of language-model-based, knowledge-guided workflows for scalable and semantically reliable P&ID digitization.
Piping and instrumentation diagrams (P&IDs) encode the functional structure of process plants and are a critical yet underutilised source of engineering knowledge for digital twins and intelligent decision support. However, digitising legacy P&IDs remains challenging due to heterogeneous drawing standards and the reliance of existing methods on brittle symbol recognition and rule-based connectivity reconstruction. This work reframes P&ID digitization as the extraction of equipment tags and inference of process topology, rather than graphical reproduction. We propose a two-stage workflow based on multimodal large language models, in which visual extraction and topology reconstruction are treated as distinct reasoning stages guided by chemical engineering process knowledge. The approach is evaluated on two ANSI-standard P&ID case studies of increasing complexity. Results show that decomposing visual extraction and topology reasoning yields more accurate and structurally consistent process representations than end-to-end digitization, highlighting the potential of language-model-based, knowledge-guided workflows for scalable and semantically reliable P&ID digitization.
Record ID
Keywords
Graph reconstruction, Multimodal large language models, P&ID digitisation
Subject
Suggested Citation
Zhu B, Duong S, Vyas J, Mercangöz M. From P&ID Drawings to Process Graphs: A Multimodal Language Model Approach. Systems and Control Transactions 5:1737-1745 (2026) https://doi.org/10.69997/sct.198584
Author Affiliations
Zhu B: Imperial College London, Department of Chemical Engineering, London, Greater London, United Kingdom
Duong S: Imperial College London, Department of Chemical Engineering, London, Greater London, United Kingdom
Vyas J: Imperial College London, Department of Chemical Engineering, London, Greater London, United Kingdom
Mercangöz M: Imperial College London, Department of Chemical Engineering, London, Greater London, United Kingdom
[Login] to see author email addresses.
Duong S: Imperial College London, Department of Chemical Engineering, London, Greater London, United Kingdom
Vyas J: Imperial College London, Department of Chemical Engineering, London, Greater London, United Kingdom
Mercangöz M: Imperial College London, Department of Chemical Engineering, London, Greater London, United Kingdom
[Login] to see author email addresses.
Journal Name
Systems and Control Transactions
Volume
5
First Page
1737
Last Page
1745
Year
2026
Publication Date
2026-06-12
Version Comments
Original Submission
Other Meta
PII: 1737-1745-415-SCT-5-2026, Publication Type: Journal Article
Record Map
Published Article

LAPSE:2026.0420
This Record
External Link

https://doi.org/10.69997/sct.198584
Publisher Version
Download
Meta
Record Statistics
Record Views
469
Version History
[v1] (Original Submission)
Jun 12, 2026
Verified by curator on
Jun 12, 2026
This Version Number
v1
Citations
Most Recent
This Version
URL Here
https://psecommunity.org/LAPSE:2026.0420
Record Owner
PSE Press
Links to Related Works
References Cited
- Paliwal S, Sharma M, Vig L. OSSR-PID: one-shot symbol recognition in p&id sheets using path sampling and GCN. 2021 International Joint Conference on Neural Networks (IJCNN) :1-8 (2021) https://doi.org/10.1109/ijcnn52387.2021.9534122
- Kim H, Lee W, Kim M, Moon Y, Lee T, Cho M, Mun D. Deep-learning-based recognition of symbols and texts at an industrially applicable level from images of high-density piping and instrumentation diagrams. Expert Systems with Applications 183:115337 (2021) https://doi.org/10.1016/j.eswa.2021.115337
- Yu ES, Cha JM, Lee T, Kim J, Mun D. Features recognition from piping and instrumentation diagrams in image format using a deep learning network. Energies 12:4425 (2019) https://doi.org/10.3390/en12234425
- Schulze Balhorn, L., Seijsener, N., Dao, K., Kim, M., Goldstein, D. P., Driessen, G. H., & Schweidtmann, A. M. (2025). Rule-based autocorrection of Piping and Instrumentation Diagrams (P&IDs) on graphs. arXiv e-prints, arXiv-2502.
- Jamieson L, Elyan E, Moreno-García CF. Few-shot symbol detection in engineering drawings. Applied Artificial Intelligence 38: (2024) https://doi.org/10.1080/08839514.2024.2406712
- Kang SO, Lee EB, Baek HK. A digitization and conversion tool for imaged drawings to intelligent piping and instrumentation diagrams (p&id). Energies 12:2593 (2019) https://doi.org/10.3390/en12132593
- Choi J, An D, Kang D. Object recognition scheme for digital transformation in marine science and engineering. JMSE 11:1914 (2023) https://doi.org/10.3390/jmse11101914
- Kim BC, Kim H, Moon Y, Lee G, Mun D. End-to-end digitization of image format piping and instrumentation diagrams at an industrially applicable level. Journal of Computational Design and Engineering 9:1298-1326 (2022) https://doi.org/10.1093/jcde/qwac056
- Mani S, Haddad MA, Constantini D, Douhard W, Li Q, Poirier L. Automatic digitization of engineering diagrams using deep learning and graph search. 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) :673-679 (2020) https://doi.org/10.1109/cvprw50498.2020.00096
- Stürmer JM, Graumann M, Koch T. From engineering diagrams to graphs: digitizing p&ids with transformers. 2025 IEEE 12th International Conference on Data Science and Advanced Analytics (DSAA) :1-11 (2025) https://doi.org/10.1109/dsaa65442.2025.11248012
- Gupta M, Wei C, Czerniawski T. Semi-supervised symbol detection for piping and instrumentation drawings. Automation in Construction 159:105260 (2024) https://doi.org/10.1016/j.autcon.2023.105260
- Hain A, Gölzhäuser S, Réhault N, Brox T, Demant M. Legend-informed symbol recognition in engineering diagrams with self-supervised learning. Lecture Notes in Computer Science :403-421 (2025) https://doi.org/10.1007/978-3-662-72243-5_23
- Lu C, Walker S, Struck C, Itard L, Saelens D. Poster abstract: p&id-to-graph: llm-assisted digitalization of HVAC diagrams. Proceedings of the 12th ACM International Conference on Systems for Energy-Efficient Buildings, Cities, and Transportation :296-297 (2025) https://doi.org/10.1145/3736425.3772104
(0.12 seconds)
[0.12 s]

