Recurrent neural nets (RNNs) are now commonly used for the surrogate modeling of process systems, leading to better control and faster real-time optimization. However, when trained with small training data sets, most experiments show that RNNs exhibit poor generalization abilities outside the range of the training data space. Nonetheless, recent advances in deep learning research have shown that certain characteristics of the loss landscape of trained models, such as the flatness around the local minimum, tend to relate to generalization ability. This paper investigates this phenomenon for the case of RNN surrogates of the well-known Three Tanks case study, which is representative of many continuous processes. We trained a total of 200 LSTMs (long short-term memory networks) differing in initialization, architecture, and training dynamics on the same data of 500 samples. The number of model parameters ranges from 238 to 11, 353. We estimated the loss curvature of each trained model usi... [more]

For the long-cycle process industry, operational cycles can be severely affected by equipment aging, catalyst deactivation, and safety limitations. As illustrated by the residue hydrotreating process, metal impurities gradually deposit on the catalyst during residue purification, leading to catalyst poisoning and eventual process shutdown. Such long-cycle processes require dynamic adjustments of operating conditions to balance immediate economics with long-term sustainability. While current practice relies on empirical tuning based on historical data, this work focuses on studying how to obtain an optimal operating trajectory to guide the monthly adjustments of operating variables. The long-cycle simulation of the residue hydrotreating process can be performed using the commercial software, PetroSIM. After adjusting the feed conditions, its embedded mechanistic model can calculate the deviation of average bed temperature from the set point and output the remaining operating time. Since... [more]

Large-scale mixed-integer programs (MIPs) typically cannot be solved by standard solvers with reasonable computational cost. MIP-heuristics decompose large-scale monolithic mixed-integer programs into polylithic programs such that they can be solved with reasonable computational cost at the price of loosing their optimality certificate. The decomposition is steered by hyperparameters that impact the solution quality and the computational cost diametrically. The proper selection of the hyperparameter values is a black-box optimization problem which is mostly solved by grid search or random search. In previous publications the authors proposed a novel hyperparameter optimization method based on Bayesian optimization and studied a use case from the PSE domain. Computational studies showed that the BO-based algorithm is superior for objective functions with few optimal solutions.This contribution generalizes the description of the MIP-Heuristic Optimization Problem (MIP-HOP) and the comput... [more]

In this study, we propose a novel trust-region funnel (TRF) optimisation framework for process systems that integrate external black-box models, such as rigorous models, within equation-oriented (EO) formulations. The framework is applied to optimise a synthetic natural gas production process combining direct air capture and catalytic CO2 conversion using dual-function material (DFM) technology, with the objective of minimising the total annualised cost. The problem is formulated in Pyomo and solved using IPOPT, treating the DFM reactor as an external black-box model. The TRF method achieves substantial improvements compared to published mixed-integer nonlinear programming and direct nonlinear programming approaches, reducing capture cost from 460 USD to 426 USD per tonne of CO2. Key design improvements include reducing the number of DFM units per train by one-third and achieving a 22% reduction in DFM capital costs. These results highlight the TRF framework's ability to overcome numer... [more]

Heuristic decomposition schemes like moving horizon schemes are a common approach to approximately solve large-scale mixed-integer programs. The authors propose Bayesian optimization as a methodological approach to systematically tune parameters of decomposition schemes for mixed-integer programs. This paper discusses detailed results of three studies of the Bayesian optimization-based approach using hoist scheduling as a case study: Firstly, two objectives of the tuning problem are examined considering sequences of incumbent solutions found by the Bayesian optimization. Secondly, the Bayesian optimization is applied to a set of test instances of the hoist scheduling problem using four types of acquisition functions; they are compared with respect to the convergence of the tuning problem solutions. Thirdly, the scaling behaviour of the Bayesian optimization is studied with respect to the dimension of the space of tuning parameters. The results of the three studies show that the solutio... [more]

Real-time optimization (RTO) is a model-based technique that drives plants to optimal operating conditions. Modifier Adaptation (MA) is a class of methods that adjusts the optimization problem using gradient information. This enables the plant to reach the optimum operating point or batch trajectory without the need of a precise model which reduces the necessary modeling efforts. However, computing the gradients of the cost function or of the plant outputs with respect to the inputs online is a challenging task. Modifier Adaptation with Quadratic Approximation (MAWQA) integrates MA with Quadratic Approximation (QA), which helps mitigate the challenges of estimating gradients from noisy measurements by utilizing historical operating data. However, the distribution of these past operating points significantly affects the effectiveness of the MAWQA strategy. To address this issue in this contribution, new methods to compute probing points which lead to fast convergence to the optimum are... [more]

Planning and scheduling are integral components of process supply chains. The presence of data correlation, particularly multivariate demand data dependency, can pose significant challenges to the decision-making process. This necessitates the consideration of dependency structures inherent in the underlying data to generate good-quality, feasible solutions to optimisation problems such as planning and scheduling. This work proposes a chance-constrained optimisation framework integrated with copulas, a non-parametric data estimation technique to forecast uncertain demand levels in accordance with specified risk thresholds in the context of a planning and scheduling problem. We focus on the integrated planning and scheduling problem following a bi-level optimisation formulation. The estimated demand forecasts are subsequently utilised within the Data-driven Optimisation of bi-level Mixed-Integer NOnlinear problems (DOMINO) framework to solve the integrated optimisation problem, and deri... [more]

Some of the most highly trusted and ubiquitous process simulators have solution methods that are incompatible with algorithms designed for equation-oriented optimization. The natively unconstrained Efficient Global Optimization (EGO) algorithm approximates a black-box simulation with kriging surrogate models to convert the simulation results into a reduced-order model more suitable for optimization. This work evaluates several established constraint-handling approaches for EGO to compare their accuracy, computational efficiency, and reliability using an example simulation of an amine post-combustion carbon capture process. While each approach returned a feasible operating point in the number of iterations provided, none of them effectively converged to a solution, exploring the search space without effectively exploiting promising regions. Using the product of expected improvement and probability of feasibility as next point selection criteria resulted in the best solution value and re... [more]

Design problems for process systems engineering applications often require multi-scale modeling integrating detailed process models. Consequently, black-box optimization and surrogate modeling have continued to play a fundamental role in mission-critical design applications. Inherent in surrogate modeling applications, particularly those constrained by “expensive” function evaluations, are the questions of how to properly balance “exploration” and “exploitation” and how to do so while harnessing parallel computing in techniques. We devise and investigate a one-step look-ahead GRASP heuristic for balancing exploration and exploitation in a parallel environment. Computational results reveal that our approach can yield equivalent or superior surrogate quality with near linear scaling in the number of parallel samples.