Generalization of a Parameter Set Selection Procedure for Nonlinear Systems
Daniel Howsmon, Wei Dai, and Juergen Hahn
Systems of ordinary differential equations (ODEs) or differential algebraic equations (DAEs) are often used to describe dynamic systems, with applications in biochemical reaction networks [1], quantitative systems pharmacology [2], pharmaceutical processes [3], fermentation processes [4], and ecological systems [5] to name a few. The accuracy of these models depends on both the model structure and parameters. Whereas the structure is often chosen to best reflect the fundamental physics, chemistry, and/or biology of the system under study, the model parameters are sourced from available literature or estimated from experimental data.
To avoid overfitting of the model to data, it is common that only a subset of the model parameters are estimated from experimental data and the remaining parameters are set to their nominal values. Parameter set selection procedures using collinearity index methods [5], genetic algorithms [6], columnpivoting methods [7], GramSchmidt orthogonalization methods [8], and clustering methods [9] have been proposed. All of these methods rely on local sensitivity analysis to describe the relationship between the model parameters and model output. Therefore, selecting uncertain parameters for estimation is dependent on their uncertain nominal values. This may result in suboptimal parameter sets, especially as the confidence in nominal parameter values decreases [10]. Global sensitivity analysis can overcome some of the challenges associated with local sensitivity analysis by simultaneously varying model parameters over the range of interest, directly incorporating the uncertainty in parameters. However, results from the Morris method [11], samplingbased methods [12], variancebased methods [13], and other global sensitivity analyses are difficult to interpret for the parameter set selection problem since they do not use the concept of sensitivity vectors.
To overcome the shortcomings of both local and global sensitivity analyses for parameter set selection, a procedure that uses local sensitivity analysis and dynamic optimization to simultaneously vary a large number of parameters was previously developed [14]. Briefly, a sensitivity cone containing all sensitivity vectors associated with a given parameter evaluated at different combinations of parameter values is constructed. The sensitivity cone is completely characterized by the nominal sensitivity vector and the angle between the cone surface and the nominal sensitivity vector. For a given parameter p_{i}_{1}, this angle ϕ_{i}_{1} can be found from the following dynamic optimization problem:

Here, x ∈ R^{n} is the state vector, u ∈ R^{m} is the input vector, p ∈ R^{ℓ} is the parameter vector, and s_{ik} is the sensitivity of state k with respect to parameter i. The ODEs are discretized using collocation on finite elements to create a nonlinear programming problem. After the ℓ optimization problems are solved for a given uncertainty level, the angle between two sensitivity cones is determined as

where is the angle between the nominal sensitivity vectors for parameters i_{1} and i_{2}. The cosine similarity scores are then used to cluster the parameters via a hierarchical clustering technique. However, clustering based upon such sensitivity cones may be too restrictive in cases of high nonlinearity or asymmetric uncertainty resulting in a very conservative estimate of the number of parameters to be estimated.
To overcome this limitation, a procedure that directly computes the minimum angle between groups of sensitivity vectors is developed. For each set of parameters {p_{i}_{1},p_{i}_{2}} this minimum angle can be directly found from the following dynamic optimization problem:

The angles computed from the solution to these optimization problems are then directly used in hierarchical clustering. This procedure is more flexible and can more accurately determine the minimum distance between groups of sensitivity vectors resulting in a more realistic number and selection of parameters for estimation. The only drawback is that the procedure is more computationally expensive as the sensitivity cone approach requires solutions to ℓ optimization problems and the presented technique requires solutions to ℓ(ℓ – 1)/2 optimization problems with 1.5x more optimization variables. However, this downside is not significant for the number of parameters that are typically found in problems under investigation. The work includes a comparison with the parameter cone approach, highlighting scenarios where the increased computational cost could be justified.
References
[1] S. Maiti, W. Dai, R. C. Alaniz, J. Hahn, and A. Jayaraman, “Mathematical Modeling of Pro and AntiInflammatory Signaling in Macrophages,” Processes, vol. 3, no. 1, pp. 1–18, Dec. 2014.
[2] O. Demin, T. Karelina, D. Svetlichniy, E. Metelkin, G. Speshilov, O. D. Jr, D. Fairman, P. H. van der Graaf, and B. M. Agoram, “Systems Pharmacology Models Can Be Used to Understand Complex PharmacokineticPharmacodynamic Behavior: An Example Using 5Lipoxygenase Inhibitors,” CPT Pharmacomet. Syst. Pharmacol., vol. 2, no. 9, p. e74, Sep. 2013.
[3] K. V. Gernaey and R. Gani, “A modelbased systems approach to pharmaceutical productprocess design and analysis,” Chem. Eng. Sci., vol. 65, no. 21, pp. 5757–5769, Nov. 2010.
[4] W. Dai, D. P. Word, and J. Hahn, “Modeling and dynamic optimization of fuelgrade ethanol fermentation using fedbatch process,” Control Eng. Pract., vol. 22, pp. 231–241, Jan. 2014.
[5] R. Brun, P. Reichert, and H. R. Künsch, “Practical identifiability analysis of large environmental simulation models,” Water Resour. Res., vol. 37, no. 4, pp. 1015–1030, Apr. 2001.
[6] Y. Chu and J. Hahn, “Integrating parameter selection with experimental design under uncertainty for nonlinear dynamic systems,” AIChE J., vol. 54, no. 9, pp. 2310–2320, Sep. 2008.
[7] M. VelezReyes and G. C. Verghese, “Subset selection in identification, and application to speed and parameter estimation for induction machines,” in , Proceedings of the 4th IEEE Conference on Control Applications, 1995, 1995, pp. 991–997.
[8] B. F. Lund and B. A. Foss, “Parameter ranking by orthogonalization—Applied to nonlinear mechanistic models,” Automatica, vol. 44, no. 1, pp. 278–281, Jan. 2008.
[9] Y. Chu and J. Hahn, “Parameter Set Selection via Clustering of Parameters into Pairwise Indistinguishable Groups of Parameters,” Ind. Eng. Chem. Res., vol. 48, no. 13, pp. 6000–6009, Jul. 2009.
[10] Y. Chu and J. Hahn, “Quantitative Optimal Experimental Design Using Global Sensitivity Analysis via QuasiLinearization,” Ind. Eng. Chem. Res., vol. 49, no. 17, pp. 7782–7794, Sep. 2010.
[11] M. D. Morris, “Factorial Sampling Plans for Preliminary Computational Experiments,” Technometrics, vol. 33, no. 2, pp. 161–174, May 1991.
[12] G. M. (Univ of V. Hornberger and R. C. Spear, “Approach to the Preliminary Analysis of Environmental Systems,” J Env. Manage, vol. 12, no. 1, pp. 7–18, 1981.
[13] R. W. Atherton, R. B. Schainker, and E. R. Ducot, “On the statistical sensitivity analysis of models for chemical kinetics,” AIChE J., vol. 21, no. 3, pp. 441–448, May 1975.
[14] W. Dai, L. Bansal, J. Hahn, and D. Word, “Parameter set selection for dynamic systems under uncertainty via dynamic optimization and hierarchical clustering,” AIChE J., vol. 60, no. 1, pp. 181–192, Jan. 2014.
See more of this Group/Topical: Computing and Systems Technology Division