327048 Markov Decision Process Based Dynamic Programming for Colloidal Self-Assembly Process Optimal Control
Markov Decision Process based Dynamic Programming for Colloidal Self-Assembly Process Optimal Control
Xun Tang1, Yuguang Yang2, Michael A. Bevan2 and Martha A. Grover1
1. School of Chemical & Biomolecular Engineering, Georgia Institute of Technology, 311 Ferst Dr. NW, Atlanta, GA. 30332-0100.
2. The Department of Chemical and Biomolecular Engineering, Johns Hopkins University, 221 Maryland Hall 3400 North Charles Street, Baltimore, MD. 21218.
Self-assembly broadly refers to the process of a disordered system converging to a well-arranged state without human intervention [1]. Using self-assembly on the nano-/micrometer scale to produce defect-free crystals for the use of metamaterials, particularly photonic crystals, has received a significant amount of interest. However, due to the formation of polycrystalline assemblies, it is challenging to achieve such regularly arranged crystals without defects from self-assembly processes.
During the past decade, various approaches have been reported on producing photonic crystals. Techniques include patterned template directed colloidal crystallization, anisotropy-driven nanorod assembly, colloidal deposition, magnetic field and electric field mediated self-assembly, laser or optoelectronic and optical tweezers directed assembly, etc. have been widely studied for defect-free crystal manufacturing [2, 3, 4].
In spite of all these different approaches, the idea of using control theories to manufacture defect-free colloidal crystals has not received much attention yet due to the difficulty of real-time sensing. Control algorithms with system state as feedback can significantly enhance the performance of a dynamic system, especially a stochastic system like colloidal self-assemblies on the micro/nano scale. Feedback control is a process of monitoring a system by taking the present or past output information as a consideration in sequential action to achieve the ultimate desired output from the system. Considering its strength in dealing with stochasticity and nonlinearity of the system being studied, feedback control has been widely used in both academic studies and real-world applications. Unfortunately, colloidal self-assembly processes with feedback control have not been studied much yet, with a few exceptions [5, 6].
In our study, we propose to apply a Markov decision process (MDP) based dynamic programming optimal control algorithm to manipulate a SiO2 colloidal self-assembly process for two-dimensional defect-free crystals [6]. The movement of the particles is manipulated by the particle-particle and particle-wall interactions, which are controlled by changing the magnitude of the voltage on the electrodes. The movement of the particles is manipulated by the dipole-dipole and dipole-field interactions, which are controlled by changing the magnitude of the voltage on the electrodes surrounding the system. A two-dimensional Langevin equation is developed to simulate the dynamics of this SiO2 colloidal system. Markov transition matrices associated with 6 discrete, constant control variables (voltages) are developed based on simulation data from the low dimensional Langevin equation. With the predictions on the system state evolution from the Markov chain model via Markov chain Monte Carlo simulation, an infinite-horizon MDP optimization problem is formulated and the optimal control policy is solved by dynamic programming with policy iteration. Our MDP-based dynamic programming control policy is able to accelerate the SiO2 colloidal self-assembly process for two-dimensional defect-free crystals.
References
[1] Whitesides, G. M.; Grzybowski, B., Self-assembly at all scales. Science 2002, 295, (5564), 2418-2421.
[2] Velev, O. D.; Gupta, S., Materials fabricated by micro- and nanoparticle assembly—the challenging path from science to engineering. Adv. Mater. 2009, 21, (19), 1897-1905.
[3] Grzelczak, M.; Vermant, J.; Furst, E. M.; Liz-Marzán, L. M., Directed self-assembly of nanoparticles. ACS Nano 2010, 4, (7), 3591-3605.
[4] Arpin, K. A.; Mihi, A.; Johnson, H. T.; Baca, A. J.; Rogers, J. A.; Lewis, J. A.; Braun, P. V., Multidimensional architectures for functional optical devices. Adv. Mater. 2010, 22, (10), 1084-1101.
[5] Juárez, J. J.; Mathai, P. P.; Liddle, J. A.; Bevan, M. A., Multiple electrokinetic actuators for feedback control of colloidal crystal size. Lab on a Chip 2012, 12, (20), 4063-4070.
[6] Juárez, J. J.; Bevan, M. A., Feedback controlled colloidal self-assembly, Adv. Funct. Mater. 2012, 22, (18), 3833-3839.
See more of this Group/Topical: Computing and Systems Technology Division