Abstract: This paper elucidates the basic ideas of optimal control theory, the optimal control problem and commonly used solution methods, and introduces several engineering applications of the optimal control method.
Keywords: optimal control; power system; medical equipment
introduction
Optimal control theory [1] is a discipline that studies and solves the problem of finding the optimal solution from all possible control schemes. It is a core part of modern control technology and a research hotspot and central topic of modern theory. It focuses on studying the basic conditions and comprehensive methods for optimizing the performance index of the control system. The problem studied by optimal control theory can be summarized as: for a controlled dynamic system or motion process, find the optimal control scheme from a class of allowed control schemes so that the system's motion is transferred from a certain initial state to a specified target state while its performance index is optimal. Such problems are widely found in the technical field or social problems. For example, determining an optimal control method to minimize fuel consumption during the transition of a spacecraft from one orbit to another, selecting a temperature regulation law and corresponding raw material ratio to maximize the output of a chemical reaction process, and formulating the most reasonable population policy to optimize the aging index, dependency index, and labor index during population development are all typical optimal control problems. Optimal control theory began to take shape and develop in the mid-1950s under the impetus of space technology. The maximum principle proposed by Soviet scholar L.S. Pontryagin in 1958 and dynamic programming proposed by American scholar R. Bellman in 1956 played important roles in the formation and development of optimal control theory. The optimal control problem of linear systems under quadratic performance indices was proposed and solved by R.E. Kalman in the early 1960s.
To solve the optimal control problem, it is necessary to establish the equations of motion describing the controlled motion process, specify the allowable range of values for the control variables, define the initial and target states of the motion process, and stipulate a performance index to evaluate the quality of the motion process. Typically, the quality of the performance index depends on the chosen control function and the corresponding motion state. The system's motion state is constrained by the equations of motion, while the control function can only be selected within the allowable range. Therefore, mathematically, determining the optimal control problem can be expressed as finding the extrema (maximum or minimum) of the performance index function (called a functional) with the control function and motion state as variables, under the constraints of the equations of motion and the allowable control range. The main methods for solving the optimal control problem include the classical variational method, the maximum principle, and dynamic programming.
1. Optimal control problem
1.1 The Essence of the Optimal Control Problem
Determine an optimal control u * (t) that transitions the system from the initial state X( t0 ) to the final state X( tf ) while maximizing (minimizing) the performance index J[u]. This optimal control action is denoted as u * (t). Substituting u * (t) into the optimal control action yields the optimal state trajectory X * (t). J represents the optimal performance index.
1.2 Performance Indicators of Optimal Control Problems
1.3 Mathematical Modeling of the Optimal Control Problem
This can be described using the following four equations:
2. Commonly Used Solution Methods in Optimal Control Theory
(1) Indirect method (also known as analytical method)
For optimization problems with simple and clear mathematical analytical expressions for objective functions and constraints, indirect methods (analytical methods) can usually be used to solve them.
The solution method is to first find the analytical solution by mathematical analysis (derivative method or variational method) according to the necessary conditions for the function's extremum, and then indirectly determine the optimal solution according to the sufficient conditions or the actual physical meaning of the problem.
(2) Direct method (numerical solution)
For optimization problems with complex objective functions, no explicit mathematical expression, or that cannot be solved analytically, direct methods (numerical solutions) are usually used.
The basic idea of the direct method is to use a direct search method to generate a sequence of points (or simply a list of points) through a series of iterations, gradually approaching the optimal point. The direct method is often derived from experience or experimentation.
(3) Numerical methods based on analytical methods. Methods that combine analytical and numerical calculations.
(4) Network Optimization Method. This method uses a network graph as a mathematical model and employs graph theory to search for optimal solutions.
3. Engineering Applications
For over 30 years, optimal control theory has not only seen numerous successful applications but has also transcended the traditional boundaries of natural control. It has found wide application in many fields, including systems engineering, economic management and decision-making, and especially space technology, yielding remarkable results. For these reasons, research on optimal control has become increasingly in-depth, and it has now become a very active discipline in academia. It is at the forefront of the interdisciplinary development of mathematics, engineering, and computer science.
With the continuous development of industrial automation, optimal control theory has been applied to many engineering fields such as power system excitation control, production and inventory management, water resource utilization and water pollution prevention input control, medical equipment control, and automobile handling inverse dynamics.
The following section will focus on two specific areas: power system excitation control and medical equipment control.
3.1 Application of optimal control in power system excitation control
In recent years, with the continuous development of modern control theory and its practical applications, research on the optimal control of power system operation performance using modern control theory has developed rapidly, and great progress has also been made in how to design multi-parameter excitation regulators using optimization methods.
3.1.1 Integrated Excitation Regulator Based on Nonlinear Optimization and PID Technology
For synchronous generators in nonlinear systems, errors will occur if a power system stabilizer based on PID technology is still used when the generator deviates from the system operating point or when the system experiences significant disturbances. Therefore, an excitation regulator based on nonlinear optimal control technology can be used. However, nonlinear optimal control regulators have a weakness in voltage control. Therefore, a novel excitation regulator design principle is adopted that organically combines nonlinear optimal excitation regulators and PID technology power system stabilizers.
This integrated excitation regulator utilizes research findings from nonlinear optimal control theory. It employs a precise linearization mathematical method in nonlinear excitation control, eliminating rounding errors after linearization of the equilibrium point. Therefore, the mathematical model of this control is theoretically accurate for all operating points of the generator. Furthermore, to address the relatively weak voltage regulation capability of nonlinear excitation control, a PID controller is added, giving it strong voltage regulation characteristics. This device has achieved excellent experimental results in small-unit tests, demonstrating good regulation characteristics both when operating near the equilibrium point and when significantly deviating from it.
3.1.2 Adaptive Optimal Excitation Controller
By combining adaptive control theory with optimal control theory, and through three steps—multivariable parameter identification, optimal feedback coefficient calculation, and control algorithm operation—adaptive optimal control of synchronous generator excitation can be achieved.
This generator adaptive optimal excitation scheme uses a multivariable real-time identifier composed of a least squares algorithm with a variable forgetting factor to make the element values in the coefficient matrices A and B of the system state equation change with the system operating conditions. After the optimal feedback coefficient is calculated, the adaptive optimal excitation control of the synchronous motor is realized.
Although the feedback coefficients are obtained using linear optimal control theory, the control action reflects the nonlinear characteristics of the power system because the element values in the coefficient matrix of the state equation change with the operating conditions of the system. In essence, it is a nonlinear control.
Digital simulation results show that the excitation control system can automatically track the system's operating status, identify constantly changing system parameters online, and ensure that the control action is always in an optimal state. This improves the dynamic quality of the control system and enhances the stability of the power system operation.
3.1.3 Nonlinear Excitation Control Based on Neural Network Inverse System Method
The neural network inverse system method combines the learning ability of neural networks to approximate nonlinear functions with the linearization ability of inverse system methods to construct physically realizable neural network inverse systems. This enables large-scale linearization of the controlled system and allows the construction of pseudo-linear composite systems without the need for system parameters, thereby transforming the control problem of nonlinear systems into the control problem of linear systems.
Under conditions of large disturbances, the controller of the neural network inverse system method exhibits a very short transient time and a small overshoot, effectively improving the transient response quality of the system and enhancing the stability of the power system. This controller also demonstrates excellent robustness. Furthermore, the neural network inverse system method does not require knowledge of the mathematical model and parameters of the original system, nor does it require measurement of the state variables of the controlled system. It only requires knowledge of the invertibility of the controlled system and the order of the input-output differential equations. Moreover, its structure is simple and easy to implement in engineering.
3.1.4 Optimal Excitation Control Based on Grey Predictive Control Algorithm
Predictive control is a computer algorithm that uses multi-step prediction to increase the amount of information reflecting the future trend of a process, thus overcoming the influence of uncertainties and complex changes. Grey predictive control is a branch of predictive control; it requires the establishment of grey differential equations and can perform a comprehensive analysis of the system. Using GM(1,N) to model the power deviation, speed deviation, and voltage deviation sequences of the generator, the predicted values of each state variable are obtained after comprehensive analysis. Simultaneously, based on optimal control theory, the optimal feedback gain of the controlled excitation control system with the predicted values as state variables is calculated, thereby deriving the optimal excitation control quantity with predictive information.
The grey modeling and "advanced control" concepts in grey predictive control theory effectively compensate for the shortcomings of precise linearization and "post-control" in linear optimal control theory. Simulation results for single-machine infinite bus systems show that this excitation control has the characteristics of fast response speed and high accuracy, enabling the power system to exhibit good dynamic characteristics under both large and small disturbances.
3.2 Application of optimal control in medical device control
Medical equipment has become one of the important symbols of the modernization level of hospitals and an important material basis for providing medical services [2]. With the continuous development and expansion of hospitals, the upgrading of medical equipment has become more common. The total amount of medical equipment in hospitals at all levels and of all types has been increasing. On the one hand, it meets the growing medical and health needs, but on the other hand, a series of problems have emerged, such as whether the configuration of medical equipment is reasonable, whether the investment funds are effectively utilized, the utilization rate and integrity rate, and the economic and social benefits. It is quite necessary for hospitals to analyze the current status of medical equipment configuration and management, strengthen the management of medical equipment, especially large medical equipment, and conduct statistical analysis of its benefits. This can guide scientific investment, improve business management, and enhance comprehensive benefits.
Therefore, establishing a cost-benefit concept for medical equipment and strengthening the analysis of equipment input and output provide an important guarantee for medical institutions to make scientific investments, improve management and social and economic benefits. At present, it is very necessary to strengthen the cost management of medical equipment and improve the economic efficiency of investment by controlling the initial investment scale, rationally allocating resources and improving equipment utilization.[3]
3.2.1 Problem Analysis and Assumptions
A hospital purchased a large medical device and put it into use. As the operating time of the medical device increases, its wear and tear also increases, and its resale price will decrease as the time of use of the equipment increases. On the other hand, medical equipment always needs to be maintained daily, which costs a certain amount of maintenance fees. Maintenance can slow down the wear and tear of the equipment, thereby increasing the resale price of the equipment. How to determine the optimal maintenance fee and resale time of the medical device [4] to maximize the economic benefits of the equipment is the research content of this paper.
Based on the above analysis, the following assumptions are made:
(1) As the equipment operates, its wear and tear increases. The wear and tear of the equipment at time t can be characterized by the loss value of the resale price at time t, which is called the wear function or scrap function, denoted as m(t).
(2) The resale price of medical equipment is a function of time t, denoted as p(t). The magnitude of p(t) is closely related to the wear and tear of the equipment and the amount of maintenance costs. Let the initial resale price be p(0) = p0 .
(3) Maintaining equipment [5] can slow down the wear and tear of equipment and increase its resale value. If u(t) is the maintenance cost per unit time and g(t) is the maintenance benefit coefficient at time (the increase in resale value for every one yuan of maintenance cost), then the maintenance benefit per unit time is g(t)u(t). In addition, the maintenance cost cannot be too large (if the maintenance cost per unit time exceeds the output value per unit time, the maintenance loses its meaning), and can only be selected from the bounded function set. Let the bounded function set be W, then u(t) ∈ W.
(4) Let k be the ratio of output value per unit time to resale price. Then kp(t) represents the output value per unit time at time t, that is, the utilization rate at time t.
(5) Both the resale price p(t) and the maintenance cost per unit time u(t) are continuously differentiable functions of time t. To standardize the calculation, their discounted values are used. For the calculation of the discounted value, such as the discounted value of the resale price p(t), if its discount factor is δ (discounting the unit cost over a unit of time), then:
Let t <sub>1</sub> = 0, so that the discount factor (e - δ <sub>t</sub>) of the unit cost at time t is e <sup>-δ<sub>t</sub>. Therefore, the discount factor of the resale price p(t) of the equipment at time t is p(t)e - δ <sub>t</sub>. Similarly, the discount factor of u(t) is u(t)e - δ <sub>t</sub>, and the discount factor of the output per unit time is kp(t)e - δ <sub>t</sub>.
(6) The resale value time tf and resale price p( tf ) to be determined are both free.
3.2.2 Model Establishment
The object of study is the wear-and-maintenance system of equipment during use; the resale price reflects a comprehensive indicator of wear and maintenance, and can be selected as the state variable of the system; the uncontrollability of equipment wear during use is strong, and its slight controllability is reflected through maintenance. Furthermore, maintenance itself has strong controllability, so the maintenance cost per unit time is chosen as the control strategy. Thus, the maximum economic benefit model for medical equipment can be constructed as the state equation (resale price) of the equipment wear-and-maintenance system:
Under this condition, we search for the optimal control strategy u*(t) in the function set W that satisfies 0 ≤ u(t) ≤ U , so that the system's economic efficiency is the performance index:
The maximum is given by x(tf), where tf and x( tf ) are both free.
3.2.3 Model Analysis and Solution
Based on the above model, the Hamiltonian function for medical equipment management can be derived.
Since g(t) is a decreasing function of time t, the left side of equation (5) is also a decreasing function of time t, meaning that u*(t) should decrease from U to 0 over time. The specific expression for the optimal control strategy of medical equipment is:
In the above, regarding the economic benefits of large medical equipment in hospitals [6], the optimal control theory was adopted to establish an optimization model, and the specific expression of the optimal control strategy for medical equipment was derived. Finally, a practical example was given, providing a basis for hospitals to improve their management and social and economic benefits.
4. Summary
This paper explains the problems that optimal control theory [7] needs to solve and illustrates through examples that the application fields of optimal control theory are very wide. It also conducts specific research on its application in power system excitation control and medical equipment control. However, practice shows that there is a certain gap between theoretical derivation and practical application. Practical problems often cannot be accurately derived theoretically. Therefore, in the future, we should continue to study optimal control theory and its solution methods, summarize the new progress of optimization methods, and use optimization theory to solve various engineering problems, thereby verifying the efficiency of optimization theory.
References
[1] Huang Ping. Optimization Theory and Methods [M]. Beijing: Tsinghua University Press, 2009.
[2] Zhang Kecong, Wang Lili, He Wenjun. Research on the policy of medical equipment scrapping [J]. Medical Equipment, 2007, (3): 37-38.
[3] Wu Shouzhang. Optimal Control Theory and Applications [M]. Machinery Industry Press, 2008.
[4] Wang Qiquan, Chen Wenmin, Li Yiyong. Evaluation of the economic benefits of hospital medical equipment [J]. Medical and Health Equipment, 2003, 34, (3): 11.
[5] Lu Qiang, Wang Zhonghong, Han Yingduo. Optimal control of power transmission systems. Beijing: Science Press, 1982.
[6] Liu Hongbo, Hu Xin. Research on nonlinear excitation control law of synchronous generator based on neural network inverse system method. Electromechanical Information. 2004, 19: 45-47
[7] Xiao Zhihong, Zhou Hui, Zhu Qichen. Optimal excitation control of generator based on grey predictive control algorithm. Power System Technology. 2006, 30(5): 81-85.
