Project topics and materials
Get Complete Project Material File(s) Now! »
Surrogate model-based optimization
Among the existing families of optimization methods, a promising candidate which can po-tentially allow to perform the global optimization of mixed-variable and variable-size design space problems while also requiring a limited amount of function evaluations are the Surrogate Model-Based Design Optimization (SMBDO) algorithms [105]. These algorithms rely on using surrogate models (i.e., analytical approximations) of the considered problem objective and constraint func-tions, which are generally characterized by a negligible computational cost, in order to iteratively determine and explore the most promising locations of the design space, thus simultaneously re-fining the surrogate model and converging towards the problem optimum. A number of different SMBDO algorithms allowing the optimize purely continuous optimization problems (i.e., with objective and constraint functions depending solely on continuous design variables) exist in the literature. Depending on the considered algorithm, different surrogate modeling techniques can be considered. Popular examples are the Radial Basis Function (RBF) [56], the Support Vec-tor Machine (SVM) [ 118] and the Moving Least Square ([68]). Furthermore, depending on the considered algorithm the design space exploration criterion can also vary.
Although the SMBDO of continuous problems is a popular research domain, only a few adap-tations of this kind of methods for the optimization of problems characterized by the presence of discrete design variables exist [16], [37], [60]. Furthermore, most of the proposed methods do not offer a solution for the mixed-variable problem in its most generic formulation. Finally, the constraint handling of these SMBDO techniques is usually penalization-based and can therefore be inefficient (in terms of necessary function evaluations) in case the weights are not properly tuned. For these reasons, the main objective of this thesis consists in adapting and extending surrogate model-based design optimization algorithms in order to allow the optimization of con-strained mixed-variable and variable-size design space problems with a limited number of function evaluations, thus providing a potentially useful tool for the integration of discrete technological and architectural choices within the computationally intensive design of complex systems. More specifically, the SMBDO methods which are considered and developed in this thesis are based on the Bayesian optimization algorithm [63], characterized by the use of Gaussian process sur-rogate modeling [107]. Throughout this work, optimization problems and test-cases related to the design of launch vehicles are considered in order to better highlight the engineering related applications (and associated challenges) of the discussed optimization methods. However, it is important to note that the topics of this thesis are actually applicable to a much wider range of design problems.
Variable-size design space optimization problem
Without loss of generality, a generic variable-size design space optimization problem can be modeled as depending on three different types of design variables: continuous, discrete and dimensional.
• Continuous variables: x
Continuous variables refer to real numbers defined within a given interval. Typical examples of continuous variables which can be encountered within the framework of complex system design are structure sizing parameters, combustion pressures, propellant masses and time related design parameters.
• Discrete variables: z
Discrete variables are non-relaxable variables defined within a finite set of choices. Typical examples of discrete variables which can be encountered within the framework of complex system design are the choice of material, the choice of propulsion, architectural and tech-nology alternatives, number of structural reinforcements and number of engines. Discrete variables are typically divided into 2 categories: quantitative and qualitative. As the name suggests, quantitative variables (sometimes also referred to as ordinal) are related to mea-surable values and by consequence, a relation of order between the possible values of a given variable can be defined (i.e., it is possible to determine whether a value is larger, smaller or equal to another). Quantitative discrete variables are often associated to integer vari-ables, although it is not a necessary requirement. When dealing with qualitative variables, instead, no relation of order can be defined between the possible values of a given variable. For instance, if the considered variable characterizes the type of material to be used for a given system structure, it is not possible to determine whether a possible choice (e.g., steel, aluminum, titanium, composite material) is larger, smaller or equal to another. Qualitative variables are sometimes also referred to as categorical or nominal variables. For clarity and synthesis purposes, no distinction between quantitative and qualitative variables is made in this thesis, and both types of variables are simply referred to as discrete variables. Finally, it is worth mentioning that some of the discrete variables considered in this work may technically handled by relaxing the integrality constraint, as is for instance proposed in [15] in combination with a B&B optimization algorithm. However, in the remainder of this work it is assumed that none of the discrete variables are relaxable in order to provide a generally applicable solution for the optimization of variable-size design space problems.
• Dimensional variables: w
Similarly to the discrete variables, dimensional variables are non-relaxable variables defined within a finite set of choices. The main distinction is that, depending on their values, the number and type of continuous and discrete variables the problem functions depend on can vary. Furthermore, they can also influence the number and type of constraints a given candidate solution is subject to. These particular variables often represent the choice be-tween several possible sub-system architectures. Each architecture is usually characterized by a partially different set of design variables, and therefore, depending on the considered choice, different continuous and discrete design variables must be optimized. For illustrative purposes, a few examples of dimensional variables as well as the associated continuous and discrete design variables and constraints which may be encountered within the framework of RLV design are discussed.
Computational cost of the design problem functions
As is mentioned in Chapter 1, the main focus of this thesis is towards the design optimiza-tion of complex engineering systems, such as launch vehicles, aircraft and automotive vehicles. Within this framework, the objective and constraint functions which characterize the consid-ered optimization problem often require a large computational effort to be evaluated. In most cases, this translates into a long computational time, which considerably limits the possibility of exploring the design space at will. Typical examples of computationally intensive functions which can characterize design problems are structural stress calculations based on Finite Element Method (FEM) models [78], Computational Fluid Dynamics (CFD) analyses, fixed point itera-tions and iterative Multidisciplinary Design Analyses (MDA) [13]. For instance, the design of RLV involves several disciplines and is customarily decomposed into interacting sub-models for propulsion, aerodynamics, trajectory, mass and structure, as is schematically shown in Figure 2.1. The launch vehicle performance estimation, which results from flight performance, reliability and cost, requires coupled disciplinary analyses [14]. The different disciplines are a primary source of trade-offs due to their opposing effects on the launcher performance, and finding a feasible compromise between disciplines can therefore be computationally intensive. The approaches pro-posed in this manuscript are developed under the assumption that the entirety of the objective
and constraint functions characterizing the considered problems are computationally intensive to evaluate. Under this assumption, the driving objective of the thesis is to develop optimization methods for VSDSP and Mixed-variable problems which provide fast convergence towards global optima (or at least optima neighborhoods) in terms of necessary number of evaluations of the problem functions. For the same reasons, the comparisons between algorithms which are pre-sented in this work are performed by allocating a fixed and identical computational budget (in terms of function evaluations) to all the algorithms. In practice, the methods are compared with respect to the feasible objective function value they yield with a fixed amount of function evalua-tions rather than by assessing the number of iterations required to reach a given feasible objective function threshold (e.g., optimum neighborhood). Finally, it is assumed that the computational overhead required by some of the proposed methods (i.e., calculations required during the op-timization process aside from the evaluation of the problem function values) is negligible when compared to the computationally intensive objective and constraint functions. As is discussed in the following sections of this thesis, this last assumption is particularly important when dealing with the creation and training of surrogate models of the problem functions
Table of contents :
Remerciements / Acknowledgements
1 Introduction
1.1 Context
1.2 Surrogate model-based optimization
1.3 Thesis plan
2 Problem statement
2.1 Introduction
2.2 Problem definition
2.2.1 Variable-size design space optimization problem
2.2.2 Optimality conditions
2.2.3 Fixed-size mixed-variable design space problem
2.2.4 Computational cost of the design problem functions
2.3 Review of existing methods dealing with mixed-variable and variable-size design space problems
2.3.1 Approach 1: Global exploration algorithms
2.3.1.1 Algorithms solving fixed-size mixed-variable problems
2.3.1.2 Algorithms solving variable-size design space problems
2.3.2 Approach 2: Nested optimization algorithms
2.3.3 Approach 3: Sequential optimization algorithms
2.3.4 Approach 4: Complete exploration of the non-relaxable search space
2.4 Literature review synthesis
2.5 Conclusions
3 Mixed-variable Gaussian Processes
3.1 Introduction
3.2 Gaussian Process surrogate models
3.3 Mixed-variable Gaussian Process modeling
3.4 Gaussian Process kernels
3.4.1 Kernel operators
3.5 Discrete kernels
3.5.1 Compound Symmetry
3.5.2 Hypersphere decomposition kernel
3.5.3 Latent variable kernel
3.5.4 Coregionalization
3.5.5 Discrete kernel comparison
3.6 Considerations on mixed-variable Gaussian Processes
3.6.1 Category-wise and level-wise mixed-variable kernels
3.6.2 Surrogate model scedasticity
3.6.3 Noisy data modeling
3.6.4 Hyperparameter optimization
3.6.5 Mixed-variable design of experiments
3.7 Modeling performance comparison
3.7.1 Benchmark analysis
3.7.2 Branin function
3.7.3 Augmented Branin function
3.7.4 Goldstein function
3.7.5 Analytical benchmark N.4
3.7.6 Analytical benchmark N.5
3.7.7 Propulsion performance simulation
3.7.8 Thrust frame structural analysis
3.8 Error model
3.9 Result synthesis
3.10 Conclusions
4 Mixed-variable Bayesian design optimization
4.1 Introduction
4.2 Mixed-variable Bayesian Optimization
4.2.1 Mixed variable acquisition function
4.2.1.1 Objective function oriented infill criterion
4.2.1.2 Feasibility oriented infill criteria
4.2.1.3 Infill criterion optimization
4.3 Applications and Results
4.3.1 Benchmark analysis
4.3.2 Branin function
4.3.3 Augmented Branin function
4.3.4 Goldstein function
4.3.5 Launch vehicle propulsion performance optimization
4.3.6 Launch vehicle thrust frame design optimization
4.3.7 Result synthesis
4.4 Conclusions
5 Bayesian optimization of variable-size design space problems
5.1 Introduction
5.2 Budget allocation strategy
5.2.1 Discarding of non-optimal sub-problems
5.2.2 Computational budget allocation
5.2.3 Bayesian optimization of remaining sub-problems
5.2.4 Algorithm overview
5.3 Variable-size design space kernel
5.3.1 Sub-problem-wise decomposition kernel
5.3.2 Dimensional variable-wise decomposition
5.3.3 Variable-size design space Gaussian Process training
5.3.4 Infill criterion optimization
5.4 Applications and Results
5.4.1 Benchmark analysis
5.4.2 Variable-size design space Goldstein function
5.4.3 Multi-stage launch vehicle architecture optimization
5.4.3.1 Liquid propulsion
5.4.3.2 Solid propulsion
5.4.3.3 Variable-size design space problem formulation
5.4.3.4 Optimization results
5.4.4 Result synthesis
5.5 Conclusions
6 Conclusions and perspectives
6.1 Conclusions
6.2 Perspectives
Bibliography
