Project Topics
Get Complete Project Material File(s) Now! »
Local actions with global effects
One of the three core ideas of our definition of complex system is that the structures and organization arises from the interactions between the system elements. These interactions happen at the local level, because they are held between the system entities. However, as emergent phenomena arise in complex systems, we can see that the repercussions of local actions (possibly control actions) will be seen at a global level. To clearly identify the nature of the influence is complicated because of thec sensitivity to initial conditions, and because of the nonlinear dynamics of complex systems. Thus any control action should be carefully designed with this in mind. However, due to the difficulty of modeling complex systems by using models considering at the same time the local and global level of description of the system, this is not a trivial task.
The difficulty in the context of complex systems control is that if we assume that control actions shall be applied at a given level (local for example), the repercussions will be observed at another level (global for example).
Entities autonomy
In some complex systems, it is not possible to tamper with the entities autonomy. Because of legal, ethical and technical reasons, it is impossible to directly modify the behavior of entities. For instance, it is not possible to consider a control action in a target system where changes to the internal working of the entities would mean to substantially alter them. We think for example of mechanical or electronic entities. And even if we could overcome the technical reasons, we would still need to have the “blueprints” or a good model of the entity, in order to efficiently modify its inner workings.
On the other hand, we can consider that it is eventually possible to change the inner workings of “inanimate” entities in a target system.
Additionally, autonomy of participants means that as time goes by, because of immergence and emergence, and also because no central entity in the system dictates how other entities should behave, the individual behavior may change, leading to changes at all levels. This is also a problem, if the new behavior is not considered in the “model” used in the control mechanisms of the system.
The difficulty posed by the participants autonomy is that it is not always possible to consider direct modification of the inner behavior of the entities and that the autonomy may lead the system to evolve to not previously contemplated conditions.
Control of reactive multi-agent system with reinforcement learning tools
The approach is intended to control a reactive multi-agent system (Klein et al., 2005, 2008; Klein, 2009) by using reinforcement learning techniques to learn the global behavior of the target system and choosing the control actions. In it, the different reachable behaviors of the system are considered as global states. Control is defined in this approach as: making the system show a target global behavior thanks to actions taken correctly chosen and performed at the right time. The type of the considered possible control actions can be local or global, depending on the capabilities of the controller.
The approach consists in the following phases. First, characterize the global behavior and its automatic measurement. Second, choose the best control actions. Third, determine a control policy that indicates the actions to perform. The decision of which control action to apply is taken on-line (while the target system is in execution). Also, the decision is taken dynamically: a decision is dependent on the state of the target system.
Examples. The feasibility of the approach is presented within a Pedestrian Model in (Klein et al., 2008). In it, reinforcement learning techniques are used to learn the behavior of the system and select the best control actions. In the example, the target system is a simulator of a reactive multi-agent system made of pedestrians walking in a circular corridor. In the system, the agents are leaded by a sum of forces.
Architecture elements
We have two different separate systems: our architecture C and a target system T . Each system has its own parameters and dynamics.
Control objective. The objective of the architecture is to guide the target system to exhibit a desired state. It is deemed D.
Future State estimation. The estimation of the future state of the target system is obtained through multi-agent model simulation. Because of the nature of multi-agent simulation, we shall have multiple models giving as result multiple future state estimations. We retake the notation for multi-agent model simulation summarized in table 2.2.
A model is deemed M, the set of parameters of the model is deemed P arM = fp1; p2; : : : ; png, the set of initial values given to the parameters of the model is deemed M . An instance of the model is deemed M1 and the result of the simulation of an instance M1 is deemed rM1 .
Observation of the target system. This is necessary to be able to identify if the control objective is attained or not. Or, put in other words, it is a way to identify the state of the target system. Also, it is necessary for the initialization, calibration and validation of multi-agent models. The set of observations is deemed O.
Simulation of control actions. The possible effects of control actions applied to the target system are estimated with multi-agent models.
Application of control actions. We have a concrete control actions to be applied to the target system. The set of control actions is deemed A. Because the outcomes of multi-agent model simulation are multiple, we shall decide at some point which outcome to use as the basis to decide which control action to apply. It is therefore necessary to have a criteria to compare them.
Internal Block Diagram of the architecture
In this diagram we specify the interactions between the different blocks of the architecture. An association between two blocks defines an interaction. If the association is made with the use of ports, the port type specifies the type of element flowing from one block to the other. Figure 3.4 illustrates the different interactions among the architecture parts. The classifier behavior of the architecture is called the “Main Control Loop” and it interacts with all the other parts. The different items flowing from one block to another are specified in the activity diagram of figure 3.5. Now that we have defined the functions of the architecture as blocks and the different inputs and outputs of the architecture as well as the interactions between the blocks, let us consider the order in which the operations of each block take place.
Execution flow of the architecture
The classifier behavior of the architecture (the “Main Control Loop”), is where the execution flow should be implemented. This part of the architecture initiates and stops as necessary the execution of the functions of the parts involved in the execution flow. The execution flow can be summarized as follows.
The first activity executed by the architecture is to estimate the future state of the target system up to a time horizon corresponding to the control objective. Then, the estimated future state is compared to the control objective to see if any actions are required. If control actions are required, they are determined by the simulate control actions block and then applied by the apply control actions block. If no control actions are necessary, the main control loop restarts. This execution flow is illustrated in figure 3.5.
Complementary view of the elements of the control architecture
The estimate future state block is responsible for carrying out this function. Since this block requires as input the observations from the target system, the observe target system block should be instantiated in advance. The main control loop is in charge of passing the observations O as well as the time horizon t to the estimate future state block. The block simulates the evolution of T up to time horizon t. The result of the simulation r is passed back to the main control loop.
Table of contents :
Chapter 1 Complex Systems
1.1. Complex Systems Definition
1.2. Relevant Characteristics
1.3. Examples
1.4. Challenges in the study of complex systems
1.4.1. Modeling
1.4.2. Engineering
1.4.3. Control
1.5. Difficulties of control of complex systems
1.5.1. Local actions with global effects
1.5.2. Entities autonomy
1.5.3. Modeling
1.5.4. Preexisting systems
1.6. Governance and control of complex systems
1.7. Synthesis
Chapter 2 Related Work
2.1. Introduction
2.2. Control Theory
2.3. Equation Free approach
2.4. Modeling complex systems
2.4.1. Multi-agent paradigm
2.4.2. Agent-Based Models
2.5. Applications of multi-agent paradigm in the control of complex systems
2.5.1. Organic Computing
2.5.2. Control of Self-Organizing systems
2.5.3. Prosa
2.5.4. PolyAgent
2.5.5. Morphology
2.5.6. Emergent Engineering
2.5.7. Control of reactive multi-agent system with reinforcement learning tools .
2.6. Synthesis
2.7. Conclusions
Chapter 3 A control architecture
3.1. Introduction
3.2. Principles
3.3. Definition of the architecture
3.3.1. Architecture elements
3.3.2. Hypothesis of the architecture
3.3.3. Concise definition
3.3.4. Preliminary synthesis
3.4. Detailed view
3.4.1. Block Definition Diagram of the architecture
3.4.2. Internal Block Diagram of the architecture
3.4.3. Execution flow of the architecture
3.4.4. Complementary view of the elements of the control architecture
3.5. Assessment on the relevance of our proposition
3.5.1. Architecture elements and difficulties of the control of complex systems challenge
3.5.2. Equation free approach and multi-agent models in the architecture
3.5.3. Governance related aspects
3.6. Conclusion
Chapter 4 Proof of concept Implementation
4.1. Introduction
4.2. Peer-to-Peer networks as complex systems
4.3. Target system specification
4.3.1. Description
4.3.2. Target system state
4.4. Target system implementation
4.4.1. Internal behavior of the peers
4.4.2. Number of peers in the network
4.4.3. Network organization
4.4.4. Initial proportion of sharing peers
4.4.5. Update Scheme
4.4.6. Summary
4.5. Preliminary study on the behavior of the target system
4.5.1. Analytical point of view
4.5.2. Beyond the analytical model
4.5.3. Results and conclusions on the preliminary study
4.5.4. Retained Initial conditions
4.5.5. Focus on the behavior of the target system associated to retained initial conditions
4.6. Experimentation platform
4.7. Experimental Setup
4.7.1. Control Objective
4.7.2. Architecture implementation overview
4.7.3. Architecture implementation details
4.8. Experiments and Results
4.8.1. Description of experiments
4.8.2. Synthesis of results
4.8.3. Discussion on the results
4.9. Discussion
4.10. Conclusions
Chapter 5 Multi-agent simulation questions inside the architecture
5.1. Introduction
5.2. Initialization of models
5.2.1. Target systems implementation
5.2.2. Nominal Behavior
5.2.3. Experimental setup
5.2.4. Experiments and Results
5.2.5. Discussion
5.2.6. Conclusions
5.3. Model selection
5.3.1. Target system implementation
5.3.2. Experimental setup
5.3.3. Experiments and Results
5.3.4. Discussion
5.4. Conclusions
Chapter 6 Conclusion
Bibliography
