Project Topics
Get Complete Project Material File(s) Now! »
The sense of motion in humans and ani-mals
Humans and animals have complex multi sensory system which provides their central nervous systems (CNS) with information about the external world and about their own state. For us, like for other living creatures, it is important to estimate the spatial orientation and localization of our body with respect to the objects of the external world. Most of our sensory systems contribute to this task, including, vision, audition, touch, proprioception, olfaction, and vestibular inputs. Changes in the visual flow on the retina indicate changes in relationship between the body and the environment, unless these changes can be entirely attributed to changes in the environment. Similarly changes in the acoustic pressure impinging on the eardrums can be attributed to movement of the self as well as to changes in the acoustic environment. Tactile and pro-prioceptive inputs result from change of our body position and configuration during mechanical interaction with the environment. Independent sensory in-formation of one modality is rarely sufficient for proper motion perception. For instance, a change of visual flow can be caused by the movements of the eyes, the movement of a sound source can be confounded with head rotations, and so on. Generally speaking, this problem is solved through multi sensory integration [56, 98]. The central nervous system is highly adept at processing multisensory information to provide the brain with a proper sensation of mo-tion. Interestingly, visual, auditory, tactile, proprioceptive or olfactory sensory inputs can be easily put out of action or uninformative under a great many types of circumstances that are easy to imagine, but the vestibular inputs are always available, even in the absence of gravity! In the latter case, the vestibu-lar organs still respond to linear accelerations and angular velocities. That is why it comes naturally that vestibular system plays a key role in perceiving self motion. It is quite a curious thing that so little attention has been paid to inertial sensing in robotics when, in fact, it takes so little resources to take advantage of it compared to all other sensing modalities. In the next subsec-tions, we describe the principles that govern the function of vestibular organs and their roles for posture control.
Vestibular system
The vestibular system detects motion of the head in space, and, in turn, gen-erates reflexes that are crucial for our daily activities, such as stabilizing the visual axis (gaze) and maintaining head and body posture. In addition, the vestibular system provides us with a subjective sense of movement and orien-tation in space. The vestibular sensory organs are located in close proximity to the cochlea. The vestibular system comprises two types of organs: two otolith organs and three semicircular canals. Two otolith organs (the saccule and utricle) sense linear acceleration which includes gravitational and translational components. Semicircular canals sense angular velocities in three planes. The generated receptor signals are conveyed by the vestibular nerve fibers to the neural structures that are responsible for eye movements, posture and balance control. As a result, the vestibular organs participate to our sixth sense – the sense of motion that allows us to perceive and control bodily movements [16]. Vestibular processing is highly multimodal, for instance, visual/vestibular and proprioceptive/vestibular sensory inputs are dominant for gaze and postural control, but at the same time vestibular system itself plays an important role in our everyday activities and contributes to various range of functions [5, 45].
Otolith organs
The otolith organ comprises the utricle and the saccule. The utricle and the saccule are sensitive to linear acceleration. They respond to both head’s linear motion and static tilt in vertical plane. The utricle and the saccule are arranged to respond to motion in all three dimensions. When the head is upright, the saccule is vertical and it responds to linear accelerations in the sagittal plane, specifically up and down movements. The utricle is horizontally oriented and responds to accelerations in the interaural transverse (horizontal) plane (anterio-posterior and medio-lateral accelerations) [93]. Both, the utricle and the saccule, contain sheets of hair cells a sensory epithelium (macula). An otolithic membrane (otoconia) composed of calcium carbonate crystals sits atop of hair cells. Fig. 1.2 shows simplified view of otolithic hair cells during still (A) and accelerated motions (B). In response to linear acceleration, the crystals are deflected due to their inertia. Linear acceleration of the head causes otoconia’s accelerated motion which in turn causes shear forces acting on the hair cells. Complex molecular level mechano-electro-chemical mechanism of interaction between hair cells results in generation of electrical signals which are sent to neural structures for further processing [32].
Semicircular canals
Each inner ear has three semicircular canals approximately arrayed at right angles to each other. Semicircular canals are sensitive to angular accelerations [167]. Each canal is comprised of a circular path of fluid continuity, interrupted at the ampulla by a water tight, elastic membrane called the cupula [5]. Fig. 1.3 shows a schematic view of one semicircular canal. Each canal is filled with a fluid called endolymph. When the head rotates in the plane of a semicircular canal, inertial forces causes the endolymph in the canal to lag behind the motion of the head [32]. Motion of the endolymph causes pressure applied to the membrane of the cupula and its deflection causes the shearing stress in the hair cells [32]. Then, the corresponding electrical signals are generated and transmitted through neurons in a way similar to the way it is done in the otoliths. Although, the semicircular canals respond to angular acceleration, the neural output from the sensory cells represents the velocity of rotation. This suggests that the operation of mathematical integration of the input signal occurs owing to the mechanics of the canals, mainly the significant viscous properties of the fluid due to the small size of the canal [46, 102]. Having measurements of three angular velocities CNS can create a three-dimensional representation of the head’s angular velocity vector.
Roles of vestibular system
Humans rely on the multiplicity of sensory inputs and sophisticated anticipa-tory mechanisms to solve the control problems subserving standing, walking, running, jumping, dancing, and so on. Vestibular inputs play a central role in all these tasks, which are achieved through a combination of postural move-ments and forces and torques exerted against the environment. We briefly describe some of the roles of vestibular system, which have or may have an important potential application in robotic systems.
Vision. In humans, the head-located vestibular system is known to partic-ipate in a number of functions that include gaze stabilization through the vestibulo-ocular reflex [18, 17, 134, 51]. The vestibulo-ocular reflex stabilizes the gaze to ensure clear and stabilized vision. It is a reflex of eye-movement that stabilizes the image projected on the retina during head movements. The eye movements are produced in the direction opposite to head movements. The reflex has both rotational and translational aspects which are driven by semicircular canals and otoliths inputs, respectively.
Self-motion perception. The vestibular system is a key sensory organ for the perception of body motion [19]. Human are always aware of body move-ments even if other senses such as vision, audition are absent. The vestibular system also provides us with the ability to distinguish between self-generated motion and external ones. It has been shown that vestibular only information is sufficient for us to reconstruct our body’s location and time history of its displacements when our body is moved passively [104, 105].
Balance. The vestibular system plays a dominant role in the coordination of postural reflexes, such as vestibulo-collic reflex. Vestibulo-collic reflex is responsible for maintaining head and body posture. This reflex stabilizes the head with respect to inertial space. It produces commands that move the head in the direction opposite to the direction of the actual velocity of the head [58, 10]. Another important role of the vestibular system is the vestibulo-spinal reflex which coordinates head and neck movement with respect to the trunk of the body. The goal of the reflex is to maintain the head in an upright position [3]. Together, the vestibulo-collic and the vestibulo-spinal reflexes are responsible for self-balancing control [205, 206, 174, 2, 131].
Perception of verticality. All of us are subjected to permanent gravita-tional forces. The vestibular system is the principal sensory system which is able to estimate these forces. When our body is still, otoliths respond to the gravitational acceleration vector only, and its components provide us with a sense of absolute verticality [20, 210]. Knowledge of gravitational vertical-ity is essential for balancing and posture control as well, since it enables the disambiguition of ‘up’ and ‘down’ for spatial orientation [34].
Frame of reference. Vestibular systems, like embodied inertial sensors, pro-vides the CNS with a head-centered frame of reference. It may be suggested that low-level balance and posture control is realized in this frame of reference. Spatial transformations from head-fixed and world inertial frame can be per-formed based on vestibular system measurements. Therefore, this embodied frame of reference is directly related to the world inertial frame, and enables the neural system to perform stable posture control independently from other sensory inputs, such as tactile or proprioception information stimulated by ground inclination. In this way, ground-independent posture and balance con-trol can be implemented.
Mathematical models of vestibular system
There has been a lot of research work about the quantification of the vestibu-lar system dynamic behavior and the creation of the sensory models of spatial orientation perception. In this thesis we refer only to a selected set of research publications which describe the basic dynamic behavior of the vestibular sen-sors. For more detailed reviews on vestibular organs mathematical modeling an interested reader can further refer to various survey papers and reports, such as [133, 213, 100, 5, 85, 44]. In this subsection we first look into the dynamics of the otoliths and semicircular channels, and then, we briefly describe most frequently used models of vestibular signal processing. Our primary interest in the dynamic model of the vestibular organs is its mechanical component, which defines the relation between motion of the head’s and sensory organs response.
Dynamics of otoliths. The displacement of the large saccular otolith of the medium-sized wading birds (ruff) was measured in [49]. The results suggested that the mechanics of the otolith could be described by a critically damped second-order system with a resonant frequency of 50 Hz.
In [63, 64, 65], series of experimental studies of the otolith organs of the squirrel monkey were presented. It was concluded that the mechanical response of the otolith to tilt and translational acceleration can be modeled in terms of linear transfer function with characteristic time Tm:
where s is the Laplace variable. The input for this transfer function is accel-eration of the head which includes translation of the head and gravitational acceleration. The output of the transfer function characterizes the deflection or displacement of the sensory organ (otoconia) with respect to its neutral position. The value of characteristic time, Tm, was estimated based on the experimental measurements. It was found to be in the range of 9 to 67 ms, de-pending on the experimental conditions. Nonlinear distortion of the measure-ments was reported, as well, but it did not exceed 10-20%. The latter model suggested that the response of the otolith organs is more heavily damped than was reported in [49].
In some other reports, in addition to second order dynamics of otolith receptors, the mechanical threshold (approx. 0.005 G) and dynamics lead terms was added to represent some neural processing [132, 213].
Dynamics of semicircular canals. Significantly more research was done on the dynamics modeling for the semicircular canals. Most of research use the dynamics of the system deduced from hydrodynamic principles and suggest the torsion-pendulum model [189, 188]. In [189], the equation that defines the angular deviation of the endolymph in the canal was:
where ξ is the deviation of the endolymph, Θ is moment of inertia of the endolymph, Π is moment of friction at unit angular velocity, Δ – directional momentum at unit angle caused by the cupula. Based on experimental observa-tions and system identification procedures it was concluded that the dynamics can be described by the differential equation.
More advanced experimental studies with human subjects supported the torsion-pendulum representation of the canal dynamics [101, 91, 149]. In [188], it was suggested to model semicircular canals with heavily-damped second-order sys-tem which behaved as an angular-velocity meter.
In [73], the frequency-response analysis of central vestibular unit activity to rotational stimulation of the semicircular canals in cats was carried out. It was found that the relation between neural response of the canals and mechanical stimulation was dominated by a single time constant of about 4 seconds. Two response regions were defined, above and below a stimulus frequency of about 0.4 Hz. Above this frequency, the canals response corresponded to the angu-lar velocity of the stimulus, and below that frequency, measurements tended towards the angular acceleration of the stimulus.
More rigorous experimental study with monkeys proposed a more compli-cated dynamics model of the canals [62]. The latter model considered the difference for low and high frequencies rotation. At low frequencies the phase lag was smaller than predicted by the torsion-pendulum model, which was a consequence of sensory adaptation. There was a gain enhancement at high frequencies which was modeled by additional high-frequency component. The nonlinear distortion was reported to be relatively low, averaging about 13%.
Sensory data integration and processing. Sensory outputs from otoliths and semicircular canals are processed in neural systems at different levels. Some information is processed in low level neural networks, while some are projected to CNS where integration with other sensory modalities takes place. In this part of the chapter we give a brief review of some existing theories on how the vestibular information is processed at the neural level. Most of these models are based on system theory approaches, which are more relevant for our robotics-oriented study. Research on modeling the vestibular signal processing aimed at developing theories of human spatial orientation perception, and was applied mainly to aerospace physiological studies.
L. R. Young and his group proposed an optimal estimator model in [22]. He introduced the concept of internal model which comprised the dynamic model information about the sensory organs and head-neck system. Internal model was considered to be known to the CNS. L. R. Young used the concept of optimal estimator (Kalman filter) to model the human’s orientation estima-tion based on the outputs from visual, vestibular, proprioceptive and tactile sensory systems. Assumptions about sensor dynamics and noise statistics of the internal model were used to correct the estimated states which represented spatial orientation. Estimated states, called perceptions in [22], contained angular orientation of the head, its angular velocity, inertial translation and inertial velocity. Some nonlinear elements were added to the model in order to reproduce the delay of the onset of visually induced motion. In a recent paper by L. R. Young and colleagues [214], an overview of their optimal filter based approaches for spatial orientation estimation in humans is given.
D. M. Merfeld and colleagues developed an observer-based model for spatial orientation estimation based only on vestibular sensory inputs. Their approach was based on the concept of an internal model, as well, but special attention was given to the neural processing of gravito-inertial cues [139]. In [215, 141] experiments were performed to analyze the vestibulo-ocular response during tilt and rotation of the head and of the body. The presented experimental findings were consistent with the hypothesis that the nervous system resolves the ambiguous measurements of gravitoinertial forces into neural estimates of gravity and linear acceleration. In [140], a human model for this vestibular signals processing was presented. The graphical representation of the model is shown in Fig. 1.4. Originally, the model was developped as a result of exper-imental studies with monkeys [137, 136]. The model consists of semicircular canals and graviceptors (otoliths), their internal models, and four feedback cor-rection channels for the state estimation. A linear system was used to model the semicircular canals’ dynamics, and a simple unity gain (identity matrix) was used to model the otoliths. The state of the model includes three vectors in R3: angular velocity of the head, ω, the translational acceleration of the head, a, and the gravitational acceleration vector g. The angular velocity of the head, ω, and its translational acceleration define the trajectory of the head in the space. The angular velocity is measured by the semicircular canals which generate the output signal αscc. The gravitational acceleration vector, g, is expressed in the head-fixed frame of reference, and, therefore, contains information on the head’s angular orientation with respect to gravitational vertical. The otoliths respond to the vector sum of vectors: f = g − a, where f is defined as the gravitoinertial force. As a result, the output of the otoliths, αoto, provides the neural system with information related to the head’s static tilt and its linear translation. The actual head’s translational acceleration, a, and its angular orientation with respect to the gravitational acceleration, g, are unknown to the neural system. Merfeld et al. proposed the model in which four types of errors between the measurements and their estimates (observations). It was suggested that the neural system is able to perform the operation of gravitational vector transformation (rotation) from the world frame to the head’s frame, where gˆ is an estimate of gravitational acceleration vector expressed in head’s frame and ωˆ is an estimate of angular velocity of the head. This nonlinear integration equation is used by neural system to estimate the relative orienta-tion of gravity with the help of rotational cues. The types of error calculations and feedback channels define the inputs to the internal model. They are used to convert the feedback errors into estimates of motion and orientation. The angular velocity feedback parameters, kω, converts the difference between es-timated and actual output of semicircular canals to a neural representation of angular velocity. The translational acceleration feedback gain, ka, converts the acceleration error to a neural representation of translational acceleration. The other two feedback errors were used to feed back the cross product error and were chosen by trial and error to yield responses that matched the exper-imental data [136]. This structure makes it difficult to interpret this model in accordance with physical and mechanical principles.
More recently, J. Laurens and J. Droulez constructed a Bayesian process-ing model of self-motion perception in [121]. It was proposed that the brain processes these signals in a statically optimal fashion, reproducing the rules of Bayesian inference. It was also proposed that this Bayesian based processing uses the statistics of natural head movements. The outputs of semicircular canals and head’s angular velocity were assumed to be subjected to Gaussian noise. Using particle filtering, the three-dimensional model of vestibular signal processing was developed based on optimal estimation. The model was suc-cessfully tested by computational experiments. It was proved to be efficient in modeling the vestibulo-ocular reflex.
Among the vestibular information processing models mentioned above, the concepts of internal model and estimator or observer are crucial. The models which use the Kalman filter, linear observer and particle filtering to model vestibular system were reviewed briefly. Relatively complete reviews of existing vestibular information processing models can be found in [129, 173].
In robotic systems, the concept of observer has been known for decades, since the early works of E. Kalman in linear filtering [111] and D. Luenberger in state estimation for linear systems [128]. In the present work we develop an observer based model of vestibular information processing in which we pay specific attention to the head’s stabilization control based on the estimated spatial orientation.
Head stabilization
Various human motion experimental studies showed that humans stabilize their heads while performing different locomoting, balancing or other postural tasks. It has been proposed by T. Pozzo and A. Berthoz that humans stabilize their heads in rotation for different locomotor tasks, such as free walking, walking in place, running in place and hopping [158]. In experiments with ten healthy subjects it was shown that humans stabilized their heads and that the maxi-mum angular amplitude of Frankfort’s plane (plane of horizontal semicircular canals) did not exceed 20 degrees. This stabilization probably uses cooperation between both the measurement of head rotations by the semi-circular canals and the measure of translations by the utriculus and sacculus (otolith organs). The plane of stabilization is determined by the task; it can vary and may be controlled by the gaze. Further experiments showed that total darkness did not significantly influence the stabilization of the head, which demonstrated the importance of this behavior in the coordination of the multiple degrees of freedom of the body during gait. Fig. 1.5 shows body links orientation mea-sured during experimental studies [158]. It can be clearly seen that during locomotion the head was stabilized. Similar results were obtained in [182].
Table of contents :
1 Introduction
1.1 Scope
1.2 Summary of contributions
1.3 Thesis overview
1.4 The sense of motion in humans and animals
1.4.1 Vestibular system
1.4.2 Roles of vestibular system
1.4.3 Mathematical models of vestibular system
1.4.4 Head stabilization
1.5 Perception of self-motion in robots
1.5.1 Inertial sensors
1.5.2 Verticality estimation methods
1.5.3 Application in humanoid robots
1.5.4 Role of head stabilization
2 Model
2.1 Models of otoliths
2.1.1 Medial model
2.1.2 Lateral model
2.1.3 Head stabilization control
2.2 Model verification
3 Observability and ambiguity in otolith measurements
3.1 Nonlinear observability
3.1.1 Definitions
3.1.2 Nonlinear algebraic method for observability test
3.1.3 Observability test of the medial model
3.1.4 Observability test of the lateral model
3.2 Role of head up-right stabilization in resolving tilt-acceleration ambiguity
3.2.1 Ambiguity in otolith measurements
3.2.2 Head stabilization in up-right position
3.2.3 Non-stabilized head
3.3 Discussion
4 Verticality estimation
4.1 Observation problem
4.2 Extended Kalman filter
4.2.1 Filter design
4.2.2 Simulation results
4.3 Newton method based observer
4.3.1 Observer design
4.3.2 Simulation results
4.4 Head stabilization and the separation principle
4.4.1 Linearization and the separation principle
4.4.2 Linear observer and controller
4.4.3 Simulation results
4.5 Observation with consideration of ambiguity
4.5.1 Resolving ambiguity during up-right head stabilization
4.5.2 Simulation results
4.6 Summary of results
5 Experimental validation
5.1 Experimental setup
5.1.1 General description
5.1.2 Mechanical design
5.1.3 Inclinometer model
5.1.4 System integration and control
5.2 Experimental results
5.3 Discussion
6 Conclusion
