The "on-road untripped rollover" is a dangerous accident, which kills thousands of vehicle occupants every year. This type of rollover accident occurs in high-speed emergency maneuvers without hitting any external objects. In fact, in this incident, the vehicle is driven through the edges or beyond its yaw and roll stability limits. Therefore, by analyzing the Lyapunov stability of accurate vehicle models, there will be a chance to prevent this type of incidents. The problem is that accurate models have complex dynamics and include nonlinear terms, which make the stability analysis difficult. On the other hand, the available theoretical approaches for nonlinear stability analysis are either not constructive or not effective. The aim of this thesis is four-fold: a) to define a new measure of dynamics called "modified Lyapunov exponents" to provide more insight into stability analysis of nonlinear systems, b) to introduce the concept of Lyapunov exponents as a constructive method for stability analysis of nonlinear vehicle models, c) to develop a proper nonlinear vehicle roll model in sense of Lyapunov stability analysis, and d) to develop a Scale Experimental Test Vehicle (SETV) with unique features as a vehicle test bed for rollover experiments. New modified Lyapunov exponents can measure the exponential convergent/divergent rate of the perturbation vector in a specific direction driven by the dynamics in the same direction. Their existence and invariant property are mathematically proven and their indications are discussed. The concept of Lyapunov exponents has been applied effectively to analyze the system and structure stability of a nonlinear two degrees of freedom (2-DOF) bicycle vehicle model and further to estimate its Lyapunov stability regions. In the absence of a proper nonlinear vehicle model for Lyapunov stability analysis, a new nonlinear 4-DOF vehicle roll model is developed that can predict the roll motion of a conventional full vehicle model, however, it has simpler dynamics. The Lyapunov stability of the model has been analyzed by Lyapunov linearization and Lyapunov exponents methods. Moreover, the accuracy of the model in predicting the roll behaviour of a real vehicle is justified by experiments on the SETV.