Optimization has been a basic tool in most of the areas of theoretical and applied sciences. Optimal design theory focuses on identifying an experimental design that makes the variances of a model's parameter estimates as small as possible, thereby allowing the model to make the most accurate predictions. In this thesis, we have tried to address an important problem in optimal regression design, namely the application of a clustering approach to solve optimization problems with respect to several probability distributions and to further improve the convergence of a class of algorithms using the properties of directional derivatives. When we run a multiplicative algorithm to construct an optimal design, the design turns out to be a distribution de fined on a disjoint set of clusters of points. This situation arises when many design weights converge to zero at the optimum. We replace the single distribution by conditional distributions and a marginal distribution across the clusters. Motivated by this, we transform this clustering approach to a general problem of optimization with respect to several distributions. The number of probability distributions depends on the number of parameters in the model. We focus on constructing designs for two criteria of interest such as D-optimality and Ds-optimality criteria. The D-optimality is the most important and popular design criterion in the literature. The Ds-optimality is also quite important when we are interested in a subset of parameters. This situation arises when we are more interested in some of the terms (for example, the even or odd power terms) in the model. We also constructed some Ds-optimal designs using analytic approach. We explore several models both in one and two design variables. The graphical interpretation was carried out using the plots of weights versus design points as well as plots of variance functions versus design points. We did a powerful improvement in the convergence of the algorithms by combining the clustering approach and the properties of the directional derivatives. The results are promising. This approach is instrumental in improving the convergence of the algorithm and allowing the model to obtain the optimal design saving cost and time.