Financial world faces many uncertainties due to decisions by governments, business entities, technology trends, natural calamities etc. Stocks and stock market form one of the major activities in financial world. Volatility is one of the main measures of uncertainty in financial stock markets. Hence, forecasting stock volatility is a critical component in many financial problems such as financial risk management, optimizing portfolios etc. In addition to traditional statistical techniques, there have been few artificial intelligence (AI) and machine learning (ML) techniques used in the literature for this volatility forecasting problem. Transformer Network (TN) architecture is one of newest ML techniques. For this thesis work, we utilized TN architecture with multi-head attention (MHA) mechanism for stock volatility forecasting. To enhance the performance of the TN, we incorporated different variations of the feed forward layer. The performance of three distinct TN models was evaluated by implementing three different deep learning (DL) layers (CNN, LSTM, and a hybrid layer CNN LSTM) in the encoder block of TN as the feed forward layer. The results clearly demonstrate that the TN model with the hybrid layer (CNN-LSTM) outperformed the other models, including a recently proposed data-driven approach. Furthermore, we assessed the performance of another latest model that is on built on TN known as Informer model on a minute-scale Bitcoin dataset across various forecast lengths. Our findings underscore the advantages of the Informer model, specifically its ProbSparse attention mechanism and distilling operation, which substantially enhances its efficiency in handling long sequence time-series forecasting (LSTF) tasks.