Infrastructure failure causes the loss of human lives and high socio-financial costs. Due to the continuous aging of infrastructure, a proper structural health monitoring (SHM) system is required to ensure the safety of structures and reduce repair costs through the early detection of structural damage. Existing visual inspection methods are not reliable due to the low frequency of inspection, subjective evaluation of structural damage, and vulnerability of inspectors’ safety, along with high costs. Traditional damage detection methods have similar limitations, since they require a large number of sensors to monitor large-scale infrastructure and involve high levels of uncertainty due to environmental noises and sensor malfunctions. Computer vison techniques have been implemented to overcome the limitations mentioned above, relying on image processing algorithms to extract damage-sensitive features. However, it is very difficult to extract a robust damage-sensitive feature. To resolve this limitation, I developed two deep learning-based damage detection methods using computer vision. The first method is a hybrid pixel-level crack segmentation and quantification method for complex cracks on rough scenes. The developed hybrid method provides robust damage detection for images, which addresses the uncertainties of traditional approaches. The second method is a real-time semantic transformer representation network (STRNet) for crack segmentation. The proposed STRNet can process 49 images per second with a mean intersection over union score of 92.6, which represents state-of-the-art performance in this area when it comes to accuracy. Using advanced deep learning methods and computer vision for damage detection still requires a great number of cameras to monitor large-scale infrastructure, which can be expensive. Consequently, another achievement of this thesis is that I developed an autonomous flight method using unmanned aerial vehicles (UAVs) for SHM purposes. Some critical parts of the bridge system, which should be monitored, are located beneath the bridge deck where global positioning system (GPS) signals are very weak or not available. Therefore, a three-dimensional pseudo map was developed using an inexpensive ultrasonic beacon system to replace the GPS signals for the autonomous flight of the UAVs.