Condition monitoring of electrical insulation of high voltage apparatus is of much importance for the reliable operation of electric power systems. An effective way to monitor the health of such systems is the measurement of partial discharges (PDs) in the insulation material. In order to prevent the PD from progressing in the insulation material, the source of PD should be known. The PD classification problem in high voltage systems changes from a single-class to a multi-class and multi-label classification problem when PD sources take place simultaneously in real-life systems. Regardless of being single-label or multi-label, supervised learning has been employed for PD source classification by academics and researchers where labelled data and the number of PD sources are two prerequisites for the training process. What makes PD classification more complicated in real life scenarios is that the number of PD sources in an insulation system is unknown. This makes the problem of PD classification an unsupervised one, where there is no prior knowledge of the number of PD sources. Machine learning techniques offer a solution for PD classification by allowing to train models based on extracted features. The performance of such algorithms heavily depends on the choice of features. This can be overcome by using deep learning where feature extraction is done automatically by the algorithm, and the input to such an algorithm is the raw input data. This research is focused on developing deep learning models for the classification of PD sources in the insulation of high voltage systems. The developed models include: model for classifying multi-source PDs and single-source PDs without introducing multi-source PDs in the training stage, interpretable attention based model that propose non linear filters that are capable of differentiating between PD signals that look alike, and an unsupervised deep learning model for predicting the number of partial discharge sources. These models present the base for smarter next generation PD monitoring software that can be used by researchers and experts in industry in order to overcome the challenges and limitations that are present in current practices.