Improving cross-dataset generalization in image classification with contrastive representation learning

Abstract

Regular monitoring of marine wildlife is essential for rapid detection of changes in the marine ecosystem allowing for adaptive strategies. However, the manual analysis of large volumes of underwater images taken by cameras is highly time-consuming. Deep learning techniques have been adopted in marine wildlife for the automatic classification of underwater photos to accelerate image analysis. However, water quality varies at different locations, depths, and acquisition times during data collection. This, along with differences in other acquisition parameters, leads to datasets with idiosyncratic footprints and, therefore, limited generalization of the trained deep learning model to other sets of images different from the training set. As a result, more work is required toward improving the cross-dataset generalization of deep learning models. In our research, we started by assessing dataset biases' impact on cross-dataset generalization in the classification of beluga whale images from empty underwater image frames. We used three underwater image datasets with varying image acquisition profiles: a dataset of good water quality photos, moderately bad water quality photos, and a dataset of images with both the horizon and water in the same frame. Then, we investigated two frameworks to improve cross-dataset generalization. One attempts to unlearn dataset-specific information for explicitly handling the dataset bias problem. The other uses a contrastive loss for learning a representation by contrasting the images with beluga whales against the images with empty frames regardless of their dataset membership. We conducted an exhaustive evaluation of proposed deep learning architectures and compared performance using cross-dataset approaches with traditional architectures. The supervised contrastive approach outperforms the other architectures. To the best of our knowledge, this was the first use of contrastive settings to implicitly address the dataset bias problem.