In real-world applications, inconsistent behavior displayed by AI models before and after updates can have severe consequences, particularly in safety-critical systems like autonomous driving. The updated model may exhibit regression issues, leading to incorrect predictions that were previously accurate. While regression problems have been extensively studied in various contexts, their investigation within the realm of online domain adaptation remains limited. This poses a unique challenge due to the continuous updating of the old model with incoming data streams, potentially worsening regression problems in the source domain. In this work, we address the regression problem in online domain adaptation by mitigating regression rates in the source data while adapting to a streaming target data. Our proposed approach introduces a novel loss incorporated into the Crodobo framework. This ensures that the new model learns from the source domain in a supportive manner, avoiding the acquisition of irrelevant information and preserving privacy by removing data after successful domain adaptation. While the Crodobo framework relies on raw source data, which may contain sensitive information. Hence, it does not guarantee overall privacy protection. Consequently, we also address the privacy concerns by replacing direct access to raw data with synthetic data. Through a generative model, we force the use of synthetic data, transforming online domain adaptation into a source-free process. This guarantees the confidentiality of source data, minimizes storage requirements, and enhances the practicality of the system. Our methodologies undergoe extensive evaluation on datasets such as Visda-C, COVID-DA, and MNIST - USPS. The experimental results validate the effectiveness of our approaches.