We examine the coupling behavior of a dielectric resonator (DR) and a microstrip line in the microwave regime, with a specific focus on three coupling regimes: over-coupling, critical-coupling, and under-coupling. Through our experiments, we observe that coupled mode theory (CMT) effectively explains the underlying mechanisms governing these three coupling regimes. Among them, critical coupling is of particular interest due to characteristics of the zero transmission magnitude, zero linewidth and infinite group delay. Besides, Floquet drive, a time-periodic modulation, has recently shown its significance in cavity magnonic systems as it can be a powerful tool for manipulating the coupling strength between magnons and photons. Floquet drive, a periodical drive in time dimension, offers a similar conceptual framework to Bloch's theorem. However, unlike spatial periodicity, which is limited by fabrication constraints, Floquet drive demonstrates its flexibility through periodic modulation in time. This study merges the concepts of critical coupling and Floquet drive, showcasing concurrent critical couplings from mirror-like sidebands in Floquet coherent cavity magnonics. We demonstrate cavity magnonics in different coupling regimes, confirm sidebands through direct Floquet drive in magnons, and present the over-coupled DR hybridizing with magnons in a Floquet condition. Due to the identical damping rates of the first-order magnon sidebands, the system exhibits a total of four modes, with the linewidths of these modes being identical in pairs. In the case of on-resonance between cavity and magnon, an over-coupled DR ensures linewidth variation between loss and effective gain, thereby guaranteeing the occurrence of critical coupling. Additionally, the identical linewidths in pairs due to Floquet ensure that this critical coupling happens concurrently. We propose achieving four simultaneous critical couplings by finely tuning the DR effective gain rate, suggesting a potential design for a perfect energy-feeding device with multiple bands.