Rock materials in an orogen are usually large-scale denuded and transferred to their surrounding basins during the post-collisional stage. Therefore, studies on sedimentary rocks in these basins are helpful for better understanding the evolutionary history of the orogen. The Hefei Basin, located north of the Dabie Orogen, was filled with clastic materials from the orogen, recording key information on its uplift and denudation history. In this study, sandstones of the Lower Cretaceous Zhuxiang Formation and siltstones of the Paleogene Dingyuan Formation in the central Hefei Basin were selected for detailed detrital zircon LA-ICP-MS U-Pb dating. The detrital zircon age frequencies in the Zhuxiang Formation show Triassic and middle Neoproterozoic age clusters, whereas the Dingyuan Formation has a predominant middle Neoproterozoic age cluster, with secondary age clusters in the Paleoproterozoic and the Archean. All the samples lacked or contained few Early Cretaceous zircon ages. Combined with the results of previous studies, it can be concluded that the provenance of the Zhuxiang Formation is mainly the Dabie Orogen and that its formation age corresponds to the late stage of the Fenghuangtai Formation in the southern Hefei Basin. Different detrital zircon age frequencies in different samples correspond to the process in which the high-pressure and ultrahigh-pressure rocks were gradually denuded in the Dabie Orogen during the Early Cretaceous. In the Paleogene, the Hefei Basin was surrounded by a series of uplifts, which greatly reduced the diversity of clastic material sources of the Paleogene Dingyuan Formation. Because the detrital zircon age frequencies of the Dingyuan Formation are consistent with those of the Sinian strata in the South China Block, it can be inferred that the source materials of the Dingyuan Formation mainly come from the Sinian of the Zhangbaling Uplift. The provenance of sedimentary rocks in the Hefei Basin changed from the Dabie Orogen to the Zhangbaling Uplift at the end of the Late Cretaceous or the beginning of the Paleogene.