As 5G wireless networks evolve to 6G, the necessity for precise channel prediction intensifies, but approaches to date have lacked generalizability in unseen frequency bands, locations, or dynamic environments. To overcome these challenges, we propose RadioTwin, a high-fidelity, physically interpretable digital twin of the real-world radio environment. In contrast with prior deep learning approaches, we model ray-object interactions in the ambient environment using physics-guided models and train a neural network to infer the intrinsic material radio parameters of the environment. We incorporate this model into a differentiable ray tracing framework to characterize how wireless signals reflect, refract, diffract, and scatter when bouncing off different objects. This integration empowers us to predict wireless channels across different links and frequency bands, even in dynamic environments. We optimize the training and inference computational efficiency of RadioTwin and fully integrate it into Sionna ray-tracing framework. Our evaluation shows that RadioTwin achieves consistently higher channel prediction accuracy than SOTA systems: over 5.5dB, 4dB, and 4.7dB median EVM improvement on cross-band, cross-link, and the more challenging cross-band and cross-link channel prediction task, respectively.