The study of the phonon Hall effect observed in a number of experiments suggested that spin-orbit coupling could introduce important corrections to the phonon dynamics of magnetic insulators. In this talk I will give a brief overview of the phonon dynamics in such gapped systems. Then I will introduce the phonon dynamics we derived in magnetic metals. In the dynamics, both a dissipationless viscosity and a dissipative viscosity arise. While the dissipationless viscosity splits the dispersion of left-handed and right-handed circularly polarized phonons, the dissipative viscosity damps them differently, inducing circular phonon dichroism. Further I will show the circular phonon dichroism in Weyl semimetals, which can be very strong when both time-reversal and inversion symmetry are broken.