Detailed phonon transport at a rough semiconductor interface is studied by the molecular dynamics method. The simulation results show that phonon transmission coefficient is a strong function of frequency, roughness characteristic length and atomic structure. When the roughness characteristic length is comparable to the phonon wave length, the phonon transmission is decreased significantly. Complex phonon mode conversions and wave interference effects are observed.