This paper presents a review on the recent progress of digital signal processing (DSP) in the high-speed optical transmission system using single-carrier based multi-level modulation formats. The principle of several digital phase and polarization tr