Temporal preconditioners to stabilize the marching-on-in-time (MOT)-based time domain integral equation (TDIE) solvers are proposed. Exponentially decaying functions are used as temporal preconditioners and the proposed scheme is applied to analyze scattering from perfect electrically conducting objects using the second-order formulation. The effectiveness of the proposed scheme is demonstrated via numerical examples. It is shown that the temporal preconditioners stabilize the MOT system and the solution. In addition, the initial condition problem of TDIEs is investigated by extending the second-order formulation of the time domain electric field integral equation to the time domain magnetic and combined field integral equations.