The installation of distributed generation (DG) can be used to minimize the total power loss in the distribution network. Besides that, other power system performances such as voltage profile, stability index, and total harmonic distortion can also be improved via DG. Although many works on DG have been done, most researchers have assumed that the distribution line is in an ideal condition (unlimited capacity limit). On the contrary, all practical lines should have their own capacity limit. Therefore, the main contribution of this paper is to determine the optimal DG output that can fulfill the maximum allowable line capacity limit (MALCL). Furthermore, a penalty factor in rank evolutionary particle swarm optimization analysis is also proposed to handle the constraint. A 33-bus distribution system is used as a test system to investigate the performance of the optimization technique. The results showed that the line capacity increment caused by optimal DG output is always less than the MALCL value. Furthermore, the total power loss value in the system is increased when the MALCL set by utility is reduced. In terms of optimization performance, the proposed algorithm gives faster computing time and consistent results compared to conventional particle swarm optimization.