A new type of resonant magnetic perturbation (RMP), generated by helical coils, is proposed for controlling the edge localized mode (ELM) in H-mode tokamak plasmas. The helical coil optimization utilizes the MARS-F code (Liu et al 2000 Phys. Plasmas 7 3681) computed linear resistive fluid response of the plasma to the applied RMP field. The optimal helical coils are found to be located near the outboard mid-plane of the torus, with relatively simple shape but tilted towards the equilibrium magnetic field line pitch. Compared to the window-frame ELM control coils, the optimal helical coils require 2-6 times less current, in order to achieve the same ELM control performance specified by various figures of merit adopted in this work. The results from the present study show a promising path forward in achieving ELM control with RMP fields in tokamak plasmas.