Materials subjected to neutron irradiation will suffer from a build-up of damage caused by the displacement cascades initiated by nuclear reactions. Previously, the main “measure” of this damage accumulation has been through the displacements per atom (dpa) index, which has known limitations. This paper describes a rigorous methodology to calculate the primary atomic recoil events (often called the primary knock-on atoms or PKAs) that lead to cascade damage events as a function of energy and recoiling species. A new processing code SPECTRA-PKA combines a neutron irradiation spectrum with nuclear recoil data obtained from the latest nuclear data libraries to produce PKA spectra for any material composition. Via examples of fusion relevant materials, it is shown that these PKA spectra can be complex, involving many different recoiling species, potentially differing in both proton and neutron number from the original target nuclei, including high energy recoils of light emitted particles such as _-particles and protons. The variations in PKA spectra as a function of time, neutron field, and material are explored. The application of PKA spectra to the quantification of radiation damage is exemplified using two approaches: the binary collision approximation and stochastic cluster dynamics, and the results from these different models are discussed and compared.