Effects of neutron irradiation on materials are often interpreted in terms of atomic recoils, initiated by neutron impacts and producing crystal lattice defects. We find that, in addition, there is a remarkable two-step process, strongly pronounced in heavy elements, involving the generation of energetic γ-photons in non-elastic collisions of neutrons with atomic nuclei followed by the production of high-energy electrons through the scattering of γ-photons by the atomic electrons. This two-step scattering creates a non-equilibrium steady-state population of high-energy electrons in the bulk of the material, which bombard atoms and stimulate vacancy diffusion, resulting in a variety of driven athermal microstructural reactions including the enhanced recombination of radiation defects. We find that tungsten converts the energy of fusion or fission neutrons into a flux of γ-radiation and subsequently high-energy electrons with the conversion efficiency approaching 99%, explaining the low defect content observed at moderate temperatures.