We develop a dynamic model for the evolution of an ensemble of hundreds of interacting irradiation-induced mobile nanoscale defects in a micrometre size sample. The model uses a Langevin defect dynamics approach coupled to a finite element model, treated using the superposition method. The elastic field of each defect is described by its elastic dipole tensor, and the long-range interaction between defects is treated using the elastic Green’s function formalism. The approach circumvents the need to evaluate the elastic energy by means of volume integration, and provides a simple expression for the energy of elastic image interaction between the migrating defects and surfaces of the sample. We discuss the underlying theory, and also the parallelization and coarse-graining numerical algorithms that help speed up simulations. The model addresses the issue of imbalanced forces and moments arising as an artefact of the modified boundary problem associated with the traction free boundary condition. To illustrate applications of the method, we explore the dynamic evolution of an ensemble of interacting dislocation loops of various size and with different Burgers vectors, which proves the feasibility of performing large-scale simulations using the proposed model.