To design efficient thermal recovery procedures for structural materials in fusion energy applications it is important to characterise quantitatively the annealing timescales of radiation-induced defect clusters. With this goal in mind, we present an extension of the Green’s function formulation of Gu et al. [1] for the climb of curved dislocations, to include in the same framework the evaporation and growth of cavities and the effects of free surfaces. This paper focuses on the mathematical foundations of the model, which makes use of boundary integral equations [2] to solve the steady-state vacancy diffusion problem. Numerical results are also presented in the simplified case of a dilute configuration of prismatic dislocation loops and spherical cavities in a finite-size medium, which show good agreement with experimental data on defect annealing in ion-irradiated tungsten [3].