We propose a method for data-driven modelling of the temporal evolution of the plasma and neutral characteristics at the edge of a Tokamak using neural networks. Our method proposes a novel fully convolutional network that is capable of predicting the time evolution of the physics parameters along the two-dimensional representation of the poloidal cross section. More specifically, we target the evolution of the temperatures and parallel velocities of the electrons, ions and neutral particles at the edge. The central challenge in this context is in modelling together the different physics principles encapsulated in the evolution of plasma and the neutrals. We demonstrate that the inherent differences in non-linear behaviour can be addressed by forking the network to process the plasma and neutral information individually before integrating as a holistic system. Our approach takes into account the spatial dependencies of the physics parameters across the grid while performing the temporal mappings, ensuring that the physics dependencies are factored in and not lost to the black-box. Having used the conventional edge plasma-neutral solver code SOLPS to build the synthetic dataset, our method demonstrates a computational gain of over 5 orders of magnitude without a considerable compromise on accuracy.