The first advanced robot many of us will experience may well be an autonomous vehicle (or driverless car). Autonomous Vehicle (AV) technology has made huge strides in recent years moving from a mainstay of science fiction to the first forays on public roads. The promise in AV stems from a wide range of potential and perceived benefits, the biggest of these is increased road safety through a reduction in the number of driver error-induced accidents, but the challenges facing the AV community as they attempt to transform the way humans interact with our cars are numerous – and not all technical. For example, will robots suffer from road rage? This paper explores these challenges, uncovers some findings around public perceptions of Autonomous Vehicles and presents a way to help de-risk the driverless car development via ‘Fenced Site’ testing. This will enable interactions between human and robots to be monitored and fed back into development for the safe and controlled introduction of driverless cars on to our roads.