It does come down to road sense and experience. I spent 8 years commuting daily in London, riding out of London to train and then dealing with the Chelsea Tractors on the country lanes. Some of the descents in the Surrey Hills are pretty sketchy. Short, steep and covered with leaf litter but you learn how to ride them. One of the best things I ever did was to join a club. You go out in a large group which is always safer and you get to learn from experienced roadmen. As long as you ride within the limits of your own bike handling ability and back off especially if you don't know an area or a corner you should be fine. Obviously you can't account for the rogue maniacs but, if you lived your life trying to, you'd never do anything.