It does occur while walking, but not while running, although I first noticed it while cycling. There does seem to be a psychological element to the condition (the bladder sending misleading messages to the brain), so here's a hypothesis (without any evidential basis!).
Maybe it was initially caused by perineal irritation (the saddles on both of my bikes feel comfortable, but that doesn't exclude the possibility of something going on down there without me noticing). There then arose an association in my mind between prolonged moderate exercise and bladder discomfort (the intensity of my cycling is usually similar to a brisk walk, and much less than the intensity of my running), so I started feeling it while walking. But when running, the brain is too distracted by the more intense exercise.
I've never tried decaffeinated beverages, due to a bias (without evidence) that they don't taste as good. Maybe I ought to try.