That's assuming the government can still collect all that money, without all the young foreigners who are currently tipping the scales of our economy into less than 1% growth. But yes, you're right - I was merely playing devil's advocate with my first point.
One thing the EU does is spend money on infrastructure projects. We're a net contributor of course, because our infrastructure is already better than some of the other countries in Europe. With more infrastructure investment, their economies could take off and allow them to import more UK goods - currently 51% of our exports go to Europe.