I've never heard of sandpapering brake blocks before. Cleaning your rims, yes - I do this after virtually every ride.
With rim brakes some particles of grit/sand/salt (in winter) can get onto the rims and is then forced into the metal of the rim when braking. If you clean the rims regularly then you prevent a build up of contamination, and hopefully prevent a particle from being forced some depth into the rim.
If you don't clean your rims then you can end up with grooves on your rims which reduces your braking area. It also makes the rim wall thinner and will eventually lead to the rims failing when riding the bike. If you have access to a Vernier gauge you can measure the thickness of your rim walls.
In most cases cleaning the rims and fitting new blocks should solve poor brake performance. If the rims have deep grooves I'd change them. Rims are relatively cheap, and it only takes me about an hour a wheel to take off an old rim and build in a new one. Importantly, if you do this you will need to measure the ERD (effective rim diameter) of your existing rims and buy ones the same - otherwise you'll need new spokes and for the cost and effort you may as well have bought a new pair of wheels.