The farm is easily avoided, just leave the farm track at the corner of it before the farm (when the track bends right) and you'll see a right of way arrow pointing over a field ahead and uphill slightly. Head past the caravans which you'll see on your left and basically head stright for the bottom of the heathery rocky ridge that you climb on the race. Cross the fields and you'll pop out on the track that leads out of the farm.
Or, you could just ask the farmer, he's always been very nice.