I think we went all the way to the end of the track (to the ruined mine buildings anyway) then found a good trod through the bracken all the way to the upper reaches, when it was a simple matter of winding upwards between the craglets. This seemed a quite efficient route.
I think the point where we started upwards in earnest was more or less directly below Birk Fell Hause.