I can understand the idea of not wanting some phantom "arrow" floating around, when part of the ethos of the game is to ground everything in some sort of rationalized reality and remove the fantasy/video game feeling as much as possible. (at least that's how it seems to me)
But obviously, there needs to be some form of long-range navigation to avoid being lost completely on a long journey.
Is there some way to incorporate a sextant and compass sort of navigation system that you would have to actively participate in, and use that activity/skill to make the party arrow appear? Or do you think that would spoil the current cartography skill? My immediate thought is that would be fun to have a hearthland "astronomy"

Maybe our "sign" could be determined by the date we created the character? OK, that's probably going overboard... but active navigation might be a fun way to make the party "arrow" seem a little less disjointed without removing it alltogether.
"Bein' a minotaur is a lot like bein' a regular human except moo" - J. Rowland