I have pilots who routinuely put that in the remarks. As far as I'm concerned as a controller if you aren't filing as a /G you obviously aren't able to file to RNAV waypoints. That being the case if I say fly heading such and such when able direct (insert VOR here) and you go direct based off of the VFR GPS 30 miles before you get a usable signal, thats your call not mine. I can't tell if you're getting a legal usable signal.
As far as does it make a difference, it might depending on who is working you and the situation. There is a fix in my airspace that gets used over and over and over and provides a big time saving shortcut for our GA pilots, but its location is one where there is little margin for error. If I have a pilot who is on a practice IFR flight and is tracking their practice ILSs and Localizer approaches a half mile off consistantly I typically won't trust them to use this fix. If however I get the feeling that they are confident and capable and that it is an issue with their equipment I might give a second thought to using that fix if I know they have a tool to enhance their situational awareness.