These limits imposed on us are there for a reason.
Adding your "safety" buffer is setting them up for failure.
Imposing personal hour reserves for day VFR or what have you, is akin to placing bubble wrap on the world around you. Same with limiting x-wind exposure to 10-12 knots instead of the demonstrated 17 or so for most light Cessna's. Same with limiting VFR flights to 5 sm - 5000' or greater.
A pilot who knows the plane's and his own limits is a much better pilot and will make much better decisions when the time comes.
Reminds me of a college flight school grad I took flying . The day was about 5 sm and was about 2500 broken. He was skeptical about going VFR for a local flight.
When flying outside the mostly controlled world of flight training refusing a flight that is still VFR (like my example) will get you fired in a heart beat. Flying a twin like a Navajo, the difference between 30 and 60 minutes of gas is closer to 100lbs. 100lbs of possible freight, perhaps a passenger.
Like I said earlier, too many people are teaching a test, and not real world flying.