That's far from obvious. Older avionics make for cheaper aircraft. Sometimes the aircraft you need (say, a 172N with > 2400 lb gross weight) doesn't have a GPS. Sometimes GPS itself breaks.
You should never be totally dependent on a toy GPS, say, due to overflying fair weather cumulus.
I fly with a GPS maybe 20% of the time. Usually Garmin 430. I find it's a distraction in busy airspace or mountain flying, both situations where you want your eyes outside the aircraft as much as humanly possible. That's how you detect sinking or rising air and anticipate turbulence, not through the GPS. It's also how you avoid terrain and traffic.