Quid Pro Quo.
There's a difference between, "Hey, here's the keys to my plane, fly it whenever or wherever you want," and "I need you to fly my plane from ABC to XYZ." The former is legal, the latter is not with only a private.
Of course that's only if someone complains to the FAA...