I mainly wore nice pants and a button up, even to my private checkride. I wanted to look professional and show the examiner some respect as well as that I was serious about the checkride.
Yes, you are graded on your flying ability, but if you were in their shoes would you want to fly with...