Denverpilot mentions carrying a camera; its a very good idea to have one in your flight gear all the time.
I experienced an incident a month or so ago that didn't involve aircraft damage, but damage to a hangar. I parked in a very tight parking area and was the last off the aircraft. Someone came on board who spoke very little English, and what they did speak was difficult to understand (we were in Latvia). They wanted to know about a percent. It took some doing, but I figured out they wanted to know what percent power I had used in the parking process. I had no idea; it wasn't much, but I was busy looking outside, not in. I was soon joined by a representative of airfield management, who informed me that two large plate-glass windows on the FBO hangar door had been "blown out."
I walked the ramp area, and took a lot of pictures. I got on a long distance call with the Chief Pilot and safety department, and we looked at the ramp area together during the call, using Google Earth. I then spent about 12 hours putting a report together with the pictures, and labeled the pictures and test to show exactly what happened. The company was informed that I did damage and carried so much power I blew out the windows in the hangar door. That wasn't true, and I had photographs of rocks in the broken glass showing the ramp area had FOD that was responsible for breaking the windows; they were never blown out, and the second pane in the double pane windows were intact.
I was able to show exactly what happened, even half-way around the world, using a small camera and my laptop. You may not run into that same circumstance, but I've had plenty of cases pop up over the years where a picture makes a big difference in putting everyone on the same page. It may be that you're at a remote airfield somewhere with your rental and have a problem; being able to snap a picture and send it to the mechanic/FBO/owner might be exactly what's needed to address the situation.
In the case of rentals, or employment, or even one's own airplane, being able to say "no" is always a crucial trait to survival. Making the go/no-go decision is something that all students are taught from the beginning of training. For many, it "takes," and for some it never does. It sounds like you're on the right track, as you've displayed a history of knowing when to say "no."
A few years ago I detected a slight vibration in a new Air Tractor AT-802 that I was flying on a fire. The airplane only had about fifteen hours or so on the airframe when I got it; new engine, propeller, and airframe. Those kinds of airplanes do experience odd buzzes and vibrations from time to time; many of the fuselage panels are attached with quick disconnect camloc fasteners, and they do vibrate a bit. I was in a lot of turbulence in mountainous areas, and tended to discount the vibration. I still gave the airplane a very thorough preflight and post flight.
I handed the airplane off to a friend and went somewhere else to fly the same type aircraft. My friend grounded the airplane a few days later; it developed excess play in one propeller blade and the prop had to be removed for an overhaul. Even new, it needed major maintenance. The vibration was the airplane talking to me, and it only took a few more flights before it became pronounced enough to demand repair.
Some years ago, in the middle of a firefighting operation, we stopped operations because of some small stains on the underside of the left wing of a C-130A. We removed the paint on the underside of the wing and did a field dye penetrant test; we discovered two small cracks, one an inch long, and the other an inch and a half long. We grounded the airplane, then got a special flight permit to return it to the United States (we were in Mexico at the time). Upon return to the home base, we drained the fuel, and did did full non-destructive testing on both wings. One wing had to be scrapped and replaced, the other required extensive repairs. Those small stains were the only warning we had. From then on, the wing got extensive inspections on a regular basis at short intervals.
Several years later, both wings came off the airplane over a fire in California, killing all three crew members. They were friends of mine. Small things become big things, with dire consequences. Never underestimate what you might find.
I was contacted by a pilot at an ambulance operator, complaining of a popping sound when the flaps were deployed in a Seneca II. The matter was brought to the attention of the Director of Maintenance, but in the shop, the problem wouldn't duplicate itself. The DoM requested the pilot to fly around the pattern with the DoM on board, so he could hear it for himself. He later recounted to me taxiing to the runway, and feeling the flooring move under his feet. He immediately cancelled the flight and demanded the airplane be brought back to the shop.
He expected to find a pulley issue, or a weak component, based on the initial descriptions. What he actually found was very different: the spar was cracked completely through in three places, and the popping sound was the spar grinding against itself as the load changed when the flaps were applied. It was only a matter of time until the wing(s) came off.
What appeared to be a very small spot of surface corrosion on a Sabreliner turned out to be intergranualar so pervasive that the aircraft was unserviceable and unairworthy. It was throughout the structure, coming to the surface, and upon inspection, the I found the aircraft wasn't safe. I informed the owner and left it in his hands. The problem was found during a routine minor inspection, part of a progressive maintenance program, that didn't even involve the surface where the corrosion was found. It was just the proverbial tip of the iceberg.
I've found slight dents in surfaces that turned out to be substantial damage beneath the surface. The understructure was damaged or cracked. I found something like that in King Air 200 once which required a trip to a repair center for spar work. In another case, the vertical stabilizer attach bolts for a large four engine airplane were discovered cracked or fatigued beyond use; all of them, and this wasn't discovered until a "D Check" was being performed. It's entirely possible that the tail could have come off the airplane. I saw something similar happen on a fleet of 200 series Cessnas (fleet size about 30, mostly 206's and 207's), in which virtually all the vertical stabilizer attach brackets were broken. Until the tailcone was removed, the only indication was a minute amount of play.
The point isn't scare stories, but rather the fact that small things become big, and that you should always take very seriously any anomaly discovered. It could be a turboprop that I learned wasn't properly bonded; the only indication was barelly noticeable; the static dischargers were installed after paining, rather than before, and painted over. The airplane didn't discharge properly, and experienced severe damage that required removal of engines and propellers, and had holes burned through control surfaces, props, and much of the structure. Small things become big. It could be a new Cessna sky master I was asked to evaluate following an engine installation; I missed the fact that a plug hadn't been safetied on the engine. It vibrated free, we lost all the oil in flight and had to shut the engine down. It could be the slight whistling noise I heard in another airplane that announced the impending failure of the door seal, and a rapid depressurization that happened shortly thereafter. The warnings might be only slight or very subtle. The results not so much so. Take everything seriously. It can quickly become that way, often at the worst time possible.