I think something to consider is you are choosing to fly on days with cold fronts in the area. Those make for nearly constantly changing weather patterns and much more difficult conditions to predict. Consider selecting days that are relatively predictable and relatively clear weather days at the start of your XC experience building. Even on those clear type days, you'll see clouds and you will slowly venture out on days less than ideal. My CFI said to me that he did not want me to be "cloud shy" meaning, clouds are worth respecting but we can't only fly on clear sky days as those are far too rare. Lowering visibility is an entirely different animal. I have flown in 5 mile visibility and you are correct it is very different from flying on clear and unlimited days. When the visibility drops, you can't mess around. Clouds are important to avoid but in order to avoid them, you have to be able to see them. I always think visibility is more important than cloud coverage. Sounds like you are handeling gaining experience slowly in a very repaonsible way. If just be a bit more selective on the days you fly at the start and eventually you can branch out into less than ideal days!