The problem with speculation is not for us. We probably benefit from it. We're also smarter than the average idiot out there.
The average idiot is the problem. Said people take speculation as fact, and then ignore the findings when investigation shows what really happened. Said people then spread the speculation as fact to others, and we wonder why aviation has a bad reputation. The media feeds on speculation, because its goal is ratings and money, not reporting the news. So the more horrifying the speculation the better, facts be damned.
When we speculate amongst ourselves, we probably learn something. When others hear it, it hurts us since they'll take the worst possible scenario and run with it.