Reported impressive reduction in accidents due to ADS-B In

Initial report of 53% reduction in accidents overall and 89% reduction in fatal accidents with use of ADS-B In. If this holds up, that is an impressive reduction. Much more significant that a lot of training programs combined.

https://www.aopa.org/news-and-media...udy-shows-accidents-less-likely-with-ads-b-in

 

Let me be the first to call bullship on this study.

 

That, and I am cursing, is ****ING AWESOME. The planes that didn't hit each other can be so happy with their stupid "safety" device.

IBTL

 

I'd be curious to what people think of the methodology. Link to full report https://download.aopa.org/advocacy/2019/dhowell_jking_DASC2019_V2.pdf

 

Possible self-selecting dataset?

 

Not sure how adsb helps on takeoffs and landings

 

Where are the p-values?

I'm guessing they would not pass muster for statistical significance.

It looks like the average proportion of ADS-B in equipage during the 5-year study period was about 5% while the average number of mid-airs per year was 0 for equipped and about 10 for non-equipped.

Some simple math, 5% (equipage rate) of 10 (number of non-equipped mid-airs) is less than 1 so how could you possibly draw a conclusion about whether or not ADS-B reduces mid-airs?

 

Donno about the study but I can definitely see a lot more that’s around me which otherwise I won’t have even know. Both good and bad but I prefer having more information that less.

 

I am skeptical of anything AOPA publishes. From my own perspective though, I think it has drastically increased airborne safety. Before, we never knew how close we really were. Now, I watch them paint and look out the window, continuously referencing back to the target and ill be damned.,.... there it is, under a mile away before I can see it.

I'm curious what you are calling BS on.

 

I'd have to look carefully at the methodology to see if the study compared similar cohorts of aircraft or, as I suspect, just looked at the entire fleet and ascribed causality to an observed correlation that may be fraught with covariance with other factors like aircraft capability or operator experience.

Just so you know, the FAA is using this study to justify increased funding for additional ground stations which is probably a good thing for all of us even if it's based on naive data analysis.

On the face of it it does seem a little fantastical that installing ADS-B reduces the accident rate by 53%, especially considering much of the fleet is already equipped with portable ADS-B in. Those aircraft should have seen similar safety gains.

 

I found it odd that in table 2, weather related accidents took a big jump in 2017 (some in 2016 too) even with the ADS-B equipped planes.

Yes, I do know that is a large reduction from non-equipped, but wondering why the increase all of a sudden for the equipped planes.

 

About 50% of the time I can identify visually the A/C after it pops. But I have made cautionary course deviations due to the traffic displayed. I like the capability

 

I love ADS-B, there I said it. It is great to have and really enhances traffic awareness, not to mention weather awareness. The actual study, posted above looks pretty thorough at first glance. This is great news if it holds up.

 

Appears they used a binomial test based on estimated total operations and reported accidents. They say they only report statistically significant results but don’t give details on the p-level (presumably 5%) or how they corrected for multiple comparisons.

Nonetheless, on first read and without the details that seems as though it is valid. The effect size here is large (surprisingly so actually) and they have estimated fairly large number of operations, so a significant effect would not be surprising.

Could be other methodological issues? It seems like a very large effect.

 

I think this is the potential big confound. Maybe people who took the trouble to have ADS-B during this timeframe are also more conscientious pilots.

Seems like that could be addressed in the dataset because the NTSB reports usually have estimates of pilot experience and currency.

 

Or combined the equipment with other maintenance items like repairing a scratchy radio or replacing a cloudy windshield.

 

Let me help you ADS-B clowns out:

1) Something didn't happen today - Praise the ADS-B system

2) Something happened less frequently in our study period than our made up prior history period - Praise the ADS-B system

3) "I've got ADS-B and I don't like looking out the window" You're families will not praise the ADS-B system, unless you are heavily insured.

 

Why does it have to be black or white Rob? Have you flown in an airplane with traffic reporting?

 

We are getting to the point where a slight gain in safety is only achieved through enormous expense.

 

ADS-B is great technology, no doubt. The "free" <ack!> weather and traffic is wonderful for SA.

That said, you'd be hard pressed to find many accidents ADS-B would have definitively prevented. Exactly how many mid-airs are there every year? It is a very small number. Weather? Until ADS-B actually changes the weather, I struggle with claiming accident reductions as a benefit.

As I wrote a while back, most pilots would get a lot more safety bang for their bucks with more recurrent training, rather than by spending the same money on a gadget.

 

Yup. C177RG, PA32-300, and I have ADS-B in on my RV. Waiting on shop schedule to install the 330ES for out.

And with all that.., not a fan.

 

Reading the link now....first thing a medical reviewer is going to look at when perusing research is its source in order to possibly detect bias. This says Regulus Group did the research then goes on to say how they contract with the FAA etc...does RG benefit from an infusion of cash into the ADS-B system full stop.

 

No, it isn't. Maybe you didn't see my edit.

Let's say there are an average of 10 mid-airs per year. Let's also say that 5% of pilots are named Charlie. Let's further say that during a particular year, zero pilots named Charlie were involved in mid-air collisions. Would that surprise you? Would you think that being named Charlie helped you prevent mid-air collisions?

 

Having more options is just too taxing ,isn’t it ? Must be hard to even tolerate your engine up there - damn all these distractions ...

 

We have to remember too that if the accident rate is for example, halved - half of nothing is still...nothing.

 

Lies, Dam* Lies and Statistics. Mark Twain.

Correlation does not verify Causation. Me (Engineer, not a Mathematician)

Cheers

 

I know I'm a bit dense, but what the **** are you asking?

Do you have some belief that ADSB is going to keep your prop spinning, or your eyes outside to see and avoid, or do anything of ANY potential value?

Other than for you to sound cool saying "Got 'em on the fish finder." Unfortunately it's going to lead to more fish getting caught, by somebody's prop.

 

It is pretty simple - it is just another option to help you out there , a poor man’s version of TCAS - nothing will guarantee you absolutely safety but why not take any additional help you can get ?

 

You missed a big one.

4) I’ve got adsb and now I know about all the times I almost, but didn’t, hit someone that before adsb I wouldn’t have even worried about.

I’d like to see a study that looks at whether having this info causes people to change their tracks more erratically increasing the possibility of an accident.

The odds of two planes on a straight and level course actually being in the same spot at the same altitude at the same moment really are very small.

PS> I use adsb in.

 

Space exploration is bad due to the increase in suicides.

Reducing consumption of Margarine reduces the divorce rate

Preventing marrige in Kentucky reduces drowning deaths.

Buy American! Japanese cars cause suicides!

 

Was out in the Long Beach practice area a few days ago doing radio calls every 3-5 minutes doing maneuvers and all of a sudden I get a traffic alert same altitude, he was heading right for me. I called out my location and altitude and started looking, couldn't see him and no responses. I finally spotted him and dove down, he never budged. Without ADSB I may have become a statistic.

 

Maybe, but odds are a million to one that you'd have blindly missed, or that one would have seen the other and would have engineered a miss.

 

I disagree with this Salty, the big sky theory is a very poor collision avoidance guarantee. Just about every flight around here I need to adjust or be vigilant about another aircraft to avoid intersecting paths. The last one was last week, I was at 2,500 feet, he was 3,000 feet. Pointed out to us by ATC, we were on flight following, he was not. We got visual, he turned and flew right over us. Not sure but I don't think he had a clue we were there. We have ads b in/out with traffic and we didn't even look at that screen instead of outside, it was really hard though.

 

While I think it's too early to cite statistics showing it's responsible for an "impressive" reduction in accidents, common sense says it has the potential to significantly reduce midairs and weather related accidents when the majority of traffic is equipped with In and Out.

I'm a believer.

 

500 feet, plus likely some amount offset to the side even if it seemed “right overhead”? You pass next to cars 10 feet away at a 100+mph speed delta all the time.

 

I think the binomial test should account for this. It is basically a test for a difference in ratios. In this case the fraction of equipped aircraft which have accidents vs the fraction of unequipped aircraft which have accidents.

They did not give the total operations which they are using for the denominator, though I gather it is in a general report.

I agree and would be surprised by the result in your example. However, the numbers they are using are different than that. 5% of the total number of GA operations under consideration is still a fairly large number. What would be needed to double check them is the values of the denominators in the ratios from the report they reference.

 

I agree there are a lot of possible biases that could cause the FAA or their contractors to want to over-inflate the benefits of ADS-B.

This type of argument though is a subtle form of attack on the source or speaker. Technically such bias does not indicate whether the argument is correct or not because even fools can occasionally speak the truth.

That is one reason I don’t entirely agree with the current policy of requiring disclosures of conflicts in medical journals. I think we should look for better ways of improving reviews and the accuracy and replicability of papers rather than trying to ferret out biases.

Disclosure of potential conflicts is required now though. OTOH, I know of no reviewers who make that a first concern. I usually find it more productive to focus on the methods and results and see if they are correct.

 

That was the latest Salty, we were at 3,000 feet, going 165 knots before ATC pointed them out. At least once every other flight, sometimes more I get turned for traffic. With big sky theory that should never happen. The issue is we are all flying to and from a limited number of fixed points and it's bound to happen. I like the extra tools.

 

It is a good point that correlation does not imply causality. But that does not mean that all such correlational studies should simply be discarded.

Rather there are other indicators of causality that need to be present to suggest causality.

One such is plausible mechanism. Is it plausible that having information about the location of other aircraft and weather in the cockpit helps pilots to avoid collisions and fatalities? Seems at least plausible to me.

Another is the lack of other potential causative factors. Are there other possible things happening in GA in these 4 years which would have caused this near halving of accidents in the planes equipped with ADS-B? I will leave it to others more knowledgeable than me to suggest those.

 

I didn't read the report cover to cover. But I see now you're right, they did check for statistical significance. And you're wrong, there was none as far as mid-airs are concerned.