FAA Exam Software

[flyingron]

I am not entirely convinced that this isn't intentional.
if you look at the knowledge test as a learning tool vs a test it makes more sense.

When they ask a question where the answer isn't obviously correct it should force the student to learn why one answer is better than the other i.e. actually research and understand the topic better. I am know this doesn't always happen as often these kinds of questions the correct answer just gets memorized. But it at least gives the student the opportunity to learn the subject a bit better.

Brian
CFIIG/ASEL

No, it's not intentional, they're poorly written. Intentional confusing distractors is a valuable test design strategy, but the FAA has no rhyme or reason to theres and sometimes the right answer is just wrong but it's the least wrong (if for no other reason than the FAA says that it is the right answer). In addition, the tests are just poorly proofread. It's clear nobody in the FAA independently took the tests in the final form to find that stupid things like fixes involved are not in the chart fragments or other typographical errors.

Then don't get me started on the erroneous content like much of the FOI or the moronic definitions of forces of flight and coordination on several of the tests.

 

[pfarber]

So an EFB is mandatory for flight?

I would normally cite FAR 91.103 as the catch all but that requires a deeper understanding which I'm not sure most people have.

The written test is a joke. Memorize the questions from the testing app and get your 90%+.

Let me guess... majority of PoA hasn't taken a written PPL exam in the last 20 years.

And What airplane is the disaster from? But you better know how to read it lol

 

[pfarber]

I am not entirely convinced that this isn't intentional.
if you look at the knowledge test as a learning tool vs a test it makes more sense.

When they ask a question where the answer isn't obviously correct it should force the student to learn why one answer is better than the other i.e. actually research and understand the topic better. I am know this doesn't always happen as often these kinds of questions the correct answer just gets memorized. But it at least gives the student the opportunity to learn the subject a bit better.

Brian
CFIIG/ASEL

There are several questions where the right answer and the FAA answer is not the same. The FAA reasons for one over the other are not about in depth knowledge... but simple 'one is more correct'. lol sure.

The real answer is regurgitate whatever the FAA wants you to know.

The written test is a joke. Its a bunch of random questions

 

[brcase]

I would normally cite FAR 91.103 as the catch all but that requires a deeper understanding which I'm not sure most people have.

The written test is a joke. Memorize the questions from the testing app and get your 90%+.

Let me guess... majority of PoA hasn't taken a written PPL exam in the last 20 years.

And What airplane is the disaster from? But you better know how to read it lol

View attachment 141416

 

[robin ardoin]

The knowledge test is just a screen and the FAA has never had a particularly good one. It's full of junk that isn't the least bit relevant, some of which is just flat wrong or a figment of the FAA's imagination, so you just have to learn what they want to hear. Shepherd or Irwin Gleim or any of the ones who specialize in greasing you through not only is legitimate, it's often essential.

I would presume that the, in my opinion, low bar of %80 correct threshold for a passing score covers those questions that have an erroneous 'correct' answer and would likely be sufficient for poor questions in an AI generated test.

 

[MauleSkinner]

And What airplane is the disaster from? But you better know how to read it lol

View attachment 141416

It’s pretty easy to read, and it doesn’t matter what airplane it’s from (if any.) The written test isn’t intended to be aircraft specific, or even specific to an ex-Air Force instructor and check airman with thousands of hours of heavy time. It’s intended to be a test of broader knowledge that applies to a broad pilot demographic.

 

[dmspilot]

I think the written tests are not as bad or worthless as people are saying, but they do indeed have many flaws. Keeping the test bank secret means the FAA gets to evade accountability and responsibility for its tests. Then we get questions like this:

The destination airport has one runway, 8-26, and the wind is calm. The normal approach in calm wind is a left-hand pattern to runway 08. There is no other traffic at the airport. A thunderstorm about 6 miles west is beginning its mature stage, and rain is starting to reach the ground. The pilot decides to ​

​

A. fly the pattern to runway 8, since the storm is too far away to affect the wind at the airport. ​

​

B. fly the normal pattern to runway 8, since the storm is west and moving north and any unexpected wind will be from the east or southeast toward the storm. ​

​

C. fly an approach to runway 26, since any unexpected wind due to the storm will be westerly.​

​

I've been flying for 27 years and I don't even know what answer they want.

Rod Machado "just happened" to see this question (it appeared on a sample test, otherwise he might have found out about it thanks to those evil cheating-enabling test prep companies), just happened to be friends with an FAA inspector who just happened to be friends with someone at AFS-630 who just happened to have the authority to delete the question from the test bank.

Even for things that are not secret, like the knowledge test supplement, there are excessive errors that the FAA won't fix. I think it's been 10 years since every chart excerpt became a different scale and rendered plotters worthless for measuring distance. Without test prep companies advising people of the mistake, many would have not noticed and would have gotten questions wrong. But even then, knowing about the mistake is not enough. What happened to the concepts of positive and negative transfer of learning the FAA likes to hammer?

 

[MauleSkinner]

I think it's been 10 years since every chart excerpt became a different scale and rendered plotters worthless for measuring distance.

I was advised 45 years ago to check the chart’s scale against my plotter before trusting measurements. Does anyone fly charts that work with a plotter anymore?

 

[dmspilot]

I was advised 45 years ago to check the chart’s scale against my plotter before trusting measurements.

That's good advice which is probably meant to ensure you do not accidentally use an incorrect scale (plotters may have 3 different scales on them). I doubt "the FAA accidentally printed the chart with the wrong scale" was the reason you were told that. If it was though, who is to say the FAA didn't also print the scale incorrectly? There is an element of trust that the FAA won't royally screw up that bad.

Does anyone fly charts that work with a plotter anymore?

Paper charts still work with plotters. But not on the written test.

 

[MauleSkinner]

That's good advice which is probably meant to ensure you do not accidentally use an incorrect scale (plotters may have 3 different scales on them). I doubt "the FAA accidentally printed the chart with the wrong scale" was the reason you were told that. If it was though, who is to say the FAA didn't also print the scale incorrectly? There is an element of trust that the FAA won't royally screw up that bad.

I believe it was based on “these are photocopies (or whatever the 1980 equivalent was) of charts, and they may not copy exactly the right size. Bottom line is that measuring the distance and comparing it to the scale on the chart is nothing new, and is probably more the norm for EFBs. Not to mention the different scales on airport diagrams, at least with Jepp charts.

Like you said, generally speaking, the written tests aren’t as bad as people like to believe.

 

[Mikey52]

If you take the sectional and copy it with a reduced scale to print on 8.5 x11 paper it would be very lucky if any of the measurements on the plotter would match the scale of the map.

 

[write-stuff]

If there is a wrong question known to be on a Knowledge Test, it is possible to get them removed. I've done it with several questions over the last few years.

 

[dmspilot]

If there is a wrong question known to be on a Knowledge Test, it is possible to get them removed. I've done it with several questions over the last few years.

How does one do that with secret questions?

 

[Half Fast]

How does one do that with secret questions?

Can't tell you; it's a secret.

 

[Sierra1]

Can't tell you; it's a secret.

You just broke the number ome rule about secret questions dude

 

[Hengelo]

It matters because the tests take time, effort and money - all of which are valuable. I think we'd all agree that they can better; regardless of what axis you measure "better" on:

contributes to better adm
helps a student retain knowledge long enough to help them "fly gooder"
doesn't incubate anti-authoritarian sentiment in every graduate...
... the list goes on.

The only interesting question (to me) is how to improve an automated pre-practical screen. "Slather some AI on it" is unsatisfying to me.

 

[flyingcheesehead]

How is FAA written exam software such as Sheppard Air not considered cheating by the FAA, and enforcement action taken?
They literally solicit pilots to remember exam questions, and provide them to them.

The FAA used to publish the question banks themselves, so it's kinda hard to call it cheating.

If this occurred in Academia it would be considered cheating and punished accordingly.

Disagree...

When I was in college, I was taking 2nd-semester Chemistry and hating it, had a prof who put people to sleep and TAs straight off the boat from China that you couldn't understand. So, I didn't do so hot the first time and had to repeat it.

I found out the night before a major exam that this prof reused the same exams every semester... But my previous one was in a box at home. Desperate, I called my parents and asked them to dig up my previous semester's exam.

My mom refused - She said that would be cheating. My dad, however, dug up the exam and read me the questions over the phone. "If he uses the same exam again, it's his own damn fault if people use them to get a better grade."

My dad was a professor.

I don't agree. The knowledge test is basically a shotgun approach. Most of the performance chart questions are in a format I've never seen (the draw a line left to right hitting the values on column). What GA AC is that from??

Piper, Mooney, Beech, Diamond...

So, pretty much everyone except Cessna and Cirrus, who just use pages and pages of tables of numbers.

How on earth have you not seen that format before?

Also there are VOR questons but nothing about GPS.

In Sheppard, or on the test?

When I took the ATP written just before the ATP-CTP rule took effect in 2014, the test prep had tons of NDB questions, stuff about flight engineers, etc that were even then pretty much irrelevant, but NONE of that showed up on the test.

Ok…so pick a question that indicates “The fundamental problem with the FAA’s written exams is that someone thinks they need to be hard questions or that it should be difficult for the test taker to discern what is being asked.” I’m curious how you discern that thought process.

There are plenty of examples of poorly worded questions, questions where the correct answer isn't even a possibility and you have to pick the answer that is the least wrong, etc... The FAA written tests are garbage. A hoop to be jumped through and not much more.

 

[Hengelo]

All past exams were in the library at my University. I think it was an incentive for profs to write new material.

 

[MauleSkinner]

There are plenty of examples of poorly worded questions, questions where the correct answer isn't even a possibility and you have to pick the answer that is the least wrong, etc... The FAA written tests are garbage. A hoop to be jumped through and not much more.

Still not evidence of the claimed psychic abilities.

 

[Matthew]

I used Sheppard for my instrument written. There were at least two questions where Sheppard said (paraphrase), “although the correct answer appears to be xxx, if you see this question the FAA answer key is looking for yyy.”

I wondered if FAA did this on purpose; the people that chose the “correct but wrong” answer were the ones that didn’t use Sheppard. But then I figured, “Nahhh, it’s just wrong.”

 

[Sierra1]

I used Sheppard for my instrument written. There were at least two questions where Sheppard said (paraphrase), “although the correct answer appears to be xxx, if you see this question the FAA answer key is looking for yyy.”

I wondered if FAA did this on purpose; the people that chose the “correct but wrong” answer were the ones that didn’t use Sheppard. But then I figured, “Nahhh, it’s just wrong.”

Pilot institute has a few of those questions noted and mention the same issue in their training. Not to mention small typos like “alt” being a WAM chart label instead of aft. I think Occam’s razor applies, the FAA doesn’t proofread very well.

 

[Lindberg]

Pilot institute has a few of those questions noted and mention{s} the same issue in [its] training. Not to mention small typos like “alt” being a WAM chart label instead of aft. I think Occam’s razor applies[;] the FAA doesn’t proofread very well.

I have a secret for you: No one proofreads well.

 

[Witmo]

Honest question here.

How is FAA written exam software such as Sheppard Air not considered cheating by the FAA, and enforcement action taken?
They literally solicit pilots to remember exam questions, and provide them to them.

If this occurred in Academia it would be considered cheating and punished accordingly.

In the USAF we had an official publication called an MQF which stands for master question file. Everyone was issued a copy and they included the correct answers. We were expected to know the answers to these questions which covered things like ops limits and operational procedures. Whenever we had a checkride or the wing was being evaluated, aircrews were tested on the MQF. No one considered it anything but a file of standardized questions and answers we were expected to know cold in order to fly safely. Busting an MQF quiz was almost as bad as busting a bold face quiz. It was considered unprofessional.

 

[Sierra1]

I have a secret for you: No one proofreads well.

Well aware but we are talking about the issues people have with the FAA test/questions. That’s one issue that even the ground school companies mention in their training. You would think it would be fixed after a while though.

 

[Capt. Geoffrey Thorpe]

Ok…so pick a question that indicates “The fundamental problem with the FAA’s written exams is that someone thinks they need to be hard questions or that it should be difficult for the test taker to discern what is being asked.” I’m curious how you discern that thought process.

Just took a couple "Wings" modules with a test at the end. Got them all right except for the "which is not the correct answer" questions.
Why? Just why?

 

[MauleSkinner]

Just took a couple "Wings" modules with a test at the end. Got them all right except for the "which is not the correct answer" questions.
Why? Just why?

Where I work, it’s a question format we’re not allowed to use. Apparently that restriction doesn’t apply to the people who train us to teach.

Most of our training seems to revolve around, “Do as I say, not as I do.”

 

[bentmettle]

I don't agree. The knowledge test is basically a shotgun approach. Most of the performance chart questions are in a format I've never seen (the draw a line left to right hitting the values on column). What GA AC is that from??

i know the statement was undoubtedly rhetorical- this is for the DA40



This is from the PHAK



The FAA study material is a good reference for many of the FAA writtens.

 

[murphey]

All past exams were in the library at my University. I think it was an incentive for profs to write new material.

Half dozen of the most highly regarded computer science depts in the US have mid-term and final exams available online at the universities. Makes it REALLY difficult to come up with original CS problems. I've dropped the mid-term exam, my finals are almost trivial so I assign substantial multiple projects during the semester. There are times I work harder than the students.

 

[KE5BM]

Currently studying for my Instrument. I got this gem tonight just before getting on here:

"What is the third Fundamental Skill of Attitude Instrument Flying?" The question references Chapter 6 of the Instrument Flying Handbook. Chapter 6, states multiple times but especially under the header "Fundamental Skills of Attitude Instrument Flying" the following: "...it is very important that two major skills be mastered. Instrument cross-check and instrument interpretation...Without mastering both skills, the pilot is not able to maintain precise control of aircraft attitude." So what's the answer to the question? "Aircraft control." Then goes on to quote an excerpt from a non-existent version of the book citing three fundamental skills. I mean...to be fair it's only been 13 years since the IFH has been updated.

 

[bentmettle]

In the glossary-
Fundamental skills. Pilot skills of instrument cross-check,
instrument interpretation, and aircraft control.

G-7

Instrument Flight
To achieve smooth, positive control of the helicopter during
instrument flight, three fundamental skills must be developed.
They are instrument cross-check, instrument interpretation,
and aircraft control.

This is from Chapter 8, Helicopter Attitude Instrument Flying


*shrug*

Are you using Shepard to study or reading the material?

 

[KE5BM]

I'm reading the material. Like I stated above. Additionally, I'm looking for the answers while taking the practice tests. I find this helps me learn much more than just what is asked in the question. The question referenced a specific section of the IFH and that phrasing is not there. Although it does still appear in the glossary now that you've pointed that out. And I'm not going for helicopter so I haven't even opened chapter 8.

My CFII has recommended I start using Shepard...I really don't want to. I took a look at it a few days ago and I'm not impressed.

 

[M20K]

They literally solicit pilots to remember exam questions, and provide them to them.

Although it's been the "norm" for years, it is about to change and some people may get burned. I don't know if there's a known date yet or not, but the FAA in the near future is going to switch the tests up to randomized questions. So, as I understand it, if you and I went into a test center at the same time, the test you're taking and the one I'm taking would be different.

Sure, a lot of the questions may be the same but there will be a lot of different questions, basically asking the same thing, that will be rotated into different tests. The order will also be randomized (to some extent). So those people that are using the most up to date test memorization aid vs. actually learning the material, one day very soon will walk into the test and be in for a bit of a shock.

 

[KE5BM]

The NREMT exams work like that. The really great thing about those exams is if you know your stuff, you could be in and out of the test in less than an hour. The testing software would change it's question difficulty based on how you answered the previous question. The downside is if you were doing really well, you'd feel like you were failing because you'd get so many difficult questions. Some folks got their tests' end in about 30 minutes, though...they didn't do so good.

 

[midlifeflyer]

I'm reading the material. Like I stated above. Additionally, I'm looking for the answers while taking the practice tests. I find this helps me learn much more than just what is asked in the question. The question referenced a specific section of the IFH and that phrasing is not there. Although it does still appear in the glossary now that you've pointed that out. And I'm not going for helicopter so I haven't even opened chapter 8.

My CFII has recommended I start using Shepard...I really don't want to. I took a look at it a few days ago and I'm not impressed.

I don’t know if your comment is about Shepard in particular or the concept of drilling for the knowledge test generally. As to the latter, two things to think about.

1. “drill & done” is a strategy for taking standardized tests that is more than just memorizing questions and answers. It also familiarizes you with the types of questions asked, the way the questions and answers are formulated, and test taking strategies, including things to look for and gotchas.. It was long ago, but I can still remember be told “the most common wrong answer to the cross country questions is 180°off,” during my weekend private pilot knowledge test course.”

2. Nothing about using “test prep” to get through this gatekeeper prevents you from learning anything.

 

[pfarber]

View attachment 141422

Show of hands who got their PPL in a 172?

An Archer III?

Yeah, it's just not relevant. the written test is a joke. i got a 95 and I never used that chart.

regurgitating the answers FTW!

 

[billsgs126]

note- women never seem to forget anything so she gets more. LOL . . .

This only holds true with what a man has done or neglected to do.

 

[robin ardoin]

This only holds true with what a man has done or neglected to do.

Indeed, that was my implication.

 

[MauleSkinner]

Show of hands who got their PPL in a 172?

An Archer III?

Yeah, it's just not relevant. the written test is a joke. i got a 95 and I never used that chart.

regurgitating the answers FTW!

probably why many countries have type ratings for Archers or 172s. They don’t learn/teach “airplanes.”

 

[midlifeflyer]

Show of hands who got their PPL in a 172?

An Archer III?

Neither. This is the chart I learned with. Do I "like" that chart? No. Do I prefer the dumbed-down Cessna version? Sure. Do I whine about learning it? No.

 

[flyingron]

Although it's been the "norm" for years, it is about to change and some people may get burned. I don't know if there's a known date yet or not, but the FAA in the near future is going to switch the tests up to randomized questions. So, as I understand it, if you and I went into a test center at the same time, the test you're taking and the one I'm taking would be different.

That's always been the case. But the pool of questions is limited.