Is passing the bar exam a relevant gauge of whether someone is qualified to practice law?

Answer 1

Yes. The bar exam tests examinees on legal principles which are required to be well understood to determine legal strategies in real applications. Your question is similar to whether a pilot can be effective without taking ground school. The answer is, yes, to a certain degree. A person could probably fly quite well with on the job training, however, when it came down to dual engine failure, could the pilot properly calculate glide distance to the nearest airport. Could that pilot properly understand the aerodynamic principles? This is what separates the novice from professionals.

The other problem is legal advice may not be put to the test for years after the advice is given. So a untrained person providing legal advice, may not realize the basic mistake for years.