Jeff Judy

Jeff's Thoughts - April 8, 2015

Do You Teach to the Test?

Standardized testing is a controversial topic in education these days.

On the one hand, there is a widely perceived need to hold teachers and schools more accountable for the results they produce. Long gone are the days when the full measure of a student's success is a grade, an A or a C or an F, that simply represents a teacher's judgment about what the student has learned.

That subjective system certainly has produced variable results, and standardized testing provides a way of comparing learning across students. It is a more objective approach, and it being applied ever more widely in the nation's schools.

But there are more and more educators and parents who question whether the standardized approach can go too far. They complain about too much time spent "teaching to the test", meaning that they are preparing students to pass the test, at the expense of other kinds of learning that could have broader benefits in their lives. They suggest that we may be producing better test takers, but not necessarily better students or even better people.

Now, there was a time when a credit decision was largely a judgment call. Sure, there were always analysis tools available to assist that judgment. (But then again, teachers always used testing as part of their student evaluation practices.)

These days, our analysis tools are much more powerful (and often quite similar across different institutions). With the widespread application of credit scoring, we have our own equivalent of "the standardized test." A potential borrower who clears a set threshold is "a good credit risk", much as a person who gets a high score on a standardized test must be "a good student".

Even in institutions that do not rely on credit scoring to make small business credit decisions, there are often key ratios that have fairly black-and-white thresholds that must be cleared.

When you have a threshold-based, highly automated system driving credit decisions, it is worth keeping a couple of cautions in mind.

The first is it that the "teachers" -- your credit staff -- learn to "teach to the test", and are rewarded for doing precisely that. I've heard loan officers and credit analysts talk about getting a customer's data "to flow", meaning that they are playing with the inputs to see if they can produce better cash flow results from the standard analysis. While it may not be possible to beat the system, it is certainly possible to play the system. Do you want your credit staff working to cross a threshold, or do you want them working to understand the customer?

The second problem is that good test takers are not always good learners or good people, and good credit scores do not always indicate good customers. Sure, a good score probably means they are capable of paying you back.

But will they? And will they be reasonable to work with, or will they demand inordinate attention? Will they be an enhancement to your portfolio, or a drain on your staff and resources?

Analytical tools help us avoid some of the mistakes we made back in the day when a more personal, subjective judgment carried most of the weight in a credit decision.

But they are not "turn key" solutions to your need to acquire good credit customers. Make sure your credit staff are focused on the customer, not just the score.