Wednesday, October 8, 2008

Truthiness, quants, software, glitches, experts, and steps towards solutions...

Here is an example of something I think touches upon the usefulness of the "truthiness" concept.

Although the following article was written in 2003, it holds up well in 2008.
The seeds of truthiness were planted well before Stephen Colbert came up with the word:

What the Quants Don't Learn in College
(Emanual Derman, Risk Magazine-Trends July 2003/Volume 16/No7

"The only universally applicable law is that of approximate similarity, which states that the best estimate of the unknown market value of a security is the price of another security that's closely similar to it. You need to find (or invent) a model to establish the similarity between two securities by demonstrating the equivalence of their future payouts under a wide range of circumstances....

Suspend disbelief
Although all you have is the limited power of this simple law, you must take your model of similarity seriously. Temporarily, like a fiction reader, you must suspend disbelief in your model. Then, when it's complete, remind yourself that economics and valuation involve the behaviour of people, and think hard about what could go wrong...

On Wall Street, no-one knows what the correct model is, but they go ahead and price and trade anyhow. It's a bit like the trial in Alice in Wonderland...Academics often over emphasise models, but much of the success of a model depends on software engineering....You need live market data, historical time series, databases, input screens and calibration. As a result, for every financial engineer who works on a model you may need three or four more software engineers to make it usable.."

I wonder if any Wall Street companies hired additional software engineers. From what has transpired over the past months, if they did, they were not the right software engineers!

UPDATE: Please read the comments to this post for clarification of Calyx software errors by a representative of the company. The passage below is my original post.

To take a closer look at how software problems might have played a small part in the current situation, I chose to browse the Calyx Software website. Calyx provides software that is used by Freddie Mac and Fannie Mae. There is a treasure trove of information about the quality of this companies mortgage processing software on its support pages.

From my armchair analysis of the types of errors reported, it seems that it is not difficult for mortgage brokers to unknowingly make errors when determining a potential borrower's risk. The Freddie Mac Loan Prospector looks like it was rolled out before important errors were discovered. In my opinion, it was designed without the capacity to prevent critical errors.

Because the Calyx support page lists many of the problems as common, it is likely that the Freddie Mac Loan Prospector was designed with a lower level of error prevention than expected for this sort of transaction. To fix this problem after the fact, the support pages offer help solutions, but some of the solutions are quite complex.

Common Problems with the Freddie Mac Loan Prospector
Second home is not included in ratios

Credit Agency Missing from the Freddie Mac Loan Prospector
Credit Agency or Lender is Missing from Point

Little things that waste time

Experts depend on accurate, reliable data in order to make effective decisions, but our current systems did not do the job. Even so, it is worth keeping track of what the experts have to say about the crisis:

Must-watch video
If you have about an hour, watch this discussion between Charlie Rose, Floyd Norris, Mohamed El-Erian, Gretchen Morgenson, and Nouriel Roubini, which aired at the time of the Fannie Mae and Freddie Mac "bailout" decision.

According to Floyd Norris, "The senior managers of the banks assumed that the whizzes under them had their financial models which proved that they did not have any value at risk, and had nothing to worry about, and they believed all of this. And now what they believed looks like nonsense, and you wonder why they did. And that left those banks very exposed... In a lot of cases, they thought they were making money, but they really weren't."

The one thing that is clear to me is that our current economic models no longer function. We can't put the blame on the quants, or the politicians, or the greedy Wall Street leaders. We can't put the blame on new homeowners with low-incomes, or pressured lenders.

Right now, there is a high level of uncertainty, and we do not have anything tangible that ensures that things will be OK. This problem can not be solved quickly. We need better models that can support effective economic decision-making on Wall Street, Main Street, and everywhere in between.

The problem of preventing future economic disasters won't be solved by politicians, government officials, economists, and Wall Street leaders. The general public is strongly against the rescue bailout. It simply is too difficult to trust those we've blamed.

My idea for a collaborative interactive time-line is just one small step (see sidebar).

We need to stretch our thinking and cast a wide net. This will require an interdisciplinary approach, and include people from a variety of disciplines, who are untainted by monetary scandals and have innovative minds, who care about future generations, and who believe strongly that in a democratic society, all citizens must have access to accurate, understandable information in order to make effective decisions - in all aspects of their lives.

Who might these people be? University researchers, practitioners in the workplace, graduate students, soccer moms, grandpas...with experience in areas such as finance, psychology, history, geography, urban planning, business, economics, banking, sociology, computer science, human-computer interaction, information visualization, graphic arts, & communications.

It is up to everyone to wake up and take action in some way.


Bryan Telford - Calyx Software said...

I work at Calyx. I just wanted to correct a couple things.

Firstly, the software is used by mortgage brokers, not Freddie Mac. The program provides a link for brokers to send loans to Freddie Mac for underwriting.

Second, the errors you had referred to regarding the Freddie Mac interface would not have contributed to bad loans being sold to Freddie Mac. They were the type of "small annoyance" errors that users were able to find workarounds for within the Freddie Mac underwriting website. Also, when those errors happened, Freddie Mac's system would not be able to score the loan, so the user would have to fix it on the Freddie Mac website.

I just had to comment so that people would have another viewpoint. Thanks.

Lynn V. Marentette said...

Thanks, Bryan. Sorry for the misunderstanding.

I wrote the post at the same time I was struggling with error-prone software at work. I wondered if people who process mortgages were faced with similar problems, and that is how I found the Calyx website.

I appreciate elegant, user-friendly software, and I know that minor annoyances can waste precious time, over time.

I will update my passage and add your comment.