When, in a former life, I used to be Systems Analyst/Administrator, I used to marvel at platforms like HP Openview, Spectrum and SunView for tracking systems processes, network connections and physical devices (IE: printers, system server environmental faults).

While Platforms to monitor an entire companies global infrastructure were difficult and costly to set up, the generally did and do work well for telling you the printer downstairs is out of paper or toner and send a technician over to fix it.   On the Dashboards of these platforms you can abstract the representation of complexity that is your environment to a bunch of color coded symbols that can be expanded and change state when there's a connectivity or environmental problem; that's a good thing and quite necessary or else troubleshooting problems would be like finding a needle in a haystack.

But with the need to abstract data, including web data, comes the responsibility and knowledge of what lies behind the abstracting; as you abstract your masses of data from the details and create symbols and abstract derivatives - it's easy to lose sight of what your actually doing - and that's what happened on Wall Street with the Sub-Prime crisis that is quickly mushrooming, perhaps to spread across the rest of the economy (or not).

I see it all the time, even in Corporations I work in; well meaning project managers work with sizing requirements for data analytics, not really understanding the reports needed, looking only at budgets and data capacity specs.  Out of that mess comes one place with 16+ datamarts that has data that overlaps and does not agree with each other or a deficiency in N-Cube filters that makes high end marketing analysis (for what we have the analytics in place in the first place) almost impossible to accomplish.

In the case of Wall Street - the warning signs in "How Missed Signs Contributed to a Mortgage Meltdown" were usually missed by many (but not all) because:

"….But the precipitous fall in the value of mortgage-backed bonds was spurred by a mad rush by so many holders to dump them simultaneously. That is because they might not have understood what they were buying in the first place, instead focusing only on the bonds’ higher yields. Unlike Mr. Melcher, they had not considered the circumstances under which the risky mortgages they essentially owned had been granted.

"…Buyers didn’t fully understand what they were getting,” said Rajiv Sobti, a portfolio manager at Proxima Alfa Investments, a New York hedge fund. In Wall Street parlance, Mr. Sobti added, “They were sold, not bought. The actual buyers were often not mortgage specialists, but generalists who looked at these bonds as a way of earning higher yields.”

The people buying up Mortgage Backed Securities did not actually understand what they were buying - they didn't really care - what these securities were based on was dis-enfranchised, abstracted, from what the reality of what they were. According to Paul Krugman:

"..Everyone knows now about the explosion in subprime loans, which allowed people without the usual financial qualifications to buy houses, and the eagerness with which investors bought securities backed by these loans. But investors also snapped up high-yield corporate debt, aka junk bonds, driving the spread between junk bond yields and U.S. Treasuries down to record lows."

From another posting Krugman is saying:

"…What’s been happening in financial markets over the past few days is something that truly scares monetary economists: liquidity has dried up. That is, markets in stuff that is normally traded all the time — in particular, financial instruments backed by home mortgages — have shut down because there are no buyers."

"..But when liquidity dries up, the normal tools of policy lose much of their effectiveness. Reducing the cost of money doesn’t do much for borrowers if nobody is willing to make loans. Ensuring that banks have plenty of cash doesn’t do much if the cash stays in the banks’ vaults.

There are other, more exotic things the Fed and, more important, the executive branch of the U.S. government could do to contain the crisis if the standard policies don’t work. But for a variety of reasons, not least the current administration’s record of incompetence, we’d really rather not go there.

Such Abstraction of data, which allows these error, I believe, to happen, while often needed to act on information in mass, can not be allowed to go forward unchecked, and that's what Wall Street did (because they wanted to and did not give a dam - they only care about profits .. why should the care how it comes about?).

"….The blame game is already beginning. The spotlight will focus first on rating agencies like Moody’s and Standard & Poor’s, because the mortgage-backed bonds that plunged in value in recent weeks were highly rated by these agencies until they downgraded billions worth of them in July."

You would expect the rating agencies to understand and care about what Sub-Prime Mortgages were - and the dangers of bundling them together, abstracting them.  Remember Soylent Green, how people began eating the remains of dead people that had been made into a nutrition bar? 

People often don't care what they're buying/eating/trading -as long as it performs.

In the 1973 movie Soylant Green:

"….The year is 2022. New York City has become overpopulated with 40 million people and pollution has caused the temperature to be risen and all natural resources have been destroyed, leaving 40 million people starving. The Soylent Company has create a new food product, Soylent Green. In the overpopulated and polluted New York City, police detective Thorn is assigned to investigate the brutal murder of an corporate official of the Soylent company, William R. Simonson. Thorn's investigation into Simonson's murder leads him to uncover a conspiracy in the Soylant company and the Soylent Green food product itself, where Thorn uncovers the horrible truth about Soylent Green. "

But I think we have to care where Mortgage Backed Securites come fr
om, or Soylent Green is made of, so to speak, if want to avoid the recurring mistakes of abstracting data, information, financials, so we can put them in dashboards, in financial analytics programs - somewhere - someone needs to understand what that abstraction, that symbol actually represents - otherwise, we'll be doomed to repeat problems like this over and over (which, is, after all, what seems to be happening).

"…Along with chief executives like Mr. Mozilo, central bankers also apparently failed to anticipate the threat posed by subprime debt. In a May 17 speech in Chicago, Ben S. Bernanke, the chairman of the Federal Reserve, acknowledged the “recent sharp increases in subprime mortgage loan delinquencies.”

But he said, “we do not expect significant spillovers from the subprime market to the rest of the economy or to the financial system.”

That same month, a former Federal Reserve governor, Edward M. Gramlich, was preparing to publish a book titled, “Subprime Mortgages: America’s Latest Boom and Bust.” Even earlier, in April, the usually bullish National Association of Realtors said that home prices across the country would decline in 2007 because of the widening fallout from subprime loans. And in March, the head of the Federal Deposit Insurance Corporation, a banking regulator, had urged Congress to create standards to protect borrowers with poor credit."

My basic feeling about all of this, The FED permitted this problem to develop since 2001 as it served a purpose of moving money around to help fuel the economy which is what Mr. Bush and The Republicans wanted (to pull the world out of recession) - that's why most who should have known better were not asking the hard questions - the alternatives at the time were worse - a tightening economy that would have depressed a housing market that will now get depressed anyway - plus spill into the rest of the economy later this fall.

But it all goes back to a fundamental quality of Analytics - when you abstract data into symbols, be it a bank of printers, a bunch of Server Processes, or a bunch of Mortgage backed loans packed as a Soylent Green Securities, you must know the details of how that abstraction, how that packaging came to be - else you will not be able to make good decisions on it, and this kind of debacle we're seeing on Wall street will repeat itself over and over in different forms.