Friday, August 06, 2010

What happened on the stock market during the May 6 "flash crash?"

Why did the Wall Street stock market suddenly go crazy and drop out of bed very suddenly on May 6th? That's still a question that has no clear answer. Tom Lauricells and Scott Patterson provide an update in the Wall Street Journal. This article is a progress report telling us the current status of the investigation into the flash crash. What isn't known is a lot more than what is known, and a lot of investors are quite leery of investing in stocks at present as a result. But what is known? The clearest thing is that all of a sudden the Dow Jones Industrial Average went into a sudden decline which was more rapid than ever before. Hundreds of stocks suddenly lost nearly all their value for no apparent reason.

It's not that there were no warnings at all. Fund managers were cutting back on buying stocks even before the crash because the market was acting strange. But there was no indication what the strange happenings meant.

Some new details include:
Stock-price data from the New York Stock Exchange's electronic-trading arm, Arca, were so slow that at least three other exchanges simply cut it off from trading. Pricing information became so erratic that at one point shares of Apple Inc. traded at nearly $100,000 apiece. And computer-driven trading models used by many big investors, apparently responding to the same market signals, rushed for the exits at the same time.


Todd Sandoz, co-head of equities in the Americas at Credit Suisse in New York, kept track as clients reduced risk in their portfolios. One way they did it was through trades that would profit if the Standard & Poor's 500-stock index fell: They sold short, or bet against, futures contracts linked to that index. They did the same with exchange-traded funds, which track baskets of stocks.

Those kinds of trades can send waves through the market. Brokers on the other side of the trades often hedge their own positions by selling the stocks contained in the index. That morning, Mr. Sandoz heard from his traders that there were relatively few buyers and sellers for some individual stocks—a sign that the market might not be able to smoothly handle big index trades.

The market was especially vulnerable because of the trading pullback identified by his colleague Mr. Vasan. The hedge funds that had been pulling back for several days—specialists in a strategy called statistical arbitrage—normally trade so much stock that they are a key source of market liquidity.

At about 2 p.m., as protests in Athens over the Greek debt crisis turned violent, the euro fell sharply, especially against the yen. The euro-yen exchange rate is watched widely by traders, with the yen seen as a safe-haven currency, the euro a proxy for riskier investments.

The euro's fall triggered concerns that a rush out of stocks was in the works. At Chicago hedge fund Sharmac Capital Management LLC, trader Jason Roney noticed the drop. "Something is wrong, look out!" he recalls shouting to his trading desk. He started shorting S&P 500 futures.

Traders across Wall Street were making similar moves, many driven by computer models that have become standard tools at banks, hedge funds and mutual funds.

Fund managers at Waddell & Reed Financial Inc. in Overland Park, Kan., moved to hedge their U.S. stock holdings, which total more than $7 billion, by betting that the S&P 500 would fall. Waddell decided on a large short sale of futures contracts known as E-minis, which mimic movement of the S&P 500. As Waddell's computers began parceling out the trade, other investors also were trying to hedge their portfolios, so trading volume in E-minis shot up to six times the usual volume.

But liquidity, the ability to buy or sell easily, was drying up. Between about 2:35 and 2:45, the six "market-making" firms that were most active that afternoon in E-mini trading—they step in as buyers or sellers on many trades—cut back their trading. Some pulled out altogether.

As a result, traders say, the big Waddell trade accelerated the sell-off. Waddell says it did not intend to "disrupt" the market.

Computers started to groan under the weight of the orders and slow by fractions of a second. It became difficult for exchanges and investors to keep track of prices.

In recent years, due in part to rules instituted by the Securities and Exchange Commission in 2007, the stock market has been opened to numerous trading venues and has evolved into a high-speed network. The rules stipulate that when an investor trades a stock, the order is routed to the venue with the best price.

On the afternoon of May 6, it was difficult for traders to trust the information they were getting, and for buyers and sellers to find each other. Nasdaq OMX Group Inc. operations personnel noticed problems with orders it had routed to Arca, the electronic trading platform of the NYSE, which handles about 12% of U.S. stock-trading volume. It was taking Arca longer to acknowledge receiving some orders. Orders for Nasdaq-listed stocks such as Apple and Inc. were hitting lags of two seconds or more on Arca—an eternity in today's markets.

Trading in Apple became especially volatile. At 2:40, its stock began falling swiftly, losing 16% in six minutes. Because Apple is a component of several indexes, weakness in the stock helped drag down the broader market.

Concerned about the impact of the delay on orders routed to Arca, Nasdaq officials used a tool called "self help," designed to prevent problems at one exchange from spreading to others. At 2:36:59, Nasdaq stopped routing orders to Arca. Other exchanges, including Chicago Board Options Exchange and BATS Global Markets, an electronic exchange near Kansas City, Mo., did the same.

The NYSE says Arca had "minor delays" on a computer server during the period, but says the problems were not significant and didn't add to the market's broader problems.

Computer systems at big brokerage firms were straining to keep up with the volume. Dark pools, trading venues that match buyers and sellers away from the major exchanges, had trouble getting accurate information. Some temporarily shut down.

2:40 p.m., Dow down 415 points

High-frequency-trading firms, which account for some two-thirds of U.S. stock-trading volume, were having their own problems. Their strategies often involve buying and selling stocks within microseconds—or one-millionth of a second. The market's plunge, along with discrepancies in data feeds from exchanges, scrambled their computer-trading systems.

With the Dow industrials down about 500 points, Tradebot Systems Inc., a Kansas City high-speed trading firm that says it can account for up to 5% of daily volume, pulled out. Other such firms did the same.

The roar on the floor of the Chicago Mercantile Exchange was deafening as the sell-off accelerated. The E-mini contract suddenly fell a massive 12.75 points in half a second, triggering a CME circuit-breaker that stopped trading for five seconds. The pause gave computerized futures-trading systems time to stabilize.

On the floor of the NYSE, the fast declines in some stocks were triggering brief slowdowns in trading, known as "liquidity replenishment points," to allow floor traders to step in and restore order. Other exchanges, such as the Nasdaq, didn't slow trading.

Among the problems this caused were "crossed" markets, where offers to buy were at prices higher than orders to sell. Around 2:46, for example, an investor offered to buy Apple for about $218, while another was willing to sell it for about $202. Such nonsensical quotes sent warning signals to computer systems and gave traders yet another reason to pull back.

Stocks everywhere started to collapse. Apple lost more than $23 a share, or 10%, between 2:44 and 2:46. Procter & Gamble Co., which had been trading around $61.50, saw huge sell orders hit the NYSE, and the exchange briefly slowed trading in the stock. By 2:47, the market for P&G was in chaos, with orders to buy from NYSE, Nasdaq and the BATS scattered from $39.89 to $44.24. The basic function of the stock market— bringing together buyers and sellers in an orderly fashion—had broken down.

Trades flickered across computer screens that made no sense. Shortly after 2:47, shares of Accenture PLC dropped in seconds from about $40 to one penny, then rebounded just as quickly. The explanation surfaced later: Market-making firms—regular buyers and sellers of certain stocks—have to maintain quotes at all times. To fulfill the requirement, they use "stub quotes," dummy quotes they never expect to be executed. But in the absence of buyers on May 6, computers matched automated sell orders with the dummy quotes.

Rumors swirled about of an erroneous "fat-finger" order by a trader at Citigroup Inc.—that the trader mistakenly entered extra zeros, turning millions into billions. Citigroup and regulators later said such an errant trade did not appear to have taken place.

But the rumor helped stabilize the market. If the massive decline was the result of a mistake and not some terrible news, that meant there were bargains to be had.

At 2:47, the Dow reached its nadir, down 998.50 points. As trading resumed in the futures market, buyers flooded in and prices started to rebound.

Within one minute, the Dow reclaimed 300 points.

But the problems weren't over. Exchange-traded funds, or ETFs, are baskets of securities that trade like a stock. NYSE's Arca is usually home to 30% of ETF trading. When other exchanges stopped routing orders to Arca, the normal flow of ETF buyers and sellers was disrupted.

Two big hedge-fund and trading firms, D.E. Shaw Group and Citadel Investment Group, detected problems in Arca's ETF computer feed. Citadel asked customers to route orders elsewhere. NYSE officials say they found no problems with Arca's ETF platform on May 6.

Some of the biggest ETF traders are firms that try to profit from discrepancies between prices of ETFs and the stocks that they track. But as questions mounted about pricing of individual stocks, these firms pulled back from trading. This hurt small investors who had placed "stop-loss orders," aimed at protecting against big losses by automatically selling once prices fell below a certain level. Those orders hit the market when there were virtually no buyers to be found.

At 3:01, Nasdaq once again began routing orders to NYSE's Arca.

Executives from several major exchanges joined a conference call to discuss, among other things, whether to declare some trades erroneous. After considerable debate, they decided to cancel trades in stocks and ETFs that had fallen or risen 60% or more.

In the final hour, trading remained erratic. At one point, Apple traded for nearly $100,000 a share on Arca, according to NYSE officials, after a buy order for 5,000 shares entered the market and only 4,105 shares were available. When Arca's computers saw that no more shares were available to sell, the system automatically assigned a default price of $99,999 to the remaining 895 shares. Those trades later were cancelled.

4 p.m., Dow closes down 342 points
This looks like a system that has grown too large for anyone to oversee, understand or in times of trouble, react to. It requires the high speed arbitrage traders to operate to create liquidity, but with the various markets getting out of synch with each other the possibility that one market would sell at a price lower than another was buying at can cause investors to back off and do nothing. The result can be a lack of stocks to buy or sell in specific markets and price reports that are delayed from one market can completely disrupt the functioning of the market.

This set of problems will be compounded by computerized trading when the trading data goes gives signals the computer is not programmed to react to. Since much of this is arbitrage trading which no human being looks at except in retrospect, the only thing the program can be programmed to do is just get out of the market. This is going to give other people in the market unpredictable signals. So everyone is going to hedge their investments all at once. Such hedge trading will then slow down and again provide signals that are fed back into the market causing more unpredictable behavior.

Regulators are already doing a few things to change the system, but since the details of what happened are not yet known the efficacy of the new regulations is not known.
New circuit breakers, now in pilot mode, require a five-minute trading halt on S&P 500 stocks that move more than 10% within five minutes. These "collars" could help keep prices from suddenly cascading.

But some forces behind the flash crash seem beyond the reach of regulators. Exchanges are unlikely to be able to prevent high-frequency trading firms or statistical-arbitrage firms from bailing out of the market en masse.
So there is not any real reason to think that there might be another flash crash any day.

That's the kind of uncertainty the markets hate.

Addendum 08/07/2010 3:08 pm
Well, well. I am going to thank Paul Hinds for sending me this link to an explanation of the flash crash by Nanex. This link is the text and back the main page at this link are the graphs. Here is the key part of Nanex's analysis:
There are 9 exchanges that route orders to NYSE listed stocks: NYSE, Nasdaq, ISE, BATS, Boston, Cincinnati (National Stock Exchange), CBOE, ARCA and Chicago. Each exchange submits a bid and/or offer price for each stock they wish to make a market in. The highest bid price becomes the National Best Bid and the lowest offer price becomes the National Best Ask. Exchanges compete, fiercely at times, to become the best bid or offer because that is where orders will be sent for execution. Exchanges also go to great lengths to ensure they avoid crossing other exchanges (bidding higher than others are offering, or offering lower than others are bidding), because if they do, many High Frequency Trading (HFT) systems will immediately execute a buy/offer and capture an immediate profit equal to the difference. Today, it is very rare to see markets crossed in stocks for longer than a few milliseconds.

Beginning at 14:42:46, bids from the NYSE started crossing above the National Best Ask prices in about 100 NYSE listed stocks, expanding to over 250 stocks within 2 minutes (See Part 1, Chart 1-b). Detailed inspection indicates NYSE quote prices started lagging quotes from other markets; their bid prices were not dropping fast enough to keep below the other exchange's falling offer prices. The time stamp on NYSE quotes matched that of other exchange quotes, indicating they were valid and fresh.

With NYSE's bid above the offer price at other exchanges, HFT systems would attempt to profit from this difference by sending buy orders to other exchanges and sell orders to the NYSE. Hence the NYSE would bear the brunt of the selling pressure for those stocks that were crossed.

Minutes later, trade executions from the NYSE started coming through in many stocks at prices slightly below the National Best Bid, setting new lows for the day. (See Part 1, Chart 2). This is unexpected, the execution prices from the NYSE should have been higher -- matching NYSE's higher bid price, unless the time stamps are not reflecting when quotes and trades actually occurred.

If the quotes sent from the NYSE were stuck in a queue for transmission and time stamped ONLY when exiting the queue, then all data inconsistencies disappear and things make sense. In fact, this very situation occurred on 2 separate occasions at October 30, 2009, and again on January 28, 2010. (See Part 2, Previous Occurrences).

Charting the bid/ask cross counts for those two days reveals the same pattern as 5/6! Looking at the details of the trade and quote data on those days shows the same time stamp/price inconsistencies. The NYSE stated that during the same intervals, they were experiencing delays in disseminating their quotes!

In summary, quotes from NYSE began to queue, but because they were time stamped after exiting the queue, the delay was undetectable to systems processing those quotes. On 05/06/2010 the delay was enough to cause the NYSE bid to be just slightly higher than the lowest offer price from competing exchanges, but small enough that is was difficult to detect (See Part 3, The Evidence). This caused sell order flow to route to NYSE -- thus removing any buying power that existed on other exchanges. When these sell orders arrived at NYSE, the actual bid price was lower because new lower quotes were still waiting to exit a queue for dissemination.
The key to the problem seems to me to be the various other markets cross-linked to the New York Stock Exchange and the problems caused when there was a delay in some but not all of the linkages. For those of you not familiar with computers, that is what the statement "If the quotes sent from the NYSE were stuck in a queue for transmission " means. Quotes that buyers and sellers depend on to make buy and sell decisions were lined up and not moving (stuck in the queue) making the buy and sell decisions made outside the New York Stock Exchange based on bad (delayed) data.

Computer trading depends on this instant transmission of data to operate. Because the computers supposedly have the very latest quotes sooner than other buyers and sellers in the market they can be programmed to buy or sell sooner than anyone else. That's how the computers perform the arbitrage function between different markets and supposedly keep prices on the different markets in synch within milliseconds.

I have NOT dug into all this data to make sure of its accuracy and guaranteed that each of the analysts or groups I am quoting performed good analysis. All I am saying is that the reports I have posted here make sense to me.

This is very similar to the set of cascading power problems that took down the Northeast power grid in 2003. Both appear to be problems when a massive interconnected system suddenly gets unbalanced demands on part of the system that are fed back into the overall system and causing a cascade failure. The initial cause this time is of course different from that of the power failure, but the problems caused when one part of the system did not reacting promptly created strains on other parts of the system. When an portion of the system failed it then increased the strains thrown onto the rest of the system. The system here is the group of 9 exchanges that route orders to NYSE listed stocks: NYSE, Nasdaq, ISE, BATS, Boston, Cincinnati (National Stock Exchange), CBOE, ARCA and Chicago. In this case, the New York Stock Exchange was the recipient of massive number of buy and sell orders caused by delays in quotes sent to the computers doing computerized trading.

The problem seems to me to be one of unbalanced workloads. If somehow regulations are put into place that prevent delayed quotes from causing outlandish workloads (orders to buy and sell instantly) on a single part of the system, then I suspect that such regulations will only solve the problem for delays in communications between the various markets. That is not to say that there aren't other possible causes of unbalanced loads screaming through the system in the future.

No comments: