"We risk becoming the best informed society that has ever died of ignorance"
- Rubén Blades

"You can't make up anything anymore. The world itself is a satire. All you're doing is recording it"
- Art Buchwald

"It's getting exciting now, two and one-half. Think of everything we've accomplished, man. Out these windows, we will view the collapse of financial history. One step closer to economic equilibrium"
- Tyler Durden

"It is your corrupt we claim. It is your evil that will be sought by us. With every breath, we shall hunt them down."
- Boondock Saints

Monday, February 13, 2012

February 13, 2012 Crude Oil Event

As was pointed out earlier, today was a shady day with exchanges misfiring in the morning, then the Crude Oil complex was halted in New York mid afternoon (more here).  Thanks to Nanex, we know what happened before the SEC can even warm up their Commodores.  What you see is a cycle that continues for 12 rounds, resending and adding to older quotes.  The blue line shows how quotes time stamped at their peak above 4 minutes were being sent through as real time, thanks to a likely malfunction of exchange queue memory functions.  Once this happened a chain reaction began as arbitrage algo's went ballistic on USO, SCO, and UCO etf's.

HFT Alert Pro enabled us to catch the price arbs in real-time, take notice of the areas within the red rectangles:

You can see the algo ran across NQEX, BATS, PACF, and PHIL.  Notice at 14:01:58 there are 10,260 quotes (there are are 11,260 but 1,000 quotes are needed for the program to take notice, so add that to 10,260 and we have 11,260). 

One minute chart highlighting the problem area

And the tick chart for the re-open at 3:15 New York Time

NYMEX Black Swan ~ The March 2012 Crude Oil Futures Quote Loop

On February 13, 2012, starting 13:59:57, quotes for crude oil began queuing. At 14:00:35, all of the queued quotes were sent at once. Again at 14:01:08 the same 38 second block of quotes sent earlier was sent again -- old timestamps and all plus a few new quotes. Again at 14:01:18, all quotes since 13:59:57 were sent again. This repeated 12 times.

From a programmers perspective, it looks like a system problem caused a blast of quotes that corrupted a memory queue causing the software to believe the queue was full all the time.

Tick chart of bid prices (red) along with quote age (blue).
Note that as the cycle repeats, it includes a few more quotes (the new quotes + those since 13:59:57). There are 500 quotes between time axis labels. 

500 millisecond chart of ETF U.S. Oil Fund (USO) showing massive quote traffic as it reacts to stale futures quotes.

500 millisecond chart of ETF U.S. Oil Fund (USO) showing massive trade executions in reaction to stale futures quotes.