Traders: Millions by the Minute - Part 2

Last night the BBC broadcast the second programme in the two-part series entitled Traders: Millions by the Minute. The first programme showed the professionals making millions by the minute, the second programme did not. Instead, with one exception, we got to see amateurs, losing. The fact is that the vast majority of home traders lose money. In financial trading it is easy to make money in a bull market, you buy and hold. In volatile markets there is the temptation to trade the swings, long and short and with disastrous results.

The problem for financial day traders is that most markets are rigged for the benefit of the professional élite. If the price isn't rigged then the market you trade in will be. Either the price is manipulated or the market you trade in is front run. Sports trading is a different matter but still, most sports traders are losing traders. Although sports rigging is commonplace and insider trading in horseracing is common too, the playing field is more level between the professional and amateur bettor. Buy and hold lasts only until the end of the event so most trades are very much day trades.

In last night's second programme we saw a house wife making a million but only on paper. When she started using her own money, she lost. A common occurence for traders. The psychological aspects of using your own money are vastly different to paper trading. You will take more risks on paper and may get a long run of success, reinforcing in your mind that you are a good trader. However, that may just be down to luck. After all, a risk taking loser on paper would probably not want to trade with their own money. We got to see a lucky loser on paper, lose it in reality.

When you start using your own money you might be more cautious and the reality of being just another trader snatching pennies in front of a steam roller might be too much to bear. You either give up because you are not going to be as rich as you thought or you take on more risk and blow up. We saw a team of retirees gambling their pensions on the stock market. They looked and acted professional but even they started to lose money. Again, they found it easy in a bull market and got out of their depth through over-trading a volatile market. A former antiques dealer knew all the terminology but he just couldn't trade. He was happy to lose money, year after year, to keep him from selling another chest of drawers.

There was a pair from Birmingham but I didn't know what to make of them. Maybe the BBC's usual multi-cult shenanigans in action. We saw the pair in their parents' house plotting to become 'trillionaires' then driving off in a flashy car to drum up business from black footballers. At least they had the sense to risk somebody else's money. At the end of the show, and unlike the other traders, we were not told how they were getting on financially. I suspect they are still living with their parents.

The only winner I could see was a rather narcissistic chap. He had made over a million pounds from day trading and had been given a multi-million pound fund from The City to trade. The narcissist probably has all the skills needed to be a winner; single-minded, self-absorbed with a touch of obsessive personality disorder. His house was immaculately clean, his body immaculately toned in a gym, his Porsche gleaming immaculately in the sunshine. A life dedicated to being good at everything and that includes making money.

BBC iPlayer - Traders: Millions by the Minute

Traders: Millions by the Minute

An interesting two-part documentary series on the BBC this evening, Traders: Millions by the Minute. The series shows us pit, hedge fund, electronic and algo traders at work in Chicago, New York, London and Amsterdam. Chicago's CBOT is still the scene of pit traders in the grain commodity markets. Although many exchanges throughout the world have gone electronic there is still some liquidity in certain commodities for face-to-face trading.

However, most trading is now done electronically, either by humans at a screen or algorithmically without human intervention. In London we see Amplify Trading, an electronic trading firm, training newcomers with mixed success. I am sure you will notice yourself amongst the trainees, some of whom are scared to trade and others trade too much. A few pass and are given jobs at Amplify or trade on their own accounts.

The hedge fund trader in New York was less interesting, as we mostly see her family and philanthropic life rather than her work. The high-frequency trading in Amsterdam could have been more interesting but a computer trading at the micro-second level probably doesn't make for good television.

All-in-all a fascinating view of people in the world of trading and maybe inspiring to sports traders too. Part two is next week and will concentrate on day traders working from home, which will probably be of more interest to sports traders. For my readers in the UK I provide a link below to the BBC iPlayer where the series will soon be available to view. In good time I am sure the series will find its way to YouTube and shhhh... BitTorrent, as does everything, eventually.

BBC iPlayer - Traders: Millions by the Minute

Is it time to turn to a bot?

A bot is a virtual robot, which performs simple repetitive tasks for you. Bots are used in trading to perform trades without human interaction. Whatever online task a bot does, if the task is simple and predicatable then you should be using bots for all of those online tasks and not just in trading.

Time means money and wasting time doing things yourself will cost you money in the long run. Bots increase your productivity. When should you decide to use a bot for your trading? Ask yourself this simple question, "Is my trading algorithmic, predicatable and profitable?" If you can answer 'yes' then you should be delegating tasks to a bot. If your answer is 'no' then you might consider a halt to your trading for obvious reasons.

Why trade one market at a time when one or more bots can be looking at all markets simultaneously? This is called scaling, through which you earn money from multiple sources in parallel. Wealthy people have scalable jobs, the poor do not. Not only that but you are diversifying your risk, which can only be a good thing.

If you find yourself doing the same thing when successfully manual trading then you should have a bot trading for you so that you can earn money elsewhere. If the entry and exit points for your trades are profitable and can be described algorithmically then you should give those trades to a bot whilst you design an entry/exit pair for another market

There is commercial bot software out there but for sports trading I find most of the software useless and so I code my own bots. But do have a trial run with the commercial software and see what it can do for you and what it cannot. Build a list of requirements and see what is out there that will satisfy those requirements. If you can't find what you are looking for then either code your own or pay someone to do it for you. An investment that is sure to pay you a handsome return.

Betfair, a form of dark pool?

I recently made a comment on Bet Angel's forum as to whether or not Betfair's exchange is a form of dark pool. There is much talk on the forum with people in disbelief that trades seem to go against them the moment they send their order to the Betfair exchange.

A dark pool is a term from finance. Such a pool is an exchange for the anonymous trading of stock. There is no market depth feed in such pools so traders are in the dark as to what volume is available and at what price stock is traded in the pool. Also, the details of who is trading what with whom is kept secret. Such dark pools led to the creation of high-frequency trading (HFT) firms who flash dark pools with small orders to judge market sentiment before sending in larger stealth orders divided up into smaller chunks, distributed over many pools.

HFT firms act as middle-men, grabbing stock just as a large institutional order is about to be traded and then selling to the institution at an inflated price, much to the consternation of many in the financial world. Think about it, your pension is being eroded everytime this happens. Your pension fund is being built with over-priced stock! Michael Lewis has written an excellent book called Flash Boys on HFT scandals. In many ways Betfair's exchange has elements of a dark pool within it. Trades are anonymous so there is no way of telling if two bots controlled by the one source (but on separate accounts) are trading with each other to create the illusion of liquidity.

More often than not during morning trading an apparent trend can be traded upon in a Betfair market only to see the trend reverse immediately, orders cleared around the spread and bots sending in new orders to reverse the weight of money, sending the trend in the opposite direction. It would appear that for most of the lifespan of a horse race's market (with the expcetion of the last few minutes) it's just a battle of the bots, herding humans and less able bots on a merry-go-round of ups and downs, false momentum and illusory liquidity.

For myself, as a researcher in computational finance and bot designer, this is a challenge, but for others trying to make a few pounds of profit, heartbreak. In a post yesterday I showed that Betfair is running a bot in tandem with its sports book. What if it has other bots in the exchanges too? Bots with no other purpose other than to fleece its customers?

After all, if Betfair is now a bookmaker then it will have a rough idea of where odds are going. Reason enough to use bots to lead unfortunate traders on a dance to penury. With people beginning to see sports betting as an asset class (link requires free registration) like stocks and commodities then it is inevitable that larger concerns start moving into sports betting and bring with them their nefarious ways from the financial world.

Betfair runs bots too

If you trade on Betfair's horse racing markets then you might see an odd price grouping somewhat below the current lay price for all the runners. For the favourite the offering will be £400 and there will be three such lays grouped with a two tick gap, sometimes three ticks or maybe more. The image on the left shows two £400s and one that has merged with another lay of £149 to give £549 at a price of 1.74.

I noticed this awhile ago and I wanted to try and understand what was going on. Obviously it's a bot but whose and why? I noticed that there are never back bets of a similar type so I assumed it was part of a 'lay only' operation. That means that it has to be a bookmaker's bot. I call the bot the '400 Bot' due to the volume offered for the favourite at each price. For other horses in the market the volume offered will be lower.

Occasionally, the group of three offerings will drop as the price on a runner drops dramatically. Sometimes some of the topmost lay will be eaten into before the group moves further down the ladder. In anticipation of a further price drop? 

I suspected that Betfair is running the bot. After all, Betfair now has a traditional bookmaking (sports book) operation in tandem with the exchange. Today, I watched the price on Betfair's sports book for the favourite in a race and just as the price on the book was lowered then so too the three lots of 400 dropped.

What the purpose of this is, I do not know. It can only be an exercise in getting to the front of the order book at those three particular prices. Nearer the off it is hard to get to the front of the order book so it is better to do it earlier in the day. The groups of three are a little lower than the book price and naturally enough the book price is considerably lower than the exchange price. It's probably some sort of risk management device.

Whether this knowledge can be used to my advantage remains to be seen (and thought of) but I doubt it. Betfair moves these three lay bets in reaction to market activity and is unlikely to be anything predictive.

Does your trading software force you to be a bad trader?

Elsewhere on my blog I have listed trading software that I have used in the past. These days I write my own software to get the precise market view that I want, along with a few metrics that I have designed myself and wish to keep secret.

Looking back, I sometimes wonder if third party software forces people to trade in a specific manner. Certainly, using a third party application "out of the box" won't get the best out of the software. You will have to examine all the features and set up the software for the precise market view that you are looking for.

Most third party software is set to update at a default update speed, typically at the 1 second rate. Such an update rate may force traders to become what is known as scalpers, who usually enter a market for a short period of time, typically less than a few minutes, before closing out their trade. Scalping is a valid trading methodology but I would only recommend to those with the right hardware and speed of Internet connection to help them get to the front of the market regularly.

Trading at the 1 second level is essentially noise trading (especially during the last ten minutes of a pre-race betting market) and it takes a lot of skill to determine a trend (if there is one) from noise. I wonder how many traders slow down the update rate for their software and investigate longer trend signals?

I know that some traders will have charts running at a different rate to get a trend but with a 1 second trading ladder next to it. Some of those traders might get spooked by a sudden movement on the ladder and trade out for a loss when there was no need to do so. This is a framing problem where you are looking at the same market but in two different time scales, which is going to be very confusing.

So, have a go at slowing down the update rate on your trading software. You might find a new niche in which to trade, away from the herd (or rather, the sheep and a small number of wolves).

Betfair API-NG progress report

It's been a number of months since I started porting from Betfair's old SOAP-based API-6 to the new JSON-based API-NG. In some ways it has been pleasurable and in other ways not.

The new API has permitted me to see Betfair's data in a new way. I have been able to quickly develop applications and get on with the task of analysing data for building new trading strategies. JSON permits easy collection and manipulation of data.

On the other hand Betfair itself has been none too helpful. Often, it's left to the users of the API to report bugs so that Betfair can improve the API. It seems to me that Betfair's developers have little knowledge of betting markets and don't appreciate the problems simple bugs create. The developers are just looking to solve the problem of disseminating data with a top down approach. They have no understanding of a trader's requirements for clean data.

I have discovered that often there are duplicate horses in the listMarketCatalogue method. An easy bug to get around as you can handle the exception when building a database quite easily. However, it's the way that Betfair handled my bug report that left me far from impressed. First they denied the bug. After I sent them evidence in a JSON string containing duplicates they said they would fix the bug. That was many months ago and the bug has still not been sorted out.

Then I found that overseas races were being added to Betfiar's database of UK races causing races from the United States, Hong Kong and Dubai to be added to my bot. Again a crude work-around was needed.

Now I find that the totalMatched volume count for runners never equals the actual colume for each price traded and often the totalMatched increases with no apparent volume increase for any price.

These bugs leave one disconcerted as to whether the data is accurate or not. A trader lives by the accuaracy of their data. If Betfair is not feeding the right data then a trader could end up in a lot of bother.

All in all, I feel that the new API is very promising indeed. It allows me to do so much more than I was able to do in the past. My productivity rate has increased greatly but I still have a suspicion in the back of mind that something could go wrong at any moment.