Traders: Millions by the Minute

An interesting two-part documentary series on the BBC this evening, Traders: Millions by the Minute. The series shows us pit, hedge fund, electronic and algo traders at work in Chicago, New York, London and Amsterdam.

Chicago's CBOT is still the scene of pit traders in the grain commodity markets. Although many exchanges throughout the world have gone electronic there is still some liquidity in certain commodities for face-to-face trading.

However, most trading is now done electronically, either by humans at a screen or algorithmically without human intervention. In London we see Amplify Trading, an electronic trading firm, training newcomers with mixed success. I am sure you will notice yourself amongst the trainees, some of whom are scared to trade and others trade too much. A few pass and are given jobs at Amplify or trade on their own accounts.

The hedge fund trader in New York was less interesting, as we mostly see her family and philanthropic life rather than her work. The high-frequency trading in Amsterdam could have been more interesting but a computer trading at the micro-second level probably doesn't make for good television.

All-in-all a fascinating view of people in the world of trading and maybe inspiring to sports traders too. Part two is next week and will concentrate on day traders working from home, which will probably be of more interest to sports traders.

For my readers in the UK I provide a link below to the BBC iPlayer where the series will soon be available to view. In good time I am sure the series will find its way to YouTube and shhhh... BitTorrent, as does everything, eventually.

BBC iPlayer - Traders: Millions by the Minute

Is it time to turn to a bot?

A bot is a virtual robot, which performs simple repetitive tasks for you. Bots are used in trading to perform trades without human interaction.

Whatever online task a bot does, if the task is simple and predicatable then you should be using bots for all of those online tasks and not just in trading.

I recently downloaded SleepBot for my Android phone and it has already made a difference to the quality of my sleep. I have so much more energy when I wake up at 6AM to start trading.

Time means money and wasting time doing things yourself will cost you money in the long run. Bots increase your productivity.

When should you decide to use a bot for your trading? Ask yourself this simple question, "Is my trading algorithmic, predicatable and profitable?" If you can answer 'yes' then you should be delegating tasks to a bot. If your answer is 'no' then you might consider a halt to your trading for obvious reasons.

Why sit in a chair trading one market at a time when one or more bots can be looking at all markets simultaneously? This is called scaling, through which you earn money from multiple sources in parallel. Wealthy people have scalable jobs, the poor do not. Not only that but you are diversifying your risk, which can only be a good thing.

If you find yourself doing the same thing when successfully manual trading then you should have a bot trading for you so that you can earn money elsewhere. If the entry and exit points for your trades are profitable and can be described algorithmically then you should give those trades to a bot whilst you design an entry/exit pair for another market

There is commercial bot software out there but for sports trading I find most of the software useless and so I code my own bots. But do have a trial run with the commercial software and see what it can do for you and what it cannot. Build a list of requirements and see what is out there that will satisfy those requirements. If you can't find what you are looking for then either code your own or pay someone to do it for you. An investment that is sure to pay you a handsome return.

I shall be covering bot trading strategies in a future post.

Betfair, a form of dark pool?

I recently made a comment on Bet Angel's forum as to whether or not Betfair's exchange is a form of dark pool. There is much talk on the forum with people in disbelief that trades seem to go against them the moment they send their order to the Betfair exchange.

A dark pool is a term from finance. Such a pool is a form of exchange for the anonymous trading of stock. There is no market depth feed in such pools so traders are in the dark as to what volume and at what price stock is traded in the pool. Also, the details of who is trading what with whom is kept secret.

Such dark pools lead to the creation of high-frequency trading (HFT) firms who flash dark pools with small orders to judge market sentiment before sending in larger stealth orders divided up into smaller chunks, distributed over many pools.

Also, HFT firms act as middle-men by grabbing stock just as a large institutional order is about to be traded and then selling to the institution at an inflated price, much to the consternation of many in the financial world. Think about it, your pension is being eroded everytime this happens. Your pension fund is built with over-priced stock! Michael Lewis has written an excellent book called Flash Boys on HFT scandals.

In many ways Betfair's exchange has elements of a dark pool within it. Trades are anonymous so there is no way of telling if two bots controlled by the one source (but on separate accounts) are trading with each other to create the illusion of liquidity.

More often than not an apparent trend during morning trading on the horse racing markets can be traded upon only to see the trend reverse immediately, orders cleared around the spread and bots sending in instantaneous new orders to reverse the weight of money and send the trend in the opposite direction.

It would appear that for most of the lifespan of a horse race's market on an exchange is just a battle of the bots, herding humans and less able bots on a merry-go-round of ups and downs, false momentum and illusory liquidity.

For myself, as a researcher in computational finance and bot designer, this is a challenge but for others trying to make a few pounds of profit, heartbreak. In a post yesterday I showed that Betfair is running a bot in tandem with its sports book. What if it has other bots in the exchanges too? Bots with no other purpose other than to fleece its customers?

After all, if Betfair is now a bookmaker and will have a rough idea of where odds are going. Reason enough to use bots to lead unfortunate traders on a dance to penury. With people beginning to see sports betting as an asset class (link requires free registration) like stocks and commodities then it is inevitable that larger concerns start moving into sports betting and bring with them their nefarious ways from the financial world.

Betfair runs bots too

If you trade on Betfair's horse racing markets then you might see an odd price grouping somewhat below the current lay price for all the runners. For the favourite the offering will be £400 and there will be three such lays grouped with a two tick gap, sometimes three ticks. The image on the left shows two £400s and one that has merged with another lay of £149 to give £549 at a price of 1.74.

I noticed this awhile ago and I wanted to try and understand what was going on. Obviously it's a bot but whose and why? I noticed that there were no such back bets of a similar type so I assumed it was part of a 'lay only' operation. That means that it has to be a bookmaker's bot.

I call the bot the '400 Bot' due to the volume offered for the favourite at each price. For other horses in the market the volume offered will be lower.

Occasionally, the group of three offerings will drop as the price on a runner drops dramatically. Sometimes some of the topmost lay will be eaten into before the group moves further down the ladder. In anticipation of a further price drop?

I suspected that Betfair is running this bot. After all, Betfair now has a traditional bookmaking (sports book) operation in tandem with the exchange. Today, I watched the price on Betfair's sports book for the favourite in a race and just as the price on the book was lowered then so too the three lots of 400 dropped.

What the purpose of this is, I do not know. It can only be an exercise in getting to the front of the order book at those three particular prices. Nearer the off it is hard to get to the front of the order book so it is better to do it earlier in the day.

The groups of three are a little lower than the book price and naturally enough the book price is considerably lower than the exchange price. It's probably some sort of risk management device.

Whether this knowledge can be used to my advantage remains to be seen (and thought of) but I doubt it. Betfair moves these three lay bets in reaction to market activity and is unlikely to be anything predictive.

Does your trading software force you to be a bad trader?

I have listed elsewhere on my blog a list of trading software that I have used in the past. These days I write my own software to get the precise market view that I want, along with a few metrics that I have designed myself and wish to keep secret.

Looking back, I sometimes wonder if 3rd party software forces people to trade in a specific manner. Certainly, using a 3rd party application "out of the box" won't get the best out of the software. You will have to examine all the features and set up the software for the precise market view that you are looking for.

Most 3rd party software is set to update at a default update speed, typically at the 1 second rate. Such an update rate may force traders to become what is known as scalpers, who usually enter a market for a short period of time, typically less than a few minutes, before closing out their trade. Scalping is a valid trading methodology but I would only recommend to those with the right hardware and speed of Internet connection to help them get to the front of the market regularly.

Trading at the 1 second level is essentially noise trading (especially during the last ten minutes of a pre-race betting market) and it takes a lot of skill to determine a trend (if there is one) from noise. I wonder how many traders slow down the update rate for their software and investigate longer trend signals?

I know that some traders will have charts running at a different rate to get a trend but with a 1 second trading ladder next to it. Some of those traders might get spooked by a sudden movement on the ladder and trade out for a loss when there was no need to do so. This is a framing problem where you are looking at the same market but in two different time scales, which is going to be very confusing.

So, have a go at slowing down the update rate on your trading software. You might find a new niche in which to trade, away from the herd (or rather, the sheep and a small number of wolves).

Betfair API-NG progress report

It's been a number of months since I started porting from Betfair's old SOAP-based API-6 to the new JSON-based API-NG. In some ways it has been pleasurable and in other ways not.

The new API has permitted me to see Betfair's data in a new way. I have been able to quickly develop applications and get on with the task of analysing data for building new trading strategies. JSON permits easy collection and manipulation of data.

On the other hand Betfair itself has been none too helpful. Often, it's left to the users of the API to report bugs so that Betfair can improve the API. It seems to me that Betfair's developers have little knowledge of betting markets and don't appreciate the problems their simple bugs create. The developers are just looking to solve the problem of disseminating data with a top down approach. They have no understanding of a trader's requirements for clean data.

I have discovered that often there are duplicate horses in the listMarketCatalogue method. An easy bug to get around as you can handle the expection when building a database quite easily. However, it's the way that Betfair handled my bug report that left me far from impressed. First they denied the bug. After I sent them evidence in a JSON string containing duplicates they said they would fix the bug. That was many months ago and the bug has still not been sorted out.

Then I found that overseas races were being added to Betfiar's database of UK races causing races from the United States, Hong Kong and Dubai to be added to my bot. Again a crude work-around was needed.

Now I find that the totalMatched volume count for runners never equals the actual colume for each price traded and often the totalMatched increases with no apparent volume increase for any price.

These bugs leave one disconcerted as to whether the data is accurate or not. A trader lives by the accuaracy of their data. If Betfair is not feeding the right data then a trader could end up in a lot of bother.

All in all, I feel that the new API is very promising indeed. It allows me to do so much more than I was able to do in the past. My productivity rate has increased greatly but I still have a suspicion in the back of mind that something could go wrong at any moment.

Finding your own way

A common question you hear from newcomers to sports trading is, "How do you...?" and the common answer is, "I can't tell you." Information is the key to success in sports trading and people will hide information from others if it is of value.

Once valuable information is shared it loses its value. You only have to look at a betting market to see information being shared and its value being divided up amongst the quick thinking.

I have spent the majority of my life learning how betting markets work. Even as a child I was fascinated by sports betting. And yet, I never excelled at school, in fact I was the exact opposite of a good student.

You don't need to have had a good education to be a successful sports bettor but you do need to gain one. The ability to learn and research is important. At school I learned nothing. The classical schooling system never worked for me.

I wasn't a bad pupil. I tried my hardest but I wasn't good at anything because I couldn't concentrate when people spoke to me. However, I am good at reading. Through reading I learned what I wanted to know for myself in the style and pace that suited me. I left school with few qualifications because I read things that 16 year olds don't need to know to pass their O-levels and yet I still ended up at Oxford University many years later.

The greatest invention during my lifetime must be the World Wide Web, although the web didn't come into being until I was 27 years old. A web browser is the best research tool available. We must all learn how to use it effectively. And by effectively I mean finding things out for yourself and not just going to a forum and asking, "How do you...?" because more often than not nobody is going to tell you and you've just wasted valuable time.

If you are weak at mathematics (statistics and probability theory especially) then learn it online. With sports markets ever more similar to financial markets you should use financial trading websites to help you understand betting markets more. Both markets work using the same principles of supply and demand. You must also learn some basic programming techniques to increase your work flow.

Read. Learn. Experiment. And keep on doing it until you succeed.