Betfair bot observations

Spend any length of time on Betfair's exchange and you will see bots in action. A lot of the bot action is instigated by Betfair itself. To a large extent Betfair's sports betting exchange is a dark pool. However, I don't believe any of the users are getting preferential treatment as the High-Frequency firms are in the financial markets. More that Betfair itself has bots running on the exchange to hoover up loose change. Maybe dark pool is a strong term, considering its negative connotations in the financial world but the exchange exists to make Betfair's shareholders wealthy people.

Betfair has its own traders and also its own fixed odds service for those who just want to bet and not get involved with trading. The more profit Betfair makes the happier  its shareholders will be. Betfair's traders will have their own idea of where prices should be. Coupled with that is demand from the fixed-odds service, which means that Betfair is always dipping into the exchange when it sees a value arb or wants to balance its book. When there are large gaps on the odds ladder I like to place minimum £2 bet here and there to see how the bots react. If there is a one or two tick gap in the spread I might place £2 within the spread to see if the bots are backing or laying. More often than not your £2 is taken within the blink of an eye.

Again, I don't think anyone is getting preferential treatment but that sort of speed suggests some form of low-latency activity. And, as Betfair does not (to my knowledge) permit co-location of private servers within Betfair's data centre then that is probably a Betfair bot snapping up my £2. Not only that but the bot appears to have a quantity in mind that it wants to take. If you fire in multiple £2 bets quickly enough you will sometimes see the amount that the bot would like to back or lay. There will be a fleeting moment where you will see the size of bet used to snap up your £2 and then it will disappear.

For example, if I continuously place a lay bet of £2 in rapid succession on an empty price in the grid and do it fast enough then I will first see maybe £50 hit my £2. The next time I place £2 the bot will jump on it with £48. The following £2 bet will be hit by £46 and so on until the bot has attained the £50 position. Obviously there is someone out there desperate to open a £50 position at that price but they do not want to advertise the fact by leaving a £50 order on the ladder.

Another thing you can do is place your £2 in a large gap that might have developed between backers and layers. Within a split second more volume will be placed on and behind your £2. On thin volume you can place your £2 on an empty price behind the front backs and lays. Again, it will set off a flurry of activity. After a few bets have been taken on either side of the spread you can see the return legs of the bets being placed on the other side of the spread. More often than not these bets are placed in anticipation of movement and bets to be matched in the future.

Finally, about 20 minutes before the race the ladder will suddenly fill with volume a good 10 to 20 ticks either side of the spread. It's not a gradual ramping up of volume. It just appears in a split second. Not only that but the amount of volume (even far away from the spread) is constantly changing. I believe it might have something to do with fixed-odds and/or SP betting but that is not my field of interest so I'll leave that for someone else to work out.

There is a lot of maths going on in Betfair horse race betting exchange and most of it doesn't have anything to do with horses. That is what I find so fascinating about exchange betting; market structure. There is so much more to making money from trading the structure of the market and I enjoy learning about it. I hope you enjoyed this article. I would be interested to hear your thoughts.

Nice hobby - I wouldn't do it for a living

These days, making money from horse races has little to do with horses. I wonder why many people (I call them 'The Old Boys' hence the photo) still analyse past horse race data. In finance we call them fundamentalists, people who read reams of data to ascertain a true price value. As with their religious counterparts these fundamentalists are very rigid in their thinking. Fundamental trading would make for a nice hobby but I wouldn't want to do it for a living. My preference is for technical trading.

Who wants to earn money whereby on some days you make a loss and you have no idea why? Is the betting system broken or does it have negative expectation and will fail anyway? If you have what it takes to be a trader then after the intial learning curve there will be very few losing days, if any at all. If I have a low yielding day from trading then I can usually narrow it down to getting stuck on a horse and over-trading. Either a reversal was missed and I had to trade my way out of it or I tried to noodle around scraping up pennies on narrow margin trades. Either of these two will add turnover for little return thus impacting on yield. Over-trading being my sole concern, trading is a lot easier than betting.

This morning, I awoke at 9:10AM, a little late as I have Seasonal Affective Disorder. I normally sleep with the curtains open so that sunlight awakes me naturally, every morning. In the summer, I leap out of be at 5AM to start the day. In the winter, if the day is as overcast as today then I drag myself out of bed at between 8:30AM and 9:30AM. As with any morning, the first thing I do is to switch on the PC and start my software running. I am very happy with the code that I created using the new Betfair API. The ability to monitor every runner in every race has increased the amount of trading I do. More trading leads to more turnover and more profit.

The names of race courses, runners, jockeys, owners and trainers are unknown to me. All I am interested in is trends. By 10AM I have started to open some positions and by the start of the first race I have opened all the positions that I am likely to open that day and have profits that give me leeway to speculate when the larger volume starts arriving before the off. For most of the day its a case of monitoring positions. A position might reverse and it's a case of easing into a new trend or closing out for a minimal loss. Nearer to the race start time I will start selling my position back to the market, a little at a time. Initially, to lock in a profit and then to make the most of it, if the position has done very well.

At no time do I look at a horse race. I know that other traders have a video link to the race but I never do. Any shenanigans in the paddock, leading up to the race start, will make itself known to me in the price but by then I will already have profits with which to cashout or use to trade a new position with. Whenever I read online forums dedicated to form, I am always amused. By and large the systems they come up with are so specific that I doubt they will be used much. Broadcasting them to the world doesn't help either. The idea of placing a bet on a horse and waiting for the end of the race is anathema to me. I am done and dusted before the race starts.

The late afternoon and evening is for me to do with as I please. No reading form nor watching videos of previous races. The hard work was in understanding how betting markets work. By that I don't mean the simplicities of how backing and laying works and the overround. There is a lot more to betting markets than that; such as structural and psychological factors. Much of the structure of a betting market is very subtle but easily understood by any finance quant. Many people who trade profitably do so minus a lot of knowledge that could make their trading even more profitable. By using third party software traders are constrained by the capabilities of their software and do not have the imagination to try new avenues of thought.

I have a simple motto and it goes like this "I like going to sleep every night having learned something new." Even though I am a profitable trader I am not resting on my laurels. I am constantly developing new ideas, coding improvements to my software and researching the effects of little anomalies I notice during the day. If you want to make money from horse racing then forget the horses. For me the only aspect of horse race betting that involves horses is the fact that without them I wouldn't be able to trade on the races. For that I am grateful.

Agent-Based Trading Models

Here is an interesting article written by J Doyne Farmer (previously mentioned in my last article as one of the creators of the first digital roulette computer) on agent-based trading models. Farmer co-founded The Prediction Company, a company that has created trading models since the early nineties. Since then Farmer has returned to academia and now researches complexity economics at Oxford Univesity.

I am beginning to look at agent based models myself. My intention is to create a model horse race betting market containing various agents; naive bettors, arbitrageurs, trend traders to see how they interact with each other. From there I will probably use genetic algorithms to evolve traders that will enter the market and attempt to maximise their return. The aim will be to see if there are any trading strategies that are superior to others under certain circumstances.

In time I will create a citations page for J Doyne Farmer as complexity is something that I am beginning to move into and his papers have a lot of interesting material (if not extremely readable compared to some papers) in them.

Toward Agent-Based Models for Investment

The Predictors - A book about the setting up of the The Prediction Company.

Gambling Connections

Did you know that the first wearable computer is linked with finance, information theory and roulette? Currently, I am writing up a seminar that I will give at my Alma Mater. I have two different seminars in mind and might (in the spirit of progressive rock) bolt the two unfinished works together into a behemoth and leave the audience to make of it what they will.

My research has sent me back to the history of information theory and the discovery of a paper written by Ed Thorp entitled "The Invention of the First Wearable Computer". Believe it or not the device was created in 1961!

At the famous Bell Labs worked a genius by the name of Claude Shannon who was allowed to let his mind flow in any direction that pleased him. To that end, in 1948 Shannon wrote A Mathematical Theory of Communication, the seminal paper on Information Theory, in part to assist Bell Telephone's headaches with increasing complexity from burgeoning post-war telephone use.

A colleague of Shannon's was John L Kelly, who used the new Information Theory to develop what is now known as Kelly Criterion, used by gamblers and investors alike to maximise the return on their investments. 

Mathematician Ed Thorp became a friend of Shannon's when they discovered that they had a joint interest in gambling. The two of them decided to look at roulette (which Thorp had done some preliminary work on) and how they could use physics and information theory to gain an edge over the house. From their research they developed the first wearable computer, to be surreptitiously taken into a casino to beat the game of roulette.

Thorp and Shannon's intentions were to treat a roulette wheel as a Newtonian system and determine into which octant of the wheel the ball, orbiting the rotating wheel, was likely to finish in. It wasn't necessary (nor feasible - chaos theory is for another day) to determine the exact cup the ball would fall in, just enough of a guess was sufficient to gain an edge over the house. They could bet on all the numbers in the octant and still have a positive expectation.

Thorp also developed card counting through computer simulations and used Kelly Criterion to gain an edge over the house in the game of blackjack. Thorp and Shannon paid many visits to Las Vegas where they profited on the blackjack tables. The team made only made dry runs with the roulette computer. Thorp had by then turned his attentions to the financial world, profiting from market inefficiencies and decided to make the roulette computer common knowledge.

In the early 1970s a team of students from the University of California, headed by Doyne Farmer, used the recently developed microprocessor to improve upon Shannon and Thorp's analogue roulette computer and beat the game in Nevada's casinos. Later, Farmer (with a few friends from his roulette escapade) went on to create The Prediction Company, which was one of the first to perform algorithmic trading in the financial markets. So you see there are quite a few connections here even for James Burke to make a television programme out of. Now, why didn't he?

Further Reading

Fortune's Formula - covers much of the work of Kelly and Thorp.

The Eudaemonic Pie (aka The Newtonian Casino) - How a team of students beat the game of roulette.

The Predictors - How a team of academics created an algorithmic financial trading company.

The Invention of the First Wearable Computer - Thorp's paper

It's all about the variance - part 2

To understand money management we must understand risk and to do that we must understand variance and its consequences. Variance is a mathematical term that is also used in finance and refers to the spread of numbers in a group or population (e.g. the spread of returns from a betting system or the different heights of people in a population).

If we perform a run of 1000 bets or trades and the percentage returns are spread out over a large range of values then we have high variance. If, on the other hand, our range of returns are tightly clustered around the mean average then we have a lower variance. The larger the variance the greater there is a chance that returns will deviate (hence the term standard deviation) from the mean. Also, the greater the spread into negative yield the greater the chance of a run of losing bankrolls.

In the following simulations we see a variety of variances. The first simulation shows a lot of variance. We can see that there are a roughly equal proporation of winning and losing bankrolls. In fact in this instance the expectation for this simulation is zero.

Remember that expectation is given by (prob of success * return) - (prob of failure * loss) and so in this instance the expecation is

 (0.4 * 1.5) - (0.6 * 1) = 0

The greater the variance the greater the chance that our bankroll will move into negative territory. We may wish to nullify this by risking smaller amounts or by having a stop loss. Again we have to balance variance (the risk) against reward (the return). We can make a foray into Kelly Criterion by running the same simulation again only using the Full Kelly staking option instead of Level stakes. After running the simulation we see that the timeseries for the bankroll has not changed. No bets were placed!

As Kelly Criterion is based on expectation then because our expectation is zero Kelly Criterion recommends that we bet none of our bank. We are as likely to lose as we are to win. A higher yielding  simulation shows a tighter spread of finishing bankrolls. The yield is higher and the variance is smaller.

We can make the following conclusions. The higher the yield the lower the variance. Strike rate also lowers variance. Next time we shall look at the differences between the various staking systems; fixed, proportional and geometric (Kelly).

Further Reading

It's all about the variance - part 1

It's all about the variance - part 1

A lot of newcomers to sports trading or betting may imagine that all they have to do is pick winners. They might think that getting as close to a 100% strike rate will guarantee millionaire status. It's just not as simple as that. For a start, we are going to pick losing bets or make losing trades. Nobody is perfect. Even if we could get close to a 100% success rate we would probably be betting or trading too little to make any serious money.

Risk and reward go hand in hand. The more we are willing to risk then the greater the potential for reward. That doesn't mean we should throw our money round and hope to get lucky. The key to success is money management, balancing risk and reward. To manage risk we protect our capital from the downside whilst making the most of the upside. We don't even have to have a 50% strike rate to make a profit. All that matters is that when we lose, we lose less than we win.

In the example simulation below (click to enlarge) on RiskSimulator 2, we see a trader who only has a 40% strike rate but who makes twice as much from winning trades as on losing trades.

This simulation models a trader who makes 2% profit on average from a 40% strike rate and who manages to keep his losses down to a 1% average. Intuitively, we know that if the strike rate falls below 33% (1/(2+1) = 0.333) then the trader will make a loss. Of course, all of these simulations can be calculated mathematically without the need for Monte Carlo methods. The point of the simulation is to show the variance graphically. A lot of novices have difficulty understanding the mathematics of gambling. A graphical representation can often aid the beginner sports bettor/trader.

The formula for expectation will tell you what you need to know about your bets without recourse to a simulation.

Expectation =  (prob of success * return) - (prob of failure * loss)

If the figure is negative then it's a losing system.

For our example simulation the expectation will be

(0.4 * 2) - (0.6 * 1) = 0.2% per bet

The simulation was pretty close (£19.46p average yield) to the expected £20 but we can see clearly in the simulation that we could have made more or less and in some cases there might have been a small drawdown before making a profit. The expectation is based on an infinite series and our betting will tend towards expectation. In theory when we calculate probabilities we are doing so from a distribution derived from an infinite series. However, we never get to place an infinite number of bets. A few hundred bets will be unlikely to produce smooth distibutions. A simulation helps us to understand short time variance very clearly.

In the example above, although all 100 trials yielded positive returns some were higher than others. The  highest yield was £32.40 and the lowest a little over £7. However, by trying to lose as little as possible we don't want to be so risk averse as to avoid any losses at all. Doing that can often prevent us from taking the risk of getting the right rewards. It's a matter of balance. Our winning bets or trades must overcompensate us for our losses. Do that and we will always be in profit.

Further Reading

It's all about the variance - part 2

RiskSimulator 2

I am currently working on an improved risk simulator. The simulator will visually depict simulations of betting, trading and staking. The picture above displays the Staking simulator tab, which I am currently working on.

You will notice Martingale and Fibonacci staking options. I am not advocating them as viable staking plans. If the release program has them then it will be to demonstrate that they don't work. However, I expect most of my readership to intelligent enough to have discounted them already so I might remove them as options.