Agent-Based Trading Models

Here is an interesting article written by J Doyne Farmer (previously mentioned in my last article as one of the creators of the first digital roulette computer) on agent-based trading models.

Farmer co-founded The Prediction Company, a company that has created trading models since the early nineties. Since then Farmer has returned to academia and now researches complexity economics at Oxford Univesity.

I am beginning to look at agent based models myself. My intention is to create a model horse race betting market containing various agents; naive bettors, arbitrageurs, trend traders to see how they interact with each other.

From there I will probably use genetic algorithms to evolve traders that will enter the market and attempt to maximise their return. The aim will be to see if there are any trading strategies that are superior to others under certain circumstances.

In time I will create a citations page for J Doyne Farmer as complexity is something that I am beginning to move into and his papers have a lot of interesting material (if not extremely readable compared to some papers) in them.

Toward Agent-Based Models for Investment

The Predictors - A book about the setting up of the The Prediction Company.

Gambling Connections

Did you know that the first wearable computer is linked with finance, information theory and roulette?

Currently, I am writing up a seminar that I will give at my Alma Mater. I have two different seminars in mind and might (in the spirit of progressive rock) bolt the two unfinished works together into a behemoth and leave the audience to make of it what they will.

My research has sent me back to the history of information theory and the discovery of a paper written by Ed Thorp entitled "The Invention of the First Wearable Computer". Believe it or not the device was created in 1961!

At the famous Bell Labs worked a genius by the name of Claude Shannon who was allowed to let his mind flow in any direction that pleased him. To that end in 1948 Shannon wrote A Mathematical Theory of Communication, the seminal paper on Information Theory, in part to assist Bell Telephone's headaches with increasing complexity from burgeoning post-war telephone use.

A colleague of Shannon's was John L Kelly, who used the new Information Theory to develop what is now known as Kelly Criterion, used by gamblers and investors alike to maximise the return on their investments.

Mathematician Ed Thorp became a friend of Shannon's when they discovered that they had a joint interest in gambling. The two of them decided to look at roulette (which Thorp had done some preliminary work on) and how they could use physics and information theory to gain an edge over the house. From their research they developed the first wearable computer, to be surreptitiously taken into a casino to beat the game of roulette.

Thorp and Shannon's intentions were to treat a roulette wheel as a Newtonian system and determine into which octant of the wheel the ball, orbiting the rotating wheel, was likely to finish in. It wasn't necessary (nor feasible - chaos theory is for another day) to determine the exact cup the ball would fall in, just enough of a guess was sufficient to gain an edge over the house. They could bet on all the numbers in the octant and still have a positive expectation.

Thorp also developed card counting through computer simulations and used Kelly Criterion to gain an edge over the house in the game of blackjack. Thorp and Shannon paid many visits to Las Vegas where they profited on the blackjack tables. The team made only made dry runs with the roulette computer. Thorp had by then turned his attentions to the financial world, profiting from market inefficiences and decided to make the roulette computer common knowledge.

In the early 1970s a team of students from the University of California, headed by Doyne Farmer, used the recently developed microprocessor to improve upon Shannon and Thorp's analogue roulette computer and beat the game in Nevada's casinos. Later, Farmer (with a few friends from his roulette escapade) went on to create The Prediction Company, which was one of the first to perform algorithmic trading in the financial markets.

So you see there are quite a few connections here even for James Burke to make a television programme out of. Now, why didn't he?

Further Reading

Fortune's Formula - covers much of the work of Kelly and Thorp.

The Eudaemonic Pie (aka The Newtonian Casino) - How a team of students beat the game of roulette.

The Predictors - How a team of academics created an algorithmic financial trading company.

The Invention of the First Wearable Computer - Thorp's paper

It's all about the variance - part 2

To understand money management we must understand risk and to do that we must understand variance.

Variance is a mathematical term that is also used in finance and refers to the spread of numbers in a group or population (e.g. the spread of returns from a betting system or the different heights of people in a population).

If we perform a run of 1000 bets or trades and the percentage returns are spread out over a large range of values then we have high variance. If, on the other hand, our range of returns are tightly clustered around the mean average then we have a lower variance.

The larger the variance the greater there is a chance that returns will deviate (hence the term standard deviation) from the mean. Also, the greater the spread into negative yield the greater the chance of a run of losing bankrolls.

In the following simulations we see a variety of variances. The first simulation shows a lot of variance. We can see that there are a roughly equal proporation of winning and losing bankrolls. In fact in this instance the expectation for this simulation is zero.

Remember that expectation is given by (prob of success * return) - (prob of failure * loss) and so in this instance the expecation is

 (0.4 * 1.5) - (0.6 * 1) = 0

The greater the variance the greater the chance that our bankroll will move into negative territory. We may wish to nullify this by risking smaller amounts or by having a stop loss. Again we have to balance variance (the risk) against reward (the return).

We can make a foray into Kelly Criterion by running the same simulation again only using the Full Kelly staking option instead of Level stakes. After running the simulation we see that the timeseries for the bankroll has not changed. No bets were placed!

As Kelly Criterion is based on expectation then because our expectation is zero Kelly Criterion recommends that we bet none of our bank. We are as likely to lose as we are to win.

A higher yielding  simulation shows a tighter spread of finishing bankrolls. The yield is higher and the variance is smaller.

We can make the following conclusions. The higher the yield the lower the variance. Strike rate also lowers variance.

Next time we shall look at the differences between the various staking systems; fixed, proportional and geometric (Kelly).

It's all about the variance - part 1

A lot of newcomers to sports trading or betting may imagine that all they have to do is pick winners. They might think that getting as close to a 100% strike rate will guarantee millionaire status. It's just not as simple as that.

For a start, we are going to pick losing bets or make losing trades. Nobody is perfect. Even if we could get close to a 100% success rate we would probably be betting or trading too little to make any serious money.

Risk and reward go hand in hand. The more we are willing to risk then the greater the potential for reward. That doesn't mean we should throw our money round and hope to get lucky. The key to success is money management, balancing risk and reward.

To manage risk we protect our capital from the downside whilst making the most of the upside. We don't even have to have a 50% strike rate to make a profit. All that matters is that when we lose, we lose less than we win.

In the example simulation below (click to enlarge) on RiskSimulator 2, we see a trader who only has a 40% strike rate but who makes twice as much from winning trades as on losing trades.

This simulation models a trader who makes 2% profit on average from a 40% strike rate and who manages to keep his losses down to a 1% average. Intuitively, we know that if the strike rate falls below 33% (1/(2+1) = 0.333) then the trader will make a loss.

Of course, all of these simulations can be calculated mathematically without the need for Monte Carlo methods. The point of the simulation is to show the variance graphically. A lot of novices have difficulty understanding the mathematics of gambling. A graphical representation can often aid the beginner sports bettor/trader.

The formula for expectation will tell you what you need to know about your bets without recourse to a simulation.

Expectation =  (prob of success * return) - (prob of failure * loss)

If the figure is negative then it's a losing system.

For our example simulation the expectation will be

(0.4 * 2) - (0.6 * 1) = 0.2% per bet

The simulation was pretty close (£19.46p average yield) to the expected £20 but we can see clearly in the simulation that we could have made more or less and in some cases there might have been a small drawdown before making a profit. The expectation is based on an infinite series and our betting will tend towards expectation.

In theory when we calculate probabilities we are doing so from a distribution derived from an infinite series. However, we never get to place an infinite number of bets. A few hundred bets will be unlikely to produce smooth distibutions. A simulation helps us to understand short time variance very clearly.

In the example above, although all 100 trials yielded positive returns some were higher than others. The  highest yield was £32.40 and the lowest a little over £7.

However, by trying to lose as little as possible we don't want to be so risk averse as to avoid any losses at all. Doing that can often prevent us from taking the risk of getting the right rewards. It's a matter of balance. Our winning bets or trades must overcompensate us for our losses. Do that and we will always be in profit.

RiskSimulator 2

I am currently working on an improved risk simulator. The simulator will visually depict simulations of betting, trading and staking. The picture above displays the Staking simulator tab, which I am currently working on.

You will notice Martingale and Fibonacci staking options. I am not advocating them as viable staking plans. If the release program has them then it will be to demonstrate that they don't work. However, I expect most of my readership to intelligent enough to have discounted them already so I might remove them as options.


I have written a simple RiskSimulator for level stakes betting. The application permits up to 100,000 bets to be simulated with graphical outputs. The user enters fair and bookmaker/exchange odds, a Monte Carlo simulation handles the rest.

When the simulation is complete the yield, risk of no profit and risk of ruin are output. Also, two charts are output; a time series progression for the bankroll and a histogram of variance.

Although using standard statistical methods such as the binomial distribution will provide more accurate results, the simulator provides a more easy to understand visual representation.

Also, a binomial distribution is based on an infinite number of trials. As nobody bets an infinite number of times a Monte Carlo simulation can show the user the wide variation between what is expected mathematically and what may happen in reality.

Above, we see that the distribution of results in the histogram is not a smooth curve (as one would expect from a binomial distribution) even for 100,000 trials.

Future work will include a StakingSimulator for users to investigate the worth of various staking systems, including level stakes, proportional stakes, Kelly Criterion and guaranteed losers such as Martingale and Fibonacci.

How to Find a Black Cat in a Coal Cellar
Being a technical trader in sports betting markets I have no interest in tipsters. However, I do enjoy reading any mathematical book on sports betting and the mathematics contained in this book are invaluable, even to those who don't use tipsters but develop their own trading and betting strategies.

How to Find a Black Cat in a Coal Cellar is written by Joseph Buchdahl, author also of Fixed Odds Betting, another book I highly recommend. Buchdahl attempts to show those who use tipsers (also called advisories) how to evaluate their performance.Most tipsters, says Buchdahl are either amateurs who don't what they are doing or are fraudsters scamming people for subscription fees.

Although the book is primarily for readers who want to evaluate tipsters, in effect the book is for the evaluation of any systematic approach to betting. Therefore the book is important to anyone who wishes to evaluate their own strategies as if they are tipping bets to themselves.

The first chapter is a description of how various forms of betting work and how the aim is not just to pick winners but winners whose odds give you an edge in the long run. Not news to experienced bettors but you will be surprised how many people still don't understand that just picking a winner is not enough.

You can't predict winners all the time so your winners have to compensate for the losers. Hence, the odds on your winners must be higher than your predicted odds so that you hopefully gain a profit. The first chapter goes on to explain the overround and how that eats into profits. The chapter also covers market inefficiencies such as favourite/long-shot bias, the over-betting of under-dogs and under-betting of favourites.

Finally, the first chapter covers staking and money management. Not exhaustively, as the author covers the subject in more detail in his previous book, Fixed Odds Betting. Needless to say, Buchdahl is against progressive betting systems such as Martingales but he is pro Kelly staking for the maximisation of returns.

In the second chapter we begin to see a methodology for evaluating betting systems be they from tipsters or created by oneself. Again some fairly familiar terms are presented; Return on Investment and Yield. Worth a read for revision, if you think you already know all about it. Many don't.

A positive yield is not necessarily going to make you rich. It's not like a bank account offering 5% per annum and you get all your money back from the government if the bank fails. Buchdahl shows by way of a simple example that even a losing strategy has the potential to yield a profit 30% of the time simply because of variance.

The chapter then proceeds to show the affect of low edge on variance. The lower the edge you have over quoted odds the greater your variance. You could show quite a run of profitable bets even with no edge at all. Worse, you could easily lose your bankroll.

Also shown is that the higher the odds you bet also affects variance. Again, the higher the odds, the higher the variance. Efficient market hypothesis tells us that market odds are close to true and so higher odds win less frequently, adding to the variance if the yield is quite low.

Even if you have two systems that yield 10% profit, if one is betting at higher odds than the other then its variance will be greater than the other and will have a greater chance of losing the bankroll than the other.

The chapter continues by introducing the t-test for statistical significance. Essentially, this is giving the reader a value for the probability that a tipster made their profit through chance and not through skill.

Chapter three covers value for money. Tipster subscription fees are factored into rates of return. Buchdahl looks at what the naive bettor can achieve by themselves and what a tipster can add to that. Arbitrage is covered as a way of guaranteeing success without the fundamental knowledge purportedly known by tipsters. If a tipster cannot beat simple arbitrage then they are no good at their trade. There is also an account of Buchdahl's attempt to win a last man standing competition using value betting.

In chapter four Buchdahl evaluates tipsters and their records. Buchdahl goes through many tipsters and their reported success rates. I won't go through any of it here as the data is in the book and the maths used to evaluate the tipsters has already been mentioned.

The level of unprofessionalism and charlatanism displayed by most tipsters is not unsurprising. Amateurish strategies involving the chasing of losses as an attempt to boost profits are evidenced. The closing down of initially profitable tipster websites, reminiscient of beginner bettors who boast of their skills on blogs that are soon left moribund when the profits start heading south.

In chapter five Buchdahl makes his assessment of tipsters. Most he says, are amateurs, half of whom make a profit but far fewer than that can claim to do so through skill rather than by luck, as has already been shown mathematically. A lot of tipsters "cook the books" and after they have been verified by a tipster evaluation service, such as Buchdahl's own service, profits usually diminish.

Finally, in chapter six Buchdahl show the reader how to professionally go about evaluating tipsters, not just mathematically but carrying out domain checks and web searchers to see if others have reviewed the tipster as legitimate or a flim-flam artist.

My review is not really enough to do justice to this book. There is so much of value to be read. The book will help you to determine which tipsters, if any, are as good as they claim. The book will also improve your own (self-tipping) strategy building.

I recommend that you have Excel or other spreadsheet software close to hand so that you can feed the functions provided into a spreadsheet and experiment with them. You can then have the functions on hand to remind you of the important facts contained in this book.

This book certainly ranks amongst the more important books on sports betting.

Amazon - How to Find a Black Cat in a Coal Cellar

Also recommended - Fixed Odds Betting