Superforecasting

Traders are in the business of forecasting future price moves. The data they use to make these forecasts can be either fundamental (past performance) or technical (past prices) but the end result is a price. Superforecasting by Philip Tetlock and Dan Gardner details the art and science of forecasting.

The book is not full of maths and howtos rather it takes the reader through the process of what makes a superforecaster better than the rest of us. On this website I have already mentioned taking care when enlisting the "talents" of a tipster. Previous results alone are not enough when judging if you should use the wisdom of the crowd. It might be that your chosen tipster got lucky and only years of success and no regression to the mean is proof that they are as good as they say they are.

The book also points out that superforecasting is not just the domain of super-intellects. One of the author's of Superforecasting carried out a study using 2800 volunteers from the general public and got them to make thousands of predictions. The study found superforecasters through Wisdom of the Crowd

Some of these forecasters did regress to the mean because they made lucky guesses but there were some who remained consistently good at forecasting over time. Such superforecasters think through problems differently to the average person and do not allow personal feelings and beliefs to cloud their judgement. Compare a trader who diligently backtests a strategy and then tests again on future events before going live with their strategy to a "scalper" who chucks money in on a whim and believes they can trade out of any situation because they've done it once or twice before.

Too often people will use their instincts to make a decision. Our ancient ancestors used their "fight or flight" instincts successfully in their uncluttered worlds. All they had to worry about was avoiding lions in tall grass. Today's complex world often needs decisions made on deeper thought processes. Here is simple question from the book.

You are told that a bat and ball cost £1.10p together but that the bat costs £1 more than the ball. How much is the ball?

If your immediate thought was 10p then you are in good company. The wrong company but at least you are in the company of those using instinct rather than deep thought. If you analyse the problem carefully then you will come up with the answer of 5p because £1.05p plus 5p equals £1.10p and £1.05p is £1 more than 5p.

Too often impatient traders go with their instincts and make bad trades that may at first glance look good but which have negative edge. Such traders have a habit of starting with a conclusion rather than an hypothesis and looking solely for data to back up their conclusion. This book will certainly make you think more carefully about how to become a better forecaster (and therefore trader) or what to look for when enlisting the help of others.


Also - Who's #1?: The Science of Rating and Ranking - Teaches the sports trader how to rate and rank sports teams using a variety of mathematical techniques.

Soccernomics

With the new English Premier League season upon us you might like to read Soccernomics, now in its 3rd incarnation with updated data. But first, let's deal with the title of the book as I know that some readers in the UK will be harumphing at the s-word.

Soccernomics trips off the tongue more easily than Footballnomics and so aesthetically it's a good title. Also, for readers in the US, Canada, Australia and Ireland it clarifies which game is being referenced. Friends in County Kerry, Ireland, can therefore avoid reading the book as there is no mention of Gaelic football. Likewise, fans of "egg ball" as some English people jokingly call the American game or Australian "Ozzie rules" Football might not find the book interesting either. All the aforementioned are variations of the game of football and all have every right to call themselves football.

The word football is of medieval origin. In those times the word was spelled foote-ball and referred to ball sports played by the peasantry "on foot" and had nothing to do with the act of kicking a ball. The aristocracy preferred to play their games on horseback, such as hunting, falconry, jousting, war etc. Quite often the aristocracy would complain that the peasants were "playing foote-ball when they should have been practising their archery" (for an upcoming war). Each medieval parish had its own football rules and all rules permitted the handling of the ball, kicking of the ball and the kicking of other players.

The official name for soccer is Association Football after the association that was set up to codify its rules. During the Victorian period many games were being codified for the first time. Britain's public school system was the progenitor of many varieties of football games, many of which are played to this day (e.g. Eton Field Game). Public schoolboys have a penchant for shortening the names of everything and during the Victorian period Rugby Football was often shortened to rugger and Association Football was shortened to soccer. It could have been called "asser" but I think the boys played that particular game in the dormitories when the lights went out.

Soccer is a perfectly acceptable term in the UK because, as Soccernomics states, the word soccer was equally in common usage in the UK as the word football was up until the 1970s. It seems that around the time the North American Soccer League was created English football fans got a bit snooty about the word soccer and often muttered "What do Americans know about football?" Because of this young English people now assume that the word soccer is of American origin but incorrectly so.

When the codification of football began in the mid 19th century some clubs couldn't agree on which rules to keep and which to remove. This led the game to differentiate itself into two different codes (rugby and association) but both still referred to themselves as football because of their common origin. Eventually, one termed itself Rugby Football and the other, Association Football.

So alike were early games of rugby and soccer that clubs would often play one set of rules in the first half of a game and the other set in the second half. All versions of the game were football but each club adhered to a slightly different set of rules rather like the parishes of the medieval period.

Similar rules in both soccer and rugger include early soccer permitting the "fair catch" which is the same as "calling a mark" in rugby today. Also, early soccer used to outlaw (as rugby still does to this day) the forward pass. Early soccer games looked like rugby games with most players in a line using a 1-1-8 formation because of the forward pass rule.

Originally, early soccer players had to dribble the ball towards their opponent's goal and then either shoot or pass the ball backwards or laterally to another member of his team. Not until the forward pass was permitted in association football (by way of an offside rule) did the game become more like the one we see today. American football started out as a similar game to rugby until it too created the forward pass and so Association and American football have something in common, after all.


David Beckhamberger (midfielder) - recent draft pick for Milwaukee United

The book? Oh yes, the book. The book itself is very interesting, containing much research. For all the billionaire businessmen and corporate interest in soccer, the game itself is not a business. At least not like the usual businesses you see that turn a profit and sometimes go bust. The game is played by clubs and turning a profit is a lesser consideration than chasing success on the field. Clubs that go bankrupt are often "phoenixed" back into existence after the former owners scuttle off and new money is brought in.

Contrary to the belief of many England team fans their national side, after being beaten by the likes of Iceland, has usually played above its ability. Of course, Iceland more so in Euro 2016 but it will be unlikely for Iceland to repeat the feat next time, just as Greece returned to the doldrums following its success in 2004. On average teams play at or around their expected ability due to a variety of factors and one of the prime factors is population size.

Not for nothing is China planning on being a soccer superpower. It will take time but the likelihood of a China versus USA or Russia World Cup final grows by the tournament. However, Russia will have to ditch its authoritarian political leader as soccer is no longer won by dictatorships, as the book demonstrates.

The fans themselves are not all as it seems. Yes, there is a hardcore "Support X till I die." but they are very few in number. The average soccer fan supports many teams and these supporters often change their allegiance over their lifetime. The world's most soccer mad country? England? Brazil? No, think again. It's Norway.

There is a lot of research in this book that will change the way you view the "beautiful game". Maybe some of it will change the way you trade the game on the exchanges.

Amazon - Soccernomics: Why England Loses, Why Spain, Germany, and Brazil Win, and Why the U.S., Japan, Australia--And Even Iraq--Are Destined to Become the Kings of the World's Most Popular Sport

A Root for Squares

Being on the more sensible side of sports trading I don't get to hear of the wackier ideas that are out there. I am sure I don't have to explain edge and trading in proportion to perceived edge again. Most of you will use some kind of edge based trade sizing mechanism. Even if it is only to increase your trade size as a percentage of an increasing bankroll.

The point is that you are chasing your winnings and not your losses, which is the correct way to trade. If you are a winning trader then it is because you have a winning strategy. As your strategy increases the size of your bankroll you then have the luxury of increasing your trade sizes. If you begin losing then your bankroll will decrease and your trade size will reduce commensurately. At no time will you be desperate enough to increase your trade size to chase losses.

Whilst wandering through various websites I noticed a curious trade sizing algorithm called the Square Root System. The system starts out with level stakes trades, which is not the best way of making the most of your edge. If you are lucky enough to have a winning strategy then with this system you increase your future trade size by the square root of your increased bankroll.

For example, if you started with a £100 bankroll, were trading £5 at a time and increased your bankroll to £125 then your new stake size is £5 + sqrt(125-100) = £10. The system also requires the trader to reset the initial bankroll size (maybe £150 in this example) when a target value is reached, otherwise the square roots run away and trade sizes become too large and too risky. As your bankroll continues to grow then you continue to reset your bankroll and increase stakes by the square root when it goes above this new reset value.

If you think this is odd then you can reward yourself with your favourite snack. This system is not as bad as a Martingale as there is no loss chasing. However, as the trade sizing is not based on edge there is bound to be some wild swings in variance. The resetting of the bankroll to keep a lid on size of the square root is not optimised in any way and can lead to large variance in returns creating the opportunity for bankrolls to be wiped out.

Level stakes trading never makes the most of any edge you have so professional traders never use it. In the example above the bankroll has increased by 25% but the trade size has doubled from £5 to £10. What has the square root of your original trade size got to do with your edge? In conclusion the Square Root System is a level stakes system that uses an illogical (no reference to edge) stake increase if the bankroll should grow. Anyone using such a system is obviously too lazy to work out their edge or is not the trading/mathematical/financial genius they purport to be.

Some of the blame for these useless "staking plans" that we see can be laid at the door of bot software vendors who have no trading experience. They just hoover up all the crazy ideas on the web, such as Martingale and Square Root trade sizing, add it to their software and leave it to the buyers of such software to implement and learn the hard way. In my book, Programming for Betfair, I make sure not to give readers all the whistles and bells that are out there because I know I will have created a lot of angry beginner traders.

To all beginner traders I suggest you learn basic maths. Use a spreadsheet to understand edge and variance in returns and the consequences of deviating from edge. You will then understand how important trade sizing is. Not only for making the best of a winning strategy but also to avoid destroying your bankroll from trying to work the strategy too hard. There are no shortcuts to being a winning trader. You either put the work in or you don't and suffer the consequences.

When I see a website like this www.professional-racing-tips.com/staking-plans/square-root-staking-plan.aspx (I do not recommend any of its "staking plans") and see no mention of edge or Kelly Criterion then I know the website is not as "professional" as it purports to be nor the people who read and implement what is on such websites. Without developing your skills to analyse what people tell you to do then are just acting in the dark.

More articles on the Mathematics of Gambling

Setting Realistic Targets

Apparently, 'in X weeks I shall be retired' is the new "This time next year, Rodney..." After you have compounded your early winnings and without regard for losing streaks, ill health or other unforseen circumstances you set yourself the bold task of retiring early.

Only you won't be retiring because you intend sports trading from now until the monarch of the day sends you a congratulatory message on your 100th birthday. Trading is not a passport to an easy life, regardless of what others might have you believe. If anything you are setting out on the highway to a harder more stressful life.

Maybe because you enjoy sports trading so much you don't count it as real work but I can assure you it will be the hardest work you have ever done and probably for less income than you could have gotten elsewhere. Again that is not a problem for you because trading is the life you want to do in your "retirement". Maybe it's also because you've only had a pension fund for less than 10 years and you could do with the cash to keep you going.

I wish I could regard sports trading as not real work then I wouldn't be so exhausted at the end of the day with my eyes seeing in double vision and my back aching from sitting in this chair all day long. Betfair is no fault resistant stock exchange and my bots need a beady eye to keep them on the straight and narrow when the exchange throws a wobbly. Other traders may have simple bots trading with low stakes. My bots are precise and use larger stakes. I use them to trade multiple markets, which I couldn't do manually. I need to monitor my bots constantly and can never walk away from them.

It is good to see that one of the blog writers I read on occasions has taken a good dose of reality and changed his mind about early retirement. Of course, he can't admit that he is not as good a trader as he think he is, which is the real reason for his change of heart. His trading statistics never match his prose. Bankrolls are split up amongst systems and asset classes and quietly forgotten when they tank. A short run of success is followed by the inevitable wipe out. Honesty is the best policy because blog readers are not mugs. A hobby that pays a small income is better than a hobby that hopes to become a money spinning business but fails because of incompetence.

I have but one target, survival. The only way I am going to be a millionaire is when the last silver bullet of our increasingly corrupt financial system fails to hit its target and we go through a period of hyperinflation. I am sure the tin of beans I shall be buying with my 1 million pound note will be very tasty.

The Perfect Bet

I have just finished reading The Perfect Bet by Adam Kucharski, a history of betting in the past, present and future. The book starts with the familiar early days of probability theory, driven by Blaise Pascal and Pierre de Fermat. The theory of probability owes much to the world of gambling. Next, the book shows how brute force methods have been used by syndicates to win lotteries and that not all scratchcards are equal.

After the Second World War Monte Carlo methods were developed as a way in which to model how neutrons would behave in a hydrogen bomb. Such methods were then used to simulate millions of games of blackjack in order to find the optimal winning strategy.

The financial world is now rife with "rocket scientists" using physics to crack open the markets. One such scientist is Doyne Farmer who intially applied his knowledge to creating an edge in the casino game of roulette. Farmer then went on to use his skills to set up a fund in the financial markets.

William Benter gets a mention for his modelling of Hong Kong horse racing and making millions of dollars from his betting. Dixon and Coles and their initial work on modelling soccer games is detailed.

For those interested in bot trading there is a lot to digest. Mostly financial trading and poker playing bots are covered but there is a lot to be read between the lines. Whilst doing so I have thought of a few new approaches that I will apply to my bots. There is a lot you can learn from financial algo-trading. Much of it has nothing to do with the underlying financial instrument being traded.

Although only 200 pages long the book is very interesting and I highly recommend it.

Amazon - The Perfect Bet

Accounts Being Closed At Betfair

It is common to hear of accounts being closed at Betfair. Usually the reason is that premium charge traders are spreading their trades over multiple accounts so as to avoid the higher commission charge. Of course, such a practice is against Betfair's regulations and users with multiple accounts will be closed down.

Since Betfair started charging £299 for access to live data via its API I have seen a growing number of people who have had their account closed for no reason whatsoever. At least, that is what it appears to be. Some people contact me and they have categorically said that they were not multiple account users. They also said that they were not selling unofficial software to other users so that multiple instances of the software were accessing Betfair, which requires a Betfair commercial licence.

Why should Betfair close accounts for not apparent reason? And if they are will I be next? One reason for Betfair being so strict is that too many users are testing bots using the live AppKey and not using the delayed AppKey. Supposedly, that is the reason for the £299 charge for the live AppKey as a barrier to casual users taking up live bandwidth. But how you prove the difference between testing and trading is not obvious. I often run a bot for a day and perform no trades. I am merely gathering data. Is that counted as testing? Maybe some users are firing in thousands of sub £2 bets, which could be construed as testing. Especially if it is to hoover up stray pennies risk free using the round-up flaw, which Betfair does frown upon.

I should point out to users of Betfair's API-NG that the best way to test the betting side of a bot is to find a closed spread (i.e. a zero tick gap between the back and the lay) and to fire in a £2 back followed by a £2 lay (both "at the money") so that both sides of the trade are instantly matched. The loss will be minimal (just a few pennies, especially if you choose a sub 2.0 spread) and the £2 trades will keep Betfair happy.

I don't know what the future holds for Betfair users, especially users of the API. Betfair now concentrates on its bookmaking operation and with the Paddy Power link-up this can only increase. Maybe low volume API users will be frozen out and the exchange will only be for commercial use by third-party software vendors, premium charge users and independent bookmakers laying off liabilities. Who knows?

Survivorship Bias

This article has been on my hard drive for many months, waiting to be finished. I thought it was a little short until a certain blog (I will spare the blog's author from his blushes this time) dumped a load of badly researched material onto my lap. Thanks to that blog I now give you Survivorship Bias...

Here's an interesting problem. Supposedly, during the Second World War, analysis was performed on aircraft returning from their sorties. Analysts wanted to know where to place additional armour on aircraft so as to protect aircraft crew.

The picture below displays an aircraft representing the sum of all returning aircraft. The parts of this summated aircraft in black represent areas which suffered hits from bullet and cannon rounds. The white areas suffered little to no hits. From this evidence where do you place the additional armour?

The answer is that you place additional armour in the white areas and not the areas that have suffered the most damage. The reason being that only aircraft that survived and returned to base are represented in the return data. Aircraft that were shot down did not make it home to enter the dataset.

The white areas represent the pilot's cockpit and the aircraft's rudder. If either of these two areas are hit then the aircraft won't be returning home to be entered into the analyst's datasheet. A dead pilot or an aircraft that can't yaw is the worst that can happen to an aircraft and it won't be returning home to be logged.

Those of you who thought of reinforcing the wings or the engine area were caught out by survivorship bias. You only considered the data in front of you and not other data that would have been pertinent to the question.

Today we are seeing a lot of survivorship bias but in a more subtle way. For example, the "Post Brexit Racism" videos of verbal assaults and worse on foreigners in the UK. Yes, racism is not to be tolerated but I would say that these racist attacks were by racists who were racist before the referendum and would be racist if the referendum never happened. Just because these videos came about after Brexit does not mean they were caused by Brexit. If the Remain vote had won then these racist acts would still have been perpetrated and these videos would not have been given such prominence or would have been called "Sour Grapes Videos".

Also, the anti-Brexit crowd were quick to point out a downturn in the stockmarket but within a few days of the referendum the FTSE 100 had recovered. "Ah but!" shout the anti-Brexit crowd. "Look at the FTSE 250. That is more representative of trade with Europe rather than the globalised trading of the FTSE 100."

Yes, the FTSE 250 is still down but then all markets in Europe are down too as can be seen in the EuroStoxx 50 chart below. After all, Euro companies are not going to be shooting themselves in the foot by refusing to trade with one of the largest economies in the world. This same anti-Brexit crowd also point out that Sterling "appears" to be in free-fall against some currencies. And yet all currencies are constantly devaluing against each other. A far better index of currency strength is gold, which has risen against all currencies and not just Sterling.


Actually, Sterling falling against the currencies of our main trading partners is good for business as it makes our exports cheaper. At any other time a devaluation of Sterling would be looked on positively in economic terms but not at this time when it is given a different significance for political purposes.

Another attack on a weaker Sterling being good for business is to say that the UK is a net importer. This is just a smoke screen. After all, so too is the United States a net importer as can be seen in the chart below. Without exports the UK would be on its knees. The fact that the UK imports so much can be interpreted in so many ways. For example, overpopulation and the UK not living within its means or the UK has a booming export business requiring raw materials. Just blurting out "the UK is a net importer" is just scaremongering to belittle our exports.


And so, on many counts, we see data being used to prove a point that is actually an untruth. Data not being looked at in the whole and just a few elements being picked out to prove a point. Articles from left-wing newspapers and left-wing think-tanks being quoted as unbiased facts. The fact is that the financial system has not been fixed since it was revealed to have been broken in (or long before) 2008. That interest rates are still very low and will probably go even lower. Not just in the UK but throughout the world.

We have only to look at Japan to see decades where there have been deflationary periods and constant low interest rates. Brexit was an excuse to let markets find their own way (rather than banks manipulating indices as they have been shown to be) and if the suits can scare people into changing their mind about Brexit then that is just an added bonus.

Nothing has been fixed since the crisis of 2008 and there is probably more financial turmoil to come and little of it the fault of the UK. Government debt continues to rise. The world continues to live beyond its means. Bankers continually create new exotics to gamble with in their casino masquerading as a market. The markets love volatility because that increases volume and more volume means more commissions for the bankers. For a banker, a chaotic market and a controlled population can only be good.

You must understand the whole picture before using a subset of data to confirm an incorrect hypothesis. Do not start with a conclusion and then fit data to create a false hypothesis. A trader should look on volatility with a positive mind and should not be deviated from opportunities by biased data or biased thought. One person's error is another's opportunity. As Liberace once said, "I cried all the way to the bank."