Backtesting

What is Backtesting?

Backtesting is a method of testing strategies and their historical returns produced throughout the years. It also determines the gain and advancement of a strategy, which helps assess whether the strategy an investor is testing is worth implementing in the live markets.

Key characteristics of backtesting:

• It assures the gain and advancement of a strategy.
• Understanding the theory of a strategy that worked or failed in history and if it is likely to do the same again.
• Testing strategies on past charts and schemes is integral in implementing new trading and investing strategies.
• Measures quantitative data such as profit/loss, annualized returns, risk analysis, and profit/loss percentages.

How Backtesting Works

Since you will be using historical data, there is no need to look to the future for more data. This system is the opposite of forward performance testing. 

Testing this way is a great way to get a feel for whether your strategy will work and is worth implementing. Another thing that makes this testing system so awesome is that you don’t have to risk your or your client’s capital. 

There are different ways that a person can back-test strategies. For example, an investor or trader can use a manual or purchase software that will allow one to place variables in the system that can generate results. Manual testing allows a trader to put absolutely no capital into their test.

While there are some drawbacks to testing a strategy using historical data, there are other testing systems that a person can use to counteract them after you complete the backtest. One of them is forward performance testing, and we will compare the two later on. 

The bottom line is that using a strategy that has been backtested and is made sure to be profitable is much more likely to be efficient in live trading. 

Backtesting main purpose

Backtesting’s main purpose is to test the statistical effectiveness of trading or investing strategies. It grants traders and investors the ability to strategically simulate a trading style using past data to generate and assess the profitability and risk before using any real capital.

Traders who test strategies on historical data and receive positive results allow them to recognize that their strategy is likely to work. In addition, it allows traders to understand that their system is dependable and likely to produce positive returns over a while or the opposite. 

If a trading strategy can be quantified, then it can be backtested. Some traders may outsource this task to professional code programmers, thus creating data that is more testable and comprehensive. Said programmers may work for large financial institutions or stock brokers. 

Programmers may make the test accessible to the trader, where they can make user-defined input variables resulting in new or different data. One example of these variables may be price channels over Bollinger bands or vice versa. 

By testing this historical data, an investor is trying to quantify the effectiveness of their prospective strategy through statistics such as profit/loss analysis, risk analysis, annualized returns, and profit percentage. These results will determine if the strategy is worth using. 

The Ideal Backtesting Scenario

When you implement your trading system, the optimal test will include current or relevant data. This will help a trader understand whether their strategy will work when needed.

The past data should ideally include stocks, ETFs, and indexes currently traded, and those not traded anymore, giving a trader a full understanding of all the possible outcomes of their trades. 

The test should include trading costs as well. Although these may seem like they are not important, these costs add up in the end and may affect the overall outcome of the strategy and the benefit of implementing it into their trading plans. 

If your sample of testing comes back with positive results, it may be a good idea to implement these tests into a sample you may not be looking to trade in. But, on the other hand, if this sample also gives positive results, your strategy is more likely worth continuing with. 

This is a great way to reassure the strategy will work before using real capital. Testing historical data can help predict and measure net profit/loss, return, risk-adjusted return, and market exposure.

backtesting Vs. performance testing

Paper trading, also called performance testing, is another way of testing the viability of a trading strategy. Paper trading allows traders to test out-of-sample data. This test allows traders to test their strategy on the live market.

When paper trading, you should follow the strategy exactly, without bias, to ensure that the system works well. However, traders must be logical with themselves and trade the same as they would if they were not using paper trading. This will make sure the strategy works.

You can also think of creating a backtest as recreating a scenario using historical data, whereas forward performance testing creates a new scenario in real time. 

While forward performance testing, it is critical to follow your strategy parameters, like when you are testing historical data. If you do not, your test may result in a false conclusion. 

If there is little to no correlation between in and out-of-sample results, then your backtest is more than over-optimized. Thus, forward performance testing would be a waste of time because it would likely fail in the live markets. 

Some platforms, such as TradingView, offer back and forward performance testing.

Who Uses Backtesting?

Anyone can run these tests, but the programming and coding use expensive data that is hard to get your hands on. As a result, institutional investors and money managers are common users of backtesting. 

Since institutional investors have so much capital, they can test whatever dataset they need. Moreover, since they manage such large amounts of money, they may be required to test their system before implementing it. 

Here is an example and what it might look like:

Suppose you just got an internship as an analyst at your favorite investment firm, and they ask you to test a new strategy. The strategy hypothesizes that you should buy shares of blue chip stocks every time the RSI (Relative Strength Index) hits thirty. 

Every time a stock you watch touches thirty(oversold) on the RSI indicator, you must acquire shares. 

If you backtest this information, it turns out you got a better return on your investment when using the new strategy. Then you have found a new system that can be implemented confidently and is expected to have great returns.

This system may work better than the current strategy in place. But first, test the correct amount of in and out-of-sample data to ensure you get the correct results. 

Results from your backtest can be shown through an equity curve. An equity curve is just a trend line that shows the profitability of one’s strategy. 

Not related to the strategy above, tests can be done with special software that can act like a plug-and-play with variables. Once the test has been completed, and the data is recorded, the equity curve should look something like this:

This chart shows a trend line positively sloped with drawbacks showing slow growth or small setbacks. 

backtesting Steps

Investors and traders may test differently; for example, an investor may look back on years worth of historical data, whereas a trader may look back on days, weeks, or months of historical data. The following are the steps involved in backtesting:

1. Define the strategy

Draft the parameters that justify the strategy being used. For example, asset classes may call for different data characteristics affecting essential historical information. 

For example, investing in bonds or indexes will need more historical data than a short-term stock options trader.

2. Find the desirable trades

Be sure to find entries that meet your parameters. Make sure to record all the entry and exit data the strategy had the trades or investments completed. 

The investor or trader should document all positions taken and the returns they produced. It is imperative that they only take positions that they would take with real capital. Otherwise, the test will be a waste of time with wasted data. 

3. Calculate the net return

Net return can be calculated by factoring in costs, including but not limited to transaction costs, commissions, subscriptions, or other tools. First, evaluate the gross return, then compare it to the costs of the investment or trade to find the net return percentage. 

If done accurately, the test will determine if the system can be implemented with confidence, tweaked a little bit, and then retested. 

If a losing strategy is a conclusion, parameters may be changed and tested again. If the trade is profitable, the system can be exercised in the live markets with faith that it will succeed. 

Some Pitfalls of Backtesting

One major component that may be a danger to only backtesting is over-optimism. This may happen when an investor or trader tweaks their parameters to receive max profit, but these parameters may be unrealistic. 

Thus, tweaking parameters for max return will result in a flawed strategy that will likely not produce the desired result. 

Another form of bias that may negatively affect your test could be survivorship bias, meaning one testing a strategy that worked based on one period. 

For example, testing a strategy when there were outlying factors such as a new virus breakout or periods of wartime. 

Traders must be careful not to unintentionally botch their strategy testing, but sometimes that can be hard. Overfitting is when investors may ruin the backtest. This happens when they don’t use an equal amount of in and out of sample data. 

Traders and investors need to test their strategy on the right amount of in-sample data and out-of-sample data. It may not always be an easy circumstance, though. Sometimes it can be hard to tell how much out-of-sample data you have tested on.

Likewise, if you don’t test enough out-of-sample data, you may produce artificial results that will fail in the future. 

For clarification, if testing a strategy with only in-sample data, your equity trendline will probably look great. Except, as soon as you use your out-of-sample data, it will start to plateau or decline. If it keeps goings up, you know your strategy is working great!

Another drawback may be forward-looking bias. This happens when you add data to your test that won’t be there when you implement your strategy. For example, if one were to add data that normally won’t be available in that strategy, it could produce a false result. 

One of the most important things that a backtest cannot capture is a company’s performance, the numbers they are producing, and what they mean. This could have a lot of influence on the market and how it moves up and down, side to side.

Stock prices are not always spot on with how the company is performing. Sometimes a stock price can be overvalued or undervalued, so when completing a backtest, the investor or trader should ensure they know the overall health of the companies they may be using. 

Tips For Backtesting Investing

There are plenty of factors and variables that one must be aware of when testing a strategy.

Here you will find a list of 10 essential things to recall when performing this investing and trading strategy:

1. Always ensure you know the sector(s) in which your test was placed. For example, if you only tested your strategy on industrial and energy stocks, then your strategy may not work on consumer staples or technology stocks. 
2. Have a general idea of your test's current and historical broad market trends. For instance, tests done on the markets after Black Monday may not be ideal while currently in a booming economy. However, you should test over a large time frame, incorporating many market cycles.  
3. Keep in mind the amount of exposure that your accounts have. More exposure could mean more profits, but it also means higher losses as well. A general account shouldn’t have any more than 70% exposure.
4. Determining the best position sizing and money management is very important. The average gain/loss statistic can be combined with the wins-losses ratio using a technique such as the Kelly criterion. The Kelly criterion allows traders to maximize their positions across fewer trades, thus decreasing their commission costs.
5. Be aware of volatility, and keep it as low as possible. A strategy high in volatility could ruin leverage accounts, which are prone to margin calls.  
6. Don’t be overly optimistic. Over-optimism can lead to a great strategy test, but it can shatter the real deal when implemented. Be realistic with yourself and the live markets.
7. Using blind data may be the best answer when testing a strategy. Finding data proving your hypothesis is an easy and unconscious mistake. On the other hand, one may try so hard to be unbiased that they don’t use data that may help their system. 
8. You can follow up your system with a forward performance test and paper trading to ensure that your testing is accurate and will have a winning record. Paper trading will allow you to test your strategy in the live markets, thus creating more confidence in the system.
9. Benchmarking your system’s return against other investments is very important. You do this by calculating the annualized return.The calculation for annualized return includes looking at the risk-adjusted return, accounting for the return, and the amount of risk required. Therefore, a strategy must get better returns with less risk than the other investments to be implemented.
10. Testing customization is essential to getting the most accurate returns, or lack thereof. There are many backtesting variables, many of which depend on the broker that the trader uses. Some examples of these variables include commission costs, position-sizing rules, stop-loss settings, and margin requirements.

Conclusion

Backtesting is essential to implementing a new investing or trading strategy; anyone can do it! It can ensure confidence in a new plan and help articulate the returns that may come from it. 

Although, one must remember it is not the end-all and be-all for implementing a new investing and trading strategy. The best way for a trader or investor to follow up with a backtest would be to perform a forward performance test by paper trading. 

Moreover, when testing, using both in-sample and out-of-sample data ensures the test works and can be implemented confidently. Be sure not to include bias in your test, which may give you a sense of overconfidence in your plan. 

Although, as stated above, you don’t want to use biased data, you do need to ensure that you are not going out of your way to use data that has nothing to do with your strategy or could potentially make it fail. 

The best way to overcome this over or under-bias is to take data that is close to or the same as the current economy and data that could be from a different market. For example, you may consider taking data from a great economy and one close to falling apart. 

When completing a backtest, you will use historical data that will test the statistical effectiveness of your strategy. If your test proves that your system is not ready to be implemented, you may want to take a step back and try to switch or tweak it a little. 

Always remember some of the drawbacks and limitations that historical testing can cause,

Lastly, everyone can do a backtest, and there are no limitations on how much you can spend on these tests. For example, an investor can complete a manual backtest and record the data for free. They can also easily pay a programmer to do it for them and be given the results. 

Researched and authored by Adam Bridges

Reviewed and Edited by Abhijeet Avhale | LinkedIn

Free Resources

To continue learning and advancing your career, check out these additional helpful WSO resources:

• Junior Tranche
• Key Currency
• Key Performance Indicators (KPIs)
• Know Your Client (KYC)
• Kuwaiti Dinar (KWD)