Click here to Login

Trading System Analysis: Backtesting report and custom measures

Updated on 2012-05-16

Properly analyzing a trading system report is crucial before trading it. Once a trading system is backtested, you will get a detailed report that displays all the measures you need to analyze and evaluate your trading system.

Backtesting Report

The strategy report of QuantShare displays the following tabs:

Summary: Displays how some important measures evolved over time (Equity, Drawdown...)
Statistics: Shows +50 measures for the backtesting process
Trades: Shows realized trades and orders flow
Details: For each trading day, you can see here portfolio positions, pending orders and portfolio stats
D.W.M.Y: Performance per day, week, month and year
Profit Distribution: Shows scaled profit and loss values
Graphs: Create custom graphs
MAE/MFE: Maximum adverse excursion distribution and maximum favorable excursion distribution. Maximum loss a trade had before it was closed and maximum profit a trade had before it was closed.
S.I.M.I: Performance per sector, industry, market and index
Exit Rules: Average performance per each exit type
Monte Carlo: Allows you to perform Monte Carlo analysis.

The "Statistics" tab of the backtesting report contains many measures that should help you analyze and decide whether to consider the trading system or not.

Here are some important ones:

Sharpe Ratio:
It is a measure of risk-adjusted performance.
Trading System: Buy stocks with the highest Sharpe ratio
Trading System: Buy Stocks based on their Sharpe Ratio Rank
Optimize a trading strategy using the Sharpe ratio
Sharpe Ratio - Part 1
Sharpe Ratio - Part 2

Sortino Ratio:
This measure was developed by Frank A. Sortino. It is the same as the Sharpe ratio, except if uses the downside deviation instead of the standard deviation.
The downside deviation is calculated by taking the standard deviation of negative asset returns. It is used in the Sortino Ratio to ignore good volatility and thus provide a risk-adjusted measure without penalizing it for upward price changes.

Profit Factor:
Calculated as the profit of profitable trades divided by the losses of losing trades, the profit factor relates the amount of profit per unit of risk.
The higher the profit factor value, the better and less risky your trading system is.

Payoff Ratio:
The higher the payoff ratio the better the system. This measure is calculated by taking the system average profit per trade then dividing it by the average loss per trade.

Performance Vs Benchmark Analysis

In the "Statistics" tab, you can also see statistics that compare the performance of your trading system with an index (Beta, Alpha, R Squared and Correlation).

To calculate these measures, you will have to specify a reference index in the trading system settings.

- Select a trading system then click on "Update"
- Select "Settings" then click on "Capital"
- Type a symbol next to "Reference Symbol". Example: ^GSPC (for the S&P 500 Index)

Create a Custom Measure

- Select a trading system
- Click on "Create a metric" (If you do not see that button, click on "+" icon to extend menu)
- Type your formula in C# or JScript then click on "Save Formula"

Example: (Metric result should be associated with "fitness" variable)

if(AnnualReturn > 10 && SharpeRatio > 1 && SortinoRatio > 1.5)
Fitness = 1;

The above measure returns "1" if the trading system report has an annual return higher than 10%, a Sharpe ratio higher than 1 and a Sortino ratio above 1.5

- After you save the metric, click on "Metric" button
- Check your item then click on "OK"
- Backtest or optimize your trading system to show this metric in the main table

Custom Measure using the Money Management Tool

The money management tool allows you to create measures for advanced analysis.

How add a script to your trading system:

- Select a trading system then click on "Update"
- Select "Money Management" tab
- Click on "Add a new money management script"
- Create your metric using one or several events then use the "OnEndSimulation" to add the it to your trading system report

Functions.AddMetric("MyMetric", value);

Example: (Number of trades whose return is higher than 10%)

OnEndSimulation Event: (C#)

int nb = 0;
MMPosition[] pos = Portfolio.GetAllPositions(); // Get all positions (open and closed ones)
for(int i=0;i < pos.Length;i++)
if(pos[i].Performance > 10) // Check position return
nb = nb + 1;

Functions.AddMetric("MyMetric", nb);

How a Measure Evolve Over Time

With the money management tool, you can also create time-series metrics. These metrics will be displayed in the "Summary" tab of the trading system simulation report.

In order to create a time-series metric, you should add the metric value on each trading bar. To do this, call the "Functions.AddMetric" function in the "OnEndPeriod" event.

Example: (Number of pending buy orders)

OnEndPeriod Event: (C#)

int pendingOrders = Orders.GetPendingBuyOrders().Length;
Functions.AddMetric("PendingOrders", pendingOrders);

The above script is executed on each trading bar and each time it adds the number of buy pending orders to the "PendingOrders" time-series.

How to display and analyze this time-series on a chart:

- Add the above script to a trading system
- Backtest the strategy by clicking on "Simulate"
- In "Summary" tab, right click on the chart then select "Create new pane"
- Right click on the "Equity" measure then select "Remove selected graph"
- Click on the dropdown control next to "Select a time-series to drag and drop into the chart"
- Select "PendingOrders"
- Click on "Drag" icon then drag and drop this item into the new pane

Use the left bar to Like or Tweet this post. Thank you.

no comments (Log in)

QuantShare Blog
Search Posts

Recent Posts

Create Graphs using the Grid Tool
Posted 1298 days ago

Profile Graphs
Posted 1403 days ago

Previous Posts

More Posts


Create an account
Affiliate Program
Contact Us
Trading Forum
How-to Lessons
About Us
Terms of Use

Copyright 2024
Social Media
Follow us on Facebook
Twitter Follow us on Twitter
Follow us on Google+
RSS Trading Items

Trading financial instruments, including foreign exchange on margin, carries a high level of risk and is not suitable for all investors. The high degree of leverage can work against you as well as for you. Before deciding to invest in financial instruments or foreign exchange you should carefully consider your investment objectives, level of experience, and risk appetite. The possibility exists that you could sustain a loss of some or all of your initial investment and therefore you should not invest money that you cannot afford to lose. You should be aware of all the risks associated with trading and seek advice from an independent financial advisor if you have any doubts.