Click here to Login

                                                   In-Sample and Out-Of-Sample Dates for Optimization


2011-07-19 11:12:39

In previous work I have done- I have found it very useful in either system or indicator optimization to have 'in-sample' and 'out-of-sample' dates for optimization.

I found that specifying 2000-2005 as in-sample and 2006 as out-sample was much more likely to result in over-optimization rather than using the entire date range for optimization and choosing random months.

This of course does not apply to all optimizations (sometimes you want recent dates optimized fully etc).

Let me see if I can explain. What you want is for a given date range that you are optimizing for (daily,weekly,monthly) is to have randomly within that date range- in-sample and out-of-sample dates.

The random is important. After optimization on in-sample- you can confirm on out-of-sample.

What I did in my optimizer I designed was have an check button next to a date range and then a drop down of % dates to include.

So for example the user chooses 1/1/2000 to 1/1/2011 for optimization next to it is option to check 'random months %' . He checks it then chooses 70%.

The optimizer then takes the total number of months- and randomly throws out 30%.

Then internally during the optimization run I just closed all positions at last day of month if the next month was 'out-of-sample'.

Now the problem is after optimizing you want to see the out-of-sample dates in your date range.

So what I did is have a function elsewhere in program to specify in-sample and out-of-sample dates and store them.

So you would have a program to generate a list of in-sample and out-of-sample dates for months/days etc.

The user would choose full date range- percentage of out-of-sample etc and two files would be made with dates to include.

Then in any optimization procedure the user would specify 'use in-sample monthly dates' or 'in-sample daily dates' for optimization.

Then when simulating a trading system- you could confirm after all your optimizations on 'out-of-sample' dates.

Anyway- there are many different ways to do this- I hope this gives you some ideas. Thanks. Marc

2011-07-20 05:16:11

2011-07-20 11:43:01

I think your function is fantastic- a very elegant and flexible way to do exactly what i want- THANKS!

No more messages


No html code. URLs turn into links automatically.

Type in the trading objects you want to include: - Add Objects
To add a trading object in your message, type in the object name, select it and then click on "Add Objects"


Trading Items
Strategy Optimization on Random in-sample and out-of-sample Perio...
Advancing-Declining-Unchanged volume for NYSE, AMEX and NASDAQ
5-Minute Historical Intraday Data for Forex, Indices and Futures/...
EOD Quotes Data for Hong Kong Stocks and ETFs
Historical EOD data for LSE-listed Stocks and ETFs

How-to Lessons
How to download EOD quotes for active and valid stocks only
How to display the number of stocks per day for different RSI gro...
How to download and use U.S. stocks earnings data
How to get stocks for a particular index using the global script
How to screen for stocks having a high correlation with the Dow J...

Related Forum Threads
Equity Curve and Trades list for out-of-sample Walk Forward test
Optimization and fitness function for robustness vs for performan...
Optimize for each symbol and walk-forward test on list
Use AI optimization (PBIL/Genetic) for walk-forward optimization
indicator for 1 symbol in screener and simulation

Blog Posts
6 ways to download free intraday and tick data for the U.S. stock...
Creating and managing notes for your stocks
Step by step on how to get free realtime/delayed data for stocks,...
How to get buy and sell orders for a portfolio based on a trading...
How to Scan for Parallel Resistance and Support Lines

Create an account
Affiliate Program
Contact Us
Trading Forum
How-to Lessons
About Us
Terms of Use

Copyright 2021
Social Media
Follow us on Facebook
Twitter Follow us on Twitter
Follow us on Google+
RSS Trading Items

Trading financial instruments, including foreign exchange on margin, carries a high level of risk and is not suitable for all investors. The high degree of leverage can work against you as well as for you. Before deciding to invest in financial instruments or foreign exchange you should carefully consider your investment objectives, level of experience, and risk appetite. The possibility exists that you could sustain a loss of some or all of your initial investment and therefore you should not invest money that you cannot afford to lose. You should be aware of all the risks associated with trading and seek advice from an independent financial advisor if you have any doubts.