Click here to Login



2020-04-29 02:04:06

I appear to be having an issue with the downloaders, that seems to be based on the GMT of the exchange where the data is coming from.
I only pull down data from the NZX and ASX (NZ and Australia) and also monitor some key US indices like the S&P500 DJI etc

Recently with my ASX symbols I needed to know the days of the week.
Used DayOfWeek() and noticed some days were listed as 0 and some 6 i.e. Sunday and Saturday.
Thinking there was an error in the data, I downloaded the entire database again with the "Historical Stock Market Data" downloader,
again it shows up this inconsistency occasionally with Sunday and Saturday listed as trading days, also tried with "Yahoo EOD Quotes" and same thing.

Created a new account with just ^NZ50,^AORD,^GSPC
Used Yahoo EOD quotes to download from April 1st to April the 24th 2020
For the NZ50 16 bars are downloaded with dates from the 31st March to the 23rd of April , but with the data from the 1st of April to the 24th of April
For the AORD It starts at the 31st of March and ends at the 24th, also 16 bars. Pricing after the 6th of April is correct, before is following day pricing.
Also it jumps from the 2nd of April to the 6th of April, Friday the 3rd is missing.
For the ^GSPC it is perfect.

NZ is GMT + 12, AU is GMT + 10, US-New York is GMT -4

Had a look at the Downloaders URL script, and tried adjusting the P1 and P2 values with UNIX time offsets, but not much luck.

Mucked around a bit with the generated URL from yahoo EOD Quotes (AORD below) and cant seem to alter the outcome.

Putting the below into a browser like Firefox will get the data nicely formatted in JSON, and you can check the first timestamp is 31st Mar 2020^aord?formatted=true&crumb=&lang=en-US&region=US&period1=1585699200&period2=1587735173&interval=1d&events=div%7Csplit&

Also had a look at indices closer to the GMT like a number of the European ones and these seemed correct

So just a "too far from everything" issue? Easy fix?


2020-05-08 21:37:33


Ok, been trying alternate downloaders like YLoader etc to get around this date issue, but found other small quirks using these packages.

Had a closer look at the Data Yahoo was presenting for download via say "Yahoo EOD" and it seems the timestamp has been adjusted for the exchanges GMT offset.

So all I have done is altered the "Pre Script" to add the exchanges GMT offset to the timestamps and this appears to have fixed the problem.

All my ASX and NZX data is now correct, i.e. no Sunday or Saturday trading days. Also checked the S&P500/DJI and also appears correct. (note, was not wrong previously, but this didn't break it)

Not 100% sure this will work with every exchange, hence not uploading, but for GMT > 8ish exchanges like Australia and New Zealand. the data and dates, line up correctly now.

Pre Script below , altered lines have //*

System.Threading.Thread.CurrentThread.CurrentCulture = System.Globalization.CultureInfo.InvariantCulture;
for(int i=0;i<Content.Rows.Length;i++)
Content.Rows[i].IsIgnoreLine = true;

string content = Content.GetContent();
JsonData j = Content.Net.ReadJsonData(content);


DateTime unix = new DateTime(1970, 1, 1);
JsonData r = j["chart"]["result"][0];
JsonData q = r["indicators"]["quote"][0];
JsonData t = r["timestamp"];
JsonData c = q["close"];
JsonData o = q["open"];
JsonData h = q["high"];
JsonData l = q["low"];
JsonData v = q["volume"];
JsonData gmt = r["meta"]["gmtoffset"]; //*
for(int i=0;i<c.Count;i++)
int tt,os; //*
if(int.TryParse(t[i].ToString(), out tt))
string open = (o[i] == null) ? "" : o[i].ToString();
string high = (h[i] == null) ? "" : h[i].ToString();
string low = (l[i] == null) ? "" : l[i].ToString();
string close = (c[i] == null) ? "" : c[i].ToString();
string volume = (v[i] == null) ? "" : v[i].ToString();
int.TryParse(gmt.ToString(), out os); //*
tt += os; //*
Content.AddRow(unix.AddSeconds(tt).ToString("MM/dd/yyyy", System.Globalization.CultureInfo.InvariantCulture), open, high, low, close, volume);

2021-04-05 11:46:34


Best Answer
Thank you David.

The downloader was updated.
Historical Stock Market Data

Historical Stock Market Data (by QuantShare, uploaded several months ago)
No notes

Rate an item Rate an item Rate an item Rate an item Rate an item Number of downloads Notes Report an item

No more messages


No html code. URLs turn into links automatically.

Type in the trading objects you want to include: - Add Objects
To add a trading object in your message, type in the object name, select it and then click on "Add Objects"


Trading Items
BSE stock quotes
Rank Industries based on the average Relative Strength Index
Israel Stock Market
Update the Industry Name of your U.S. Stocks
Price to Book Ratio Historical Data - Market to Book

How-to Lessons
How to quickly select stocks based on the last value of a databas...
How to download EOD quotes for active and valid stocks only
How to quickly download the most recent EOD data for your stocks
How to download a trading item from the Sharing Server
How to save and restore charts

Related Forum Threads
downloaders using google trends data
Daily and Historical Downloaders
SecureChannelFailure on downloaders
Script to remove Symbols from an Index
Symbol Information

Blog Posts
Creating a download item: Initial Jobless Claims
The Ultimate Guide to Create Trading Systems in QuantShare
How to Download Trading Data for Certain Securities Only
Backtesting trading strategies using intraday data
Trading Forex with the commitments of traders report

Create an account
Affiliate Program
Contact Us
Trading Forum
How-to Lessons
About Us
Terms of Use

Copyright 2021
Social Media
Follow us on Facebook
Twitter Follow us on Twitter
Follow us on Google+
RSS Trading Items

Trading financial instruments, including foreign exchange on margin, carries a high level of risk and is not suitable for all investors. The high degree of leverage can work against you as well as for you. Before deciding to invest in financial instruments or foreign exchange you should carefully consider your investment objectives, level of experience, and risk appetite. The possibility exists that you could sustain a loss of some or all of your initial investment and therefore you should not invest money that you cannot afford to lose. You should be aware of all the risks associated with trading and seek advice from an independent financial advisor if you have any doubts.