Click here to Login

How to speed up quotes and news downloads

Updated on 2009-09-29

The US stock exchanges list more than 7000 companies. This huge number of stocks make it time consuming to download quotes, news or any other data. As an example, if you want to download news data for US symbols, the "yahoo news downloader" (Yahoo News) will have to send as many requests as the number of symbols that exist in your account; that is, +7000 in the case you are trading US stocks.

For the rest of this article, we will take the "yahoo news downloader" as an example.
One way to speed up the downloading process is by removing the stocks or symbols that do not have any news items. These symbols may not be recognized by Yahoo or may be too small to have news reported in the Yahoo website.

In order to remove these stocks from the list of symbols used by the downloader, we first need to search for these tickers. This can be done using the screener plug-in (Note that prior to this, you will have to execute the downloader for a few days to collect some data).

- Select "Analysis->Screener" in the menu to open the screener plug-in
- Select "Bar" and keep "n" at zero.
- We will use the "GetDataCount" function to get the number of news items for each bar. -- GetDataCount("yahoo_news", "title") --
This function will return the number of news items for each symbol, but only for the last bar. However, because we need to know whether a stock has at least one news item in all its history, we should add the "sum" function. The new formula will be: Sum(GetDataCount("yahoo_news", "title")) > 0;

- Start the screener to display the symbols that should not be removed
- Copy these symbols by clicking on the "Copy Symbols to Clipboard" button, then close the screener form.
- Select the "downloader manager" form , then open the "yahoo news" item.
- In the "Symbols" tab and under the "Name" column, select "list"
- Click on "Custom Symbols" (under the "Values" column)
- In the "Choose symbols" form, press on CTRL+V to paste the symbols or right click using your mouse then select "Paste"; Click on "OK".
- That is it; in my computer I have reduced the number of symbols from +7000 symbols to approximately 5000 symbols, which will speed up the downloading process by nearly 28%.

If you are interested in liquid stocks, you can further decrease the number of symbols by modifying the rule used in the Screener plug-in.

no comments (Log in)

QuantShare Blog
Search Posts

Recent Posts

Create Graphs using the Grid Tool
Posted 1318 days ago

Profile Graphs
Posted 1423 days ago

Previous Posts

The 'inside period' function
Posted 5407 days ago

Quantshare version 1.4
Posted 5421 days ago

Looking for trading ideas
Posted 5427 days ago

A vector-based language
Posted 5430 days ago

Data snooping bias
Posted 5443 days ago

More Posts


Create an account
Affiliate Program
Contact Us
Trading Forum
How-to Lessons
About Us
Terms of Use

Copyright 2024
Social Media
Follow us on Facebook
Twitter Follow us on Twitter
Follow us on Google+
RSS Trading Items

Trading financial instruments, including foreign exchange on margin, carries a high level of risk and is not suitable for all investors. The high degree of leverage can work against you as well as for you. Before deciding to invest in financial instruments or foreign exchange you should carefully consider your investment objectives, level of experience, and risk appetite. The possibility exists that you could sustain a loss of some or all of your initial investment and therefore you should not invest money that you cannot afford to lose. You should be aware of all the risks associated with trading and seek advice from an independent financial advisor if you have any doubts.