|
The downloader tool, one of the best features of QuantShare, let you retrieve data from Internet and add it to the quote or any custom database. But did you know that you can use the downloader tool to automate importing data files from your hard drive? By simply replacing a URL by a file path, you can use the downloader tool to read files data and import that data into your database. Here is what you will learn in this blog post: - Using the downloader to parse local files - Import all files in a specific directory - Automate data import How to create a downloader to parse local files - Select "Download -> Download Manager" - Click on "Add" - Click on "Add URL" - Under "URL" column, type the file you want to parse. Example: c:\data\appl.csv - Under "Content Type", select "CSV" or "Zip" if the files you want to parse are compressed - Click on "Parser" button under "Parser" column - In "Parser Settings", specify the correct settings to successfully parse the file If the symbol name of each security is specified in the file content then you simply have to reference it within parser settings ("Quotes -> Symbol" column). Otherwise, you must update the downloader so that the symbol name is picked from the file name. In the latter case, set the file location to: C:\data\[SYMBOL].csv Click on "0 Field(s)" under "Fields" column then add a new field. Set the field type to "Symbol" then type a symbol to test under "Value" column. Also, uncheck the option under the last column. Now, that we have parsed a single file, let me show you how to parse several files using a single downloader. Import all files in a specific directory - Select the downloader we have previously created then click on "Update" in the download manager. - Click on "Settings" button under "Settings" column - Check "Load the URL-Script only once" - Click on "Create URL-Script" This script allows you to dynamically create URLs or file paths to parse. - Type the following script then click on "OK" string[] files = System.IO.Directory.GetFiles(@"C:\data\", "*.csv"); int count = files.Length; for(int i=0;i < files.Length;i++) { string file = files[i]; Functions.AddURL(file, "Parsing: " + i + "/" + count); } As you can see from the above code, the directory location is specified in the first line. The "GetFiles" function returns all files with the "csv" extension. If your files have another extension then simply update the "*.csv" value with "*.[extension]". To read all files, type "*.*". In case the symbol name is not specified within the file content, we must get it from the file path. In "CSV Settings" (click on "Parser" to bring this control), add a new "Quotes -> Symbol" column, click on "Pre-Script" then type the following code: int symbolColumnIndex = 0; System.IO.FileInfo path = new System.IO.FileInfo(Content.GetURLORFileName()); string name = path.Name.Replace(path.Extension, ""); for(int i=0;i < Content.Rows.Length;i++) { Content.Rows[i].Data[symbolColumnIndex] = name; } Make sure you update the "symbolColumnIndex" variable so its value correspond to the position of the "Quotes -> Symbol" column. If that column is set first then you should set a value of 0 to "symbolColumnIndex". Automate data import If you want to import data every day (or every few minutes/hours) then you will be probably interested in automating this process. The script editor is a QuantShare tool that allows you to create scripts to control QuantShare and automate things. The script we will implement now, simply starts a specific downloader. If you mix that tool with the task manager then you will be able to create a task that automatically starts any downloader at a specific time. - Create a new script using "Tools -> Script Editor" - Add the following code to your script: Downloader.StartDownloader("Name"); // Replace "Name" by the name of your download item - Now, open the task manager by selecting "Tools -> Task Manager" - Click on "Click here to add a task" - Select the "Minute/Hour/Week/Month" settings - Click on "Add" under "Scripts to be executed" - Select the previously created script then click on "Add" under "Task" panel to create your new task If you have any questions then please add them below.
|
|
