Click here to Login





                                                   Downloader Pre-Script and "NUL"-character

  0

0
SergejSuperstock
2016-10-20 14:25:06


Hey guys,
I'm trying to create a downloader for wikifolio.com, german social trading platform.

Link to the CSV is:
https://www.wikifolio.com/dynamic/de/de/invest/download?type=daily&name=SG1979&dateFrom=31.12.2002&dateTo=20.10.2016

Sample is:
W F 0 0 Q U I N T E
Q u a l i t � t , a n g e l e h n t a n S u s a n L e v e r m a n n
A l l p r i c e s i n E U R


B e g i n d a t e ; T i m e i n t e r v a l ( m i n ) ; O p e n ; C l o s e ; H i g h ; L o w
1 1 . 0 9 . 2 0 1 6 0 0 : 0 0 : 0 0 ; 1 . 4 4 0 ; 2 8 3 , 9 4 6 ; 2 8 3 , 8 1 0 ; 2 8 3 , 9 5 1 ; 2 8 3 , 8 1 0
1 2 . 0 9 . 2 0 1 6 0 0 : 0 0 : 0 0 ; 1 . 4 4 0 ; 2 8 2 , 6 0 7 ; 2 8 3 , 8 7 1 ; 2 8 4 , 4 5 3 ; 2 8 2 , 0 0 0

I can get most of the problems solved, but one I cant: I seem they put a NUL-character after each character (Notepad++ shows them). Already tried to delete them with the pre-script, but seems like it does not work.

My Pre-script is:
for(int i = 0; i < Content.Rows.Length; i++)
{

PreScriptRow myRow = Content.Rows[i];
string myRowText = myRow.Data[1];
myRowText = myRowText.Trim();
//myRowText = myRowText.Replace("\0", "");
myRowText = myRowText.Replace(Convert.ToChar(0x0).ToString(), "");
myRowText = myRowText.Replace(".", "-");
myRow.Data[1] = myRowText;

}

The Input sample Quantshare shows always is just "W" which is the first character of the file. Seems like it can not handle the NUL.

Any ideas anyone?

Cheers.
Sergej



QuantShare
2016-10-21 07:32:51

  0

Hi Sergej,

You need to ignore all the rows: (loop through each row)
Content.Rows[i].IsIgnoreLine = true;

Get the content, remove the null character then parse the data:
string ct = Content.GetContent();
ct = ct.Replace("\0", "");
string[] rows = ct.Split('
');
....

After that, insert rows dynamically using "Content.AddRow" function




SergejSuperstock
2016-10-21 13:10:23

  0

Hi QS,

thank you for the answer! All dates are now read correctly, but no open/close/high/low data. My pre-script is:

for(int i = 0; i < Content.Rows.Length; i++)
{
Content.Rows[i].IsIgnoreLine = true;
}

string ct = Content.GetContent();
ct = ct.Replace("\0", ""); //Remove NULL-chars
ct = ct.Replace(".", "-"); //Fix date format to readable
ct = ct.Replace(",", "."); //Change decimal comma to point

string[] rows = ct.Split('
'); //Split into rows

//Parse rows into content, skip unnecessary rows
for(int i2 = 6; i2 < rows.Length; i2++)
{
Content.AddRow(rows[i2]);
}

Any idea?

Sergej



QuantShare
2016-10-22 04:26:39

  0

"Content.AddRow" will add rows to be parsed. Check the content of "rows[i2]" or check the separator.

Also, the "Split" may be wrong. Try with:
string[] rows = ct.Split('
'); instead



SergejSuperstock
2016-10-22 06:18:31

  0

*Edit: Deleted


SergejSuperstock
2016-10-22 11:56:22

  0

I cannot figure out, why QS is not able to parse the values. As I tested with an external file, everything should be fine now. Can anyone help me?

--> Settings:
Columns: Date, Skip, Skip, Open, Close, High, Low
Semicolon, DMY, Zero, -, 1, :, noting, HMS, Daily (left to right, bottom to top)

--> Pre-script:
using System.IO;

for(int i = 0; i < Content.Rows.Length; i++)
{
Content.Rows[i].IsIgnoreLine = true;
}

string ct = Content.GetContent();

ct = ct.Replace(" ", ";"); //Change "Space" between date and time to semicolon
ct = ct.Replace("\0", ""); //Remove NULL-chars, "backslash-zero"
ct = ct.Replace("\r", ""); //Remove CR-chars, "backslash-r"
ct = ct.Replace(".", "-"); //Fix date format to QS-readable
ct = ct.Replace(",", "."); //Change decimal comma to point

string[] rows = ct.Split('
'); //Split into rows, "backslash-n"

StreamWriter myStreamWriter = new StreamWriter(File.OpenWrite(@"C:\temp\myTestResult.csv"));

//Parse rows into content, skip unnecessary rows
for(int i2 = 6; i2 < rows.Length; i2++)
{
Content.AddRow(rows[i2]);

myStreamWriter.WriteLine(rows[i2]);
myStreamWriter.Flush();
}


--> QS output:
Conversion error: 22-01-2013;00:00:00;1-440;100.000;98.151;100.000;96.912
Conversion error: 23-01-2013;00:00:00;1-440;98.071;97.782;98.091;97.478
etc.

--> QS Log:
Error parsing column 'Date' --> Value: 22-01-2013;00:00:00;1-440;100.000;98.151;100.000;96.912
--> File : https://www.wikifolio.com/dynamic/de/de/invest/download?type=daily&dateFrom=01.01.2000&dateTo=21.10.2016&name=200775
Error parsing column 'Date' --> Value: 23-01-2013;00:00:00;1-440;98.071;97.782;98.091;97.478
--> File : https://www.wikifolio.com/dynamic/de/de/invest/download?type=daily&dateFrom=01.01.2000&dateTo=21.10.2016&name=200775
etc.

--> C:\temp\myTestResult.csv
22-01-2013;00:00:00;1-440;100.000;98.151;100.000;96.912
23-01-2013;00:00:00;1-440;98.071;97.782;98.091;97.478
24-01-2013;00:00:00;1-440;97.780;98.838;99.059;97.780
etc.



QuantShare
2016-10-24 04:50:03

  0

"rows[i2]" is a string not an array. You need to pass an Array to "AddRow" function.

Example:
Content.AddRow(rows[i2].Split(';'));



SergejSuperstock
2016-10-25 13:33:44

  0

Cool, its working now. Documentation did not make this clear to me. Thank you for your quick support!


No more messages
0




Reply:

No html code. URLs turn into links automatically.

Type in the trading objects you want to include: - Add Objects
To add a trading object in your message, type in the object name, select it and then click on "Add Objects"










QuantShare

Trading Items
Advancing-Declining-Unchanged volume for NYSE, AMEX and NASDAQ
Metatrader Historical Data Downloader
Quandl.com Universal Downloader
Historical EOD data Downloader for Delisted/Bankrupt Stocks
Brent and Crude Oil Continuous Futures Contracts - Intraday Data

How-to Lessons
How to automatically start a downloader every few minutes?
How to download EOD quotes for active and valid stocks only
How to download and use U.S. stocks earnings data
How to speed up watchlist and screener plug-ins when working with...
How to backup your databases (EOD, Intraday, Tick and Custom) and...

Related Forum Threads
Downloader - Login and Download a file
Problem with the "Individual stocks put and call volume" download...
Is the adjusted and unadjusted price downloader is broken?
Downloader doesn't download - why?
Fundamental data for canadian and mexican stocks

Blog Posts
How to filter quotes data from high and low price spikes
Backtesting a Strategy Based on Bond and Stock Index ETFs
How to predict and trade the stock market using pivot points
Intermarket Analysis - Correlation and Trading Strategies
How to create buy and sell trading rules based on News Data









QuantShare
Product
QuantShare
Features
Create an account
Affiliate Program
Support
Contact Us
Trading Forum
How-to Lessons
Manual
Company
About Us
Privacy
Terms of Use

Copyright © 2024 QuantShare.com
Social Media
Follow us on Facebook
Twitter Follow us on Twitter
Google+
Follow us on Google+
RSS Trading Items



Trading financial instruments, including foreign exchange on margin, carries a high level of risk and is not suitable for all investors. The high degree of leverage can work against you as well as for you. Before deciding to invest in financial instruments or foreign exchange you should carefully consider your investment objectives, level of experience, and risk appetite. The possibility exists that you could sustain a loss of some or all of your initial investment and therefore you should not invest money that you cannot afford to lose. You should be aware of all the risks associated with trading and seek advice from an independent financial advisor if you have any doubts.