Click here to Login

RSS feeds transformation

Updated on 2009-07-25

QuantShare is capable of downloading RSS feeds and converting them into CSV files for easier parsing. This task is accomplished by the downloader and is fully automated. It loads the specified RSS feeds, converts them into CSV content, then applies the CSV parsing and inserting rules to add the data into your local databases.

The RSS feed comes generally in a format like the following one: (This is an example of an element (item) of the yahoo news feed for the Google ticker symbol)

<title>This Week Dumbest Stock Moves (at Motley Fool)</title>
<description>The description</description>
<pubDate>Fri, 24 Jul 2009 19:37:46 Etc/GMT</pubDate>

After reading and parsing the RSS feed content of the Google stock, the downloader will convert every 'item' tag into a row. The 'item' node children will be separated by a string or a character. (You decide what string or character to use as a separator).

The result will be: (assuming we used a semi-colon ';' as a separator)

This Week Dumbest Stock Moves (at Motley Fool) ; ; The description ; Fri, 24 Jul 2009 19:37:46 Etc/GMT

The next step will be to define the CSV parser settings, adding columns and assigning a database field to each one of them. For example, in the above line, the forth column must be associated with the 'date' field of the database of your choice. The transformed content is then parsed and referenced database fields are filled.

Common problems and solutions:

How to deal with nodes content that contain more than one line?
Multiple lines nodes’ content will be transformed into a single line text; the 'new line' character will be transformed into the following text '||'. You can write a Pre-Script formula to transform this '||' into a 'new line' character, leave it as it is, or perform any other transformation.
Example: The Google finance RSS feed, that downloads news for stocks, has multiline nodes’ content.

What if a node’s content contains the separator I choose?
The separator you choose is removed from every node’s content. So whatever separator you pick, your content will always be correctly parsed.

What if the description or title contains html tags?
All html tags are removed from nodes' content, only visible text is kept.

no comments (Log in)

QuantShare Blog
Search Posts

Recent Posts

Create Graphs using the Grid Tool
Posted 1270 days ago

Profile Graphs
Posted 1375 days ago

Previous Posts

RSS feeds transformation
Posted 5419 days ago

Stocks: Market Capitalization
Posted 5421 days ago

Ranking System Engine
Posted 5423 days ago

How to deal with StockTwits data
Posted 5433 days ago

Trend following systems
Posted 5436 days ago

Working with the formula editor
Posted 5438 days ago

Optimization of trading rules
Posted 5449 days ago

Volatility and trading systems
Posted 5451 days ago

More Posts


Create an account
Affiliate Program
Contact Us
Trading Forum
How-to Lessons
About Us
Terms of Use

Copyright © 2024
Social Media
Follow us on Facebook
Twitter Follow us on Twitter
Follow us on Google+
RSS Trading Items

Trading financial instruments, including foreign exchange on margin, carries a high level of risk and is not suitable for all investors. The high degree of leverage can work against you as well as for you. Before deciding to invest in financial instruments or foreign exchange you should carefully consider your investment objectives, level of experience, and risk appetite. The possibility exists that you could sustain a loss of some or all of your initial investment and therefore you should not invest money that you cannot afford to lose. You should be aware of all the risks associated with trading and seek advice from an independent financial advisor if you have any doubts.