How to use big data to profit from the stock market?

I have to admit I am avid believer that everyone should invest in the stock market. We are in the age of online discount brokerages and super fast connections for both wireless and wired. Combine that with companies taking away pension plans and allowing employees to manage 401K’s and more people working a contract basis and receiving a 1099 instead of a W-2. The average person will have to have some basic knowledge of how the stock market works and take control and actively manage their own retirement portfolio. We are now in the AG (After Google) age and with that access to information is only a search away. With so much data and companies and Internet Portals offering free financial data on almost every publicly traded company I will present how to use 3 simple techniques from big data to profit from the stock market and take control of your financial future whether you are techie geek or a novice computer person using these methods will make your money.

Method 1: Making Money from Market Mood or Market Sentiment Trading

One of the most profound facts is that hedge funds, mutual fund managers, and large banks spend lare amounts of money it is that you will have spend money from companies that provide data sets or plenty of data for investors. Companies such as factset, Capital IQ, and CRSP (Center for Research in Security Prices) provide data and datasets for large amounts of money but what about the average retail investor that does not have the capital or several thousand dollars to spend on this data as well as hire a data scientist or team of people to mine the data for them that they are purchasing??

There was a great gold rush during early 2000 until 2010 and from my hometown and other places across the United States and the entire World the basic investor sentiment was to purchase Gold or bullish Gold. I finally had enough capital in my 401K and disposable income to invest and I was ready to purchase a Gold mining stock company, GLD (Gold ETF), or another gold related stock. Before I purchased I happened to review a chart and see that Gold had basically increased over 2000% since its lows in 2000 and not only that every person I met was bullish Gold. Everyone wanted to purchase Gold at 2000 per ounce and felt that Gold would continue its rise. This was one of my 1^st investment lessons of investor sentiment and that usually when the crowd or majority is bullish then you may need to consider being bearish. I have heard Warren Buffett quote a trader from Goldman Sachs stating “Be greedy when people are fearful and be fearful when people are greedy”. There has been plenty of research to prove that market sentiment changes with major sporting events, weather, natural disasters, and news events. So how can a retail investor profit from it? One of the 1^st big data tools to determine market sentiment that I would recommend is a website http://tweetchup.com/

in which you can create a profile and use the search analytics tool and search different hastags for the sentiment of Twitter for a bullish or bearish sentiment of the market.  To give further reading on this topic three researchers have written an article named “Twitter Mood Predicts the Market” http://arxiv.org/abs/1010.3003. While twitter may be a popular social media tool I would take it a step further and use a tool or software that would let you search all the popular social media sites such as Facebook, linked, twitter, stocktwits,  and blogs below to determine the sentiment and trade according to that. Datasift (Datasift.com) allows anyone to input data from popular social media sources and website and output the data to your app or tool that you choose. This method may cost more money and you may need a certain level of technical knowledge but it is an affordable solution and worth considering if you have the additional time and money to trade from market sentiment. There are currently hedge funds and institutional investors that use this method as discussed in this article http://www.zerohedge.com/news/2013-12-12/worlds-largest-hedge-fund-uses-twitter-real-time. While this may not be the best method but it is a simple method to begin using to make money from market mood.

Method 2: Social Media Algorithm

Another method that I have found for investing with market mood and sentiment is to use the finsent website http://www.finsents.com/.  This website helps investors scan financial resources and content for the mood of the market. I would use this tool to find the mood of a particular sector such as real estate or commodity. Then you may develop your own algorithm from Quantopian.com and create

trading algorithm from that website. The figure above displays how investors can retrieve the information and then use an algorithmic engine to make a trading decision. This can powerful as you build your own trading algorithm based on market sentiment from big data software.

Method 3: Market Sentiment Trading Platform

Since hedge funds and institutional investors with large amount of capital are already mining through the web and using social media and blogging website and trading from the trend the latest tool will be the average retail investor to do the same. One Hedge fund company Derwent Capital has already developed a trading platform named DCM Dealer which has an Interface to allow retail investors trade on market sentiment from data from Facebook, Twitter, and other social media sites. The interface will help retail investors review the market sentiment and build and trade the market sentiment of the overall market or individual equities or sectors they may choose.

Conclusion:

These tools and methods are simple and should be viewed as only tool in a toolset to manage your stock or retirement portfolio. Of course you should do your own research and make sure to come up with additional methods. The reason for me writing this article is for the average retail investor to begin to take control of their financial future and use the same tools that the large hedge funds and banks use to make money from the stock market. I wish everyone reading this the best of luck investing.

Disclaimer:  This article intends to illustrate the use of big data for investing in the stock market. It is not intended to be used as investment advice. The readers are expected to apply their own discretion & judgement over and above these methods.

This article was submitted by David Avery Jr. as part of his application for Analytics Vidhya Apprentice programme. David is currently a Database Administrator and has been a contractor for 8 years now specializing in SQL Server and MySQL databases. He is proficient in SQL, Networking, and JAVA programming. In his spare time, he is also a adjunct for a couple of universities where he teaches Database Administration and networking courses

Images and references:

Gettyimages

http://tweetchup.com/#/keywords

DCM Dealer: http://mashable.com/2013/01/14/dcm-dealer/

http://www.zerohedge.com/news/2013-12-12/worlds-largest-hedge-fund-uses-twitter-real-time

https://www.quantopian.com/posts/market-sentiment-market-mood-finsents-signals-detection

If you like what you just read & want to continue your analytics learning, subscribe to our emails, follow us on twitter or like our facebook page.