Sentiment Analysis: Potential And Limits Of Opinion Fueled Trading

The article was written by Anna Latini, a Financial Analyst at I Know First.

“Since the beginning of times, short-term traders can only rely on just 2 factors to predict returns: price and volume, Therefore the addition of sentiment as another short-term factor is like adding the airplane to cars and boats.”- Ernest Chan 

Sentiment Analysis


  • How Sentiment Analysis Could Impact Trading
  • Accern: The Start-up That Leverages Social Media For Stock Market Insights
  • Limits of Sentiment Analysis
  • How Is I Know First Algorithm Different?

How Sentiment Analysis Could Impact Trading

First of all, let’s start by defining what we mean by sentiment analysis. Sentiment analysis (sometimes known as opinion mining or emotion AI) refers to the use of natural language processing, text analysis, computational linguistics, and biometrics to systematically identify, extract, quantify, and study affective states and subjective information.

sentiment analysis

Among other things, sentiment analysis is now being introduced to analyze the textual information contained in the news items in order to assign a ‘sentiment score’ to each article that might bring an impact on the stock’s price. Creating an aggregation of these sentiment scores from multiple news related articles or posts within a certain timeframe has been proven to be predictive of the stock’s future performance.

The reasons why sentiment analysis is becoming more and more popular can be described as followed:

First of all, it is very time-consuming to analyze news and derive a picture of a market segment. AI can certainly cut times and provide much more efficient and precise answers. Furthermore, although traditional theory suggests that it is impossible to beat the market based on publicly available information, studies show that this is not necessarily true as sentiment analysis seems to actually challenge the efficient market hypothesis. This is nothing new, as one of the most cited studies, conducted by Cornell University researchers, came out in 2010 already. It found public mood states to be predictive of changes in DJIA closing values. The results indicated the accuracy of DJIA predictions could be significantly improved by the inclusion of specific public mood dimensions but not others. The study had an accuracy of 87.6% in predicting the daily up and down changes in the closing values of the DJIA and saw a reduction of the Mean Average Percentage Error by more than 6%. Although the definition of prediction used in this particular study has been criticized by some, who suggest the accuracy rate should have been lower; there have been several other studies pointing at the same correlation between public sentiment and stock returns.

News might be unpredictable, however, there are very early indicators of market sentiment that can be extracted from social media which can effectively predict changes in various economic and financial indicators.

 Accern: The Start-up That Leverages Social Media For Stock Market Insights

AI has the potential to turn social media into a fundamental input for making trading decisions.

This is the idea behind New York based start-up Accern that employs artificial intelligence to analyze different social media platforms and make consequent market predictions. For the moment the software analyses Twitter and Tumblr but the company is planning on adding also Facebook and Reddit to the mix.

Traditional analysis, everything from reports to market trends issued by players like Bloomberg and Reuters is also incorporated into the process and in total the company monitors data from 300 million websites.

The start-up already works with big clients like IBM  and with around 12 quant hedge funds, funds where machines totally handle investments and no human intervention is needed.

Accern runs internal test that showed it could help increase investor returns by as much as 90% over several years. However, results have not been proven yet by reports from its clients so we will have to wait and keep an eye on the data. It is interesting to consider that Accern does not only give buy or sell recommendation. The algorithm displays a variety of options attached to a certain degree of risk versus benefit using a long term perspective. Such options also derive from past reactions to similar events.

Accern is not the only company to use social media, however. Dataminr for instance already offers to financial institutions dynamic insights from Twitter. The company goals and means are fairly similar to Accerns. Datamir is also headquartered in New York, has more than 250 employees and received funding for $180 million by leading venture capital and growth investors.

The key to including social media in stock market analytics is using machine learning to assess each account’s reliability, combined with the post’s sentiment and content.  For example, a tweet from a few verified CEOs has more weight than lots of faceless Twitter accounts. The software tools that tackle news articles and industry reports are equally complex and are also aided by language processing tools. These two elements together provide a more nuanced, almost human insight on the market.

sentiment analysis

Source: Accern website

Limits of Sentiment Analysis

Since this type of analysis is not based on quantitative data, the first challenge to be overcome is accurately translating non-homogenous, subjective data from a qualitative to a quantitative form. According to a research conducted by the ECB, this is one aspect where sentiment analysis has come short in the past. It is possible to fine tune sentiment measurement, however, even so, other limits still stand.                                                                                                           Relying heavily on market sentiment can be dangerous as it can fuel bubbles and lead investors into making poor investment choices. Furthermore, as soon as sentiment analysis becomes mainstream and people are aware of its functioning, twitter and other social media will not be “pure” anymore as people could start using them to play the market. It would be especially easy for influential players to manipulate market trends as sentiment analysis does not weight everyone’s opinion equally.

This said sentiment analysis can offer a valid tool especially for investors.  Yet, it can not be the only driver of investment choices. An approach that considers the results of sentiment analysis together with signals by an algorithm like I Know First’s could produce a much more reliable forecast of market movements as it would take into account a more variegated set of data.

sentiment analysis

How Is I Know First Algorithm Different?

I Know First algorithm uses purely quantitative data as inputs. I Know First developed a prediction system that uses artificial neural networks that are self-learning, flexible, and adaptive to the capital markets. The program is able to establish, modify and delete relationships between different financial assets. It is self-learning hence the resulting formula is constantly evolving, as new daily data is added and a better machine-proposed “theory” is found.

The system gives the prediction as a number, positive or negative, together with a wave chart that predicts how the waves will overlap the trend, thus at what point to enter or exit the trade. There is no human derived assumption as all inputs are empirical. The only moment a human factor is involved is in building the mathematical framework and initially presenting to the system the “starting set” of inputs and outputs. For more detailed information on I Know First algorithm applications, I suggest reading this article written by Tali Soroker.

The absence of any human input puts this algorithm at the antipodes of sentiment analysis algorithms and makes it less subject to errors to the extent there is no interpretation of subjective data required.

Both models have clearly their strengths and weaknesses, however, since both can bring precious insights to investors I believe it will become increasingly important for companies to be able to incorporate the two approaches. Creating a system that is able to take into consideration both empirical data and information coming from traditional and social media would give investors a complete view on markets trends, enabling them to take better decisions.