Big Data in Investing: Hopeful Prospects


The article was written by David Nie, a Financial Analyst at I Know First.


Big Data in Investing


  • What is Big Data?
  • Use of AI and Technology
  • Downsides
  • Big Data and the Industry
  • How does I Know First play a role?
  • Conclusion

What is Big Data? 

As more and more attention falls on investment and finance as sources of power and wealth, investors will continue to search for leverage over their competitors. In the past, the edge has revolved more around powerful technology and smarter human resources. However, the increasing availability of technology and human resources to more firms has made these factors less relevant.

Starting recently, data has really risen in importance as it has become more accessible through internet and technology. Hence, investors now are considering new forms of data as a means to gain the upper hand over competitors.

As defined by industry analyst Doug Laney in 2001, data is now more prevalent on three different scales: volume, variety, and velocity (as shown in Figure 1). Together, the three V’s have been used as a conventional description of Big Data and its various aspects.

Figure 1

Use of AI and Technology

Along with the surfacing of Big Data, artificial intelligence (AI) has begun to emerge as a hot research subject. AI is the idea that computer systems can perform tasks that normally require human intelligence. With AI, researchers and analysts can process significantly more data and derive trend that are not easily detectable. One major sector of AI that is being used alongside Big Data is machine learning. Machine learning involves the development of computer systems that can change independently when given new data. With this technology, data scientists will be able to make probabilistic models of the future and correlations between trends that are more precise and realistic.

The technology and artificial intelligence (or machine learning) produced information is currently something that will allow investors to be more and more confident in their financial and business choices. Therefore, I agree that we are advancing towards the point of mass usage.


Fundamentally, there are limitations. In the forefront for investors, Big Data and machine learning research are very expensive. Big Data is also a very young research area, so lack of expertise is a major issue. Thus, many investors are still wary of the ultimate significance of data analytics. However, we are making significant progress in the field of machine learning and algorithmic processing. The future is working towards understanding how Big Data works and how to process it. It is also increasingly becoming a subject of education and a field of study in modern colleges and universities. Many companies in fact have already made great leaps in the field and have began to revolutionize the way Big Data plays into the economy.

Big Data in the Industry

In 2016, the words Machine Learning and AI flooded the market. This represents the rise of a ‘new stack,’ as Matt Turk coins it, that involves the widespread use of machine learning and artificial intelligence across the business landscape. 2017 itself marked a big increase in IPOs for many Big Data startups.

Many digital efforts are helping the industry progress forward. Take for example Hadoop, an open-source data storage and concurrent process running framework released in 2011. Emerging from an area of the economy that is financially inaccessible, Hadoop provides a scalable and economical alternative to commercial big data packages. The Hadoop big data analytics market is projected to be worth 40.69 billion USD by 2021. However, the same problems of lack of expertise and talent in the field apply – creating a shift in educational and skill focus, as well as a shift in software and hardware usage and development.

Overarching the entire industry of big data as a whole, we see consistent positive trends for market value and revenue for companies like Tableau Software, Google, IBM, HP, Alteryx, Datos IO, Amazon, and many more. These companies currently focus on data analytics (AI) and solutions to processing big data, and all play a role in the growth of the big data market. The big data market estimates to surpass 200 billion USD by 2020.

How does I Know First play a role?

Founded in 2010, I Know First emerged before the trend and has been using machine learning to predict the market for 7 years already. It is cited as one of Bank Innovation’s “5 Israeli Startups You Should Be Watching,” and will continue to be an active competitor in the market. Founded by Yaron Golgher and Dr. Lipa Roitman, whom has personally done over 20 years of AI research, the company produces accurate predictions for stock market behavior for investors.

I Know First does this by taking in Big Data from the past 15 years of stock trading and analyzing it with machine learning algorithms. These algorithms can then process the current state of the market and offer predictions based on past market behavior. The company continues to backtest and adapt its algorithm to be more accurate and robust.

I Know First’s business model is currently built on two-tiers: Tier 1 is for personal and small-scale use, and Tier 2 is for corporal and large scale use. Tier 1 provides users with a stock forecast that displays long/short positions, predictability, and signal for thousands of securities (shown on the right). This allows users to personally to filter through stocks and make smarter financial investments. On the other hand, Tier 2 offers larger organizations a day-to-day investment plan. The algorithm will automatically provide the corporation with its optimal long and short positions to take daily.

I Know First’s performance has been positive and stable. Take for example the recent analysis of its performance. Investors in the 1st tier have received 73% returns in the GSOL stock over the past 2 weeks. The algorithm also provided 100% accuracy in long positions taken in Fundamental High Peg stocks in the past month. Long term investments prove especially worthwhile; investments in KEM have had 400% returns and those in TGB and AMD made over 100% over the past year.


Big Data offers significant hope for the future and lots of potential in usage. For example, algorithmic trading and investing is becoming increasingly influential in the market. I think that with the incoming technological advancements and focus on data analytics, the field of Big Data will eventually have a huge impact on the economy and investment, and more investors will use the technology to optimize their strategies. It is something worth investing in now.

I Know First is working alongside many other machine learning focused companies to develop accurate algorithms to predict future behavior. Utilizing these technical assets will allow consumers to be more confident in their choices and make more financially sound investments and purchases.

I Know First

I Know First, Ltd. is a financial technology company that provides daily investment forecasts based on an advanced, self-learning algorithm. Thus, the company’s algorithm predicts over 3,000 securities (and growing). Thus, it has capabilities to discover patterns in large sets of historical stock market data.

The underlying technology of the algorithm is artificial intelligence. The company’s product bases itself on machine learning and incorporating elements of artificial neural networks and genetic algorithms. Moreover, the algorithm generates daily market predictions for stocks, commodities, ETF’s, interest rates, currencies, and world indices for the short, medium and long-term time horizons.

For more information, visit I Know First.

To subscribe today click here.