As US equity markets closed out 2013 at new highs, the future of equity research is facing significant change. With “price targets” being reset for many soaring social, cloud and big data analytics stocks let’s meet the new software analyst. But first, a little background.

Equity research has marginally evolved with investment styles and trading strategies over the past couple of decades. The days of primary fundamental research, particularly on the sell-side, faded long ago. Most analysts don’t have the gumption or the time.

Shrinking commissions and heightened regulatory scrutiny yield lower returns on investment, continuing a cycle of reducing research resources. The sell-side analyst role now has three principal components: 1) to provide access to company managements in their existing coverage universe; 2) to provide coverage for companies that are underwriting clients; and, 3) to provide “hot data points” – particularly for handicapping quarterly results. Buy-siders compete for management access and seek to combine these data points with their own findings to feed trading decisions.

Unfortunately, individual data points legally obtained and disseminated rarely move the needle in providing an adequate sample size on which to base an investment, no less a trading decision. For buy-siders, even aggregating data points from numerous analysts covering a particular sector or company does not provide a relevant statistical sample.

Limitations of today’s analytics

For example, let’s say a mid-sized publicly-traded technology company goes to market with a blend of 100 direct sales teams (one salesperson and one systems engineer per team) and 500 channel partners (mixed 75%/25% between resellers and systems integrators). Further, assume that these teams and partners are dispersed in proportion to the company’s 65%/35% sales mix between North America and international. How many salespeople and channel partners would an analyst have to survey to get an accurate picture of the company’s business in any given quarter?

If a typical sell-side analyst covers 15-20 companies (quintuple that for buy-side analysts), the multiplier effect of data points that an analyst would have to touch makes it humanly impossible to gather sufficient information. Moreover, with 50% of most tech company deals closing in the final month of a quarter, of which half often close in the final two weeks of that month, how much visibility can an analyst have?

Further, why would a company’s sales team talk to anyone from the investment community in the final weeks of a quarter when the only people they are interested in speaking with are customers who can sign a deal? Now consider that many companies throughout the supply chain have instituted strict policies in response to recent scandals to prevent any employee from having any contact with anyone from the investment community.

Even the best-resourced analysts lack the tools to correlate the data points he/she does gather to identify meaningful patterns for either an individual company or an entire sector. Finally, with shorter-term investing horizons and high-frequency trading dominating volume, how relevant are these data points anyway?

The big data approach to research

Stocks generally tend to trade on either sector momentum or overall market momentum. Macro news or events are far more likely to impact a sector’s movement, and therefore a stock’s in that sector. This includes volatility around quarterly earnings – which can run 10%-30% for technology stocks – because the majority of “beats” or “misses” are frequently impacted by macro factors. Excuses such as “sales execution” or “product transition” or “merger integration” issues are less frequent than conference calls would suggest. “Customers postponed purchases” or “down-sized deals” or “customers released budgets” or “a few large deals closed unexpectedly” are more likely explanations.

Now, major sell-side and buy-side institutions are trialing new software that leverages cloud infrastructure and big data analytics to model markets and stocks. Massive data sets can include macro news from anywhere in the world, such as economic variables, political events, seasonal and cyclical factors. These can be blended with company-specific events, including earnings, financings or M&A activity. Newer data sources, including social media, GPS and spatial can also be layered into models. Users can input thousands of variables to build specific models for an entire market or an individual security.

As with any predictive analytics model the key is to ask the right questions. However, the machine learning capabilities of the software will allow the system to not only answer queries but to also determine what questions to ask. Tech New Master

The advantages to both sell-side and buy side firms are significant. They include:

• Lower costs. Firms can avoid major technology investments by leveraging the scale and processing power of cloud-based infrastructure and analytics software. They can collect, correlate and analyze huge, complex data sets and built models in a fraction of the time and cost that it takes in-house analysts to do.
• Accuracy. Machine learning and advanced predictive analytics techniques are far more reliable and scalable than models built in Excel spreadsheets. Patterns can be detected to capture small nuances in markets and/or between securities that high-frequency trading platforms have been exploiting for years.
• Competitiveness. The software can make both sell-side and buy-side firms more competitive with the largest, most technologically advanced hedge funds that have custom-built platforms to perform analytics on this scale in real time. In addition to enhancing performance, the software can be leveraged to improve client services by making select tools available to individual investors.

Analysts become data scientists

The analyst skill set must evolve. They will still have to perform fundamental analysis to understand the markets they follow and each company’s management, strategy, products/services and distribution channels. And they will still have to judge whether a company can execute on these factors.

But to increase their value, analysts will have do statistical modeling and use analytics tools to gain a deeper understanding of what drivers move markets, sectors or particular stocks. Data discovery and visualization tools will replace spreadsheets for identifying dependencies, patterns and trends, valuation analysis, and investment decision making. Analysts will also need a deeper understand client strategies and trading styles in order to tailor their “research” to individual clients.

These technologies may well continue to shrink the ranks of analysts because of their inherent advantages. But those analysts who can master these techniques to complement their traditional roles may not only survive, but lift their va