Testimonial
Through their company data feeds, Contify has provided us with an outstanding service for the past seven years. Contify identifies and sources information which is vital on an ongoing basis for many of our clients to make critical business decisions and strategy. I am convinced that the quality of Contify’s work has made an important contribution to the competitive effectiveness of the many large organisations we serve.
– Mike Gates, CEO, MarketWatch
About the Company
Industry
Energy & Utilities
Headquarters
United Kingdom
Employees
200 – 500
Founded
2000s
Business Challenge
MarketWatch, a UK-based market research firm, provides decision support and strategy research to clients in the energy domain. It monitors more than a 1000 companies for business updates and intellectual property for industry analysis. This was a labor and cost intensive process given the number of companies to track on a daily basis. MarketWatch approached Contify to explore automation of monitoring so that its teams could focus on higher value tasks of analysis and recommendations.
The Solution
We configured Contify’s Market Intelligence Platform to deliver custom feeds to MarketWatch.
This included:
- Sourcing: Set up a system to track the websites of 1000+ companies for unique business updates. Integrated the EPO’s API to track patent filings.
- Processing: Configured text mining algorithms to weed out content that wasn’t relevant for MarketWatch’s client, such as stock calls, CSR activities, etc.
- Delivery: Developed a custom program to create headlines and summaries for the updates so readers could easily understand them.
- used a potent competitive intelligence tool to monitor market trends and rival activity in real time. To produce thorough insights, this program analyzes a variety of data sources, including financial reports, social media posts, and press releases. Decision making and strategic planning are improved by users’ ability to simply monitor and compare key performance metrics, product launches, and market strategies throughout their industry.
Methodology
There were three key challenges that Contify solved while developing this market intelligence solution for MarketWatch:
- Developing and Maintaining Website Scrapers: Of the 1000+ companies, several hundred had dynamic web pages. But MarketWatch needed updates only for select segments that its clients operated in. To do this, we developed custom scrapers that fetched information from relevant sections of websites. Further, we put together a system for maintaining page integration in case the structure of a webpage changed, which happens frequently. With this process, MarketWatch never misses an update on companies they track. Here’s an example from GE – one of the tracked companies. MarketWatch needed to get updates only from the ‘Oil & Gas’ section of the GE Newsroom and not from other sections such as ‘GE Capital’.
Removing Irrelevant content: MarketWatch’s clients were not interested in updates about stock calls, CSR, philanthropic activities of companies, news on sponsorship of sports, or dated conference calls. Using Contify’s custom modules, we easily tweaked our algorithm to identify such updates and reject them from the news data feed. Here’s another example: while tracking Siemens, our system removed updates that did not meet the requirements while letting others pass.
Custom processing of patent filings: MarketWatch’s clients needed a specific type of patent filings of companies. A simple extract using API integration wouldn’t suffice. So we added an additional layer of filtering to fetch patent filings with predefined keywords in the title. Another challenge was the overtly scientific language of the patent filings. To understand what the patent was about, each document link needed to be opened. To address this, we developed a module to generate logical headlines and summaries. Here’s an example.
