Data collection automation for a leading UK-based investment firm

In a data-centric research landscape, staying competitive demands innovative solutions. Our latest case study delves into the challenges faced by a client heavily reliant on manual data collection from public news sources and forums. Discover how Netscribes collaborated with the client to revolutionize their data collection and processing. Join the data collection automation revolution and learn how businesses can upgrade their operations without compromising quality.

Challenges

The client’s core operations heavily relied on extracting information from public news sources and forums to support their research efforts. However, this process faced significant challenges:

• The data collection process was void of any automation, leading to a time-consuming and resource-intensive operation.
• Manual data collection introduced errors at various stages, affecting data quality.
• The manual approach resulted in prolonged data-gathering timelines.
• The combination of quality issues and extended turnaround times negatively impacted the client’s customer experience.

Approach and solution

To address these challenges, Netscribes collaborated closely with the client to identify critical data points and outline a comprehensive data collection automation strategy for each step in the process.

Data points to be collected –

Contacts |  Company profiles | Fund managers | Fund performance | Investors | Service providers| ESG profiles | Deal news | Company financials and more

Process – 

• Data source identification: Netscribes identified data sources for collection, including third-party aggregators, automating approximately 50% of this step.
• Data collection: The team employed web crawlers to extract content and metadata from identified sources, storing them in searchable indexed databases, with nearly 100% automation.
• Data categorization: Leveraging Natural Language Processing (NLP) models, events in articles were classified and tagged for further processing, achieving approximately 50% automation.
• Data identification: The classified events underwent Named Entity Recognition (NER) using an automated model, automating around 35% of this stage.
• Data capture: Human agents qualified the tagged data, entering them into a workflow system. Validation and rules were applied according to the front-end data format, automating roughly 40%.
• Data Quality Control (QC): The QC process verified the data and granted approval, with approximately 70% automation.
• Data update: Automatic data updates were initiated through API pushes to the backend of the firm’s platform, achieving full automation.

Results delivered

The implementation of data collection automation resulted in substantial improvements:

• 70% Reduction in turnaround time: The streamlined process significantly reduced the time required to collect and process data.
• Enhanced data quality: Automation reduced human errors, leading to a marked improvement in data quality.
• Improved CX: With quicker access to high-quality data, the client was able to offer a superior customer experience.
• Continuous refinement: The use of NLP modeling allowed for ongoing enhancements in automation, ensuring sustained efficiency gains.