Revitalising a Custom-Built NLP Solution With Generative AI for Archer Technologies LLC
Firemind developed a Generative AI solution that identifies differences between compliance obligations and defined control standards, across thousands of data inputs. Our POC focused on SEC regulations to start, prior to a production ready state. The application of ML to this problem is expected to provide a novel capability to Archer’s core platform whilst using Generative AI to differentiate Archer’s compliance and risk management offering.
Revitalising a Custom-Built NLP Solution With Generative AI for Archer Technologies LLC
Firemind developed a Generative AI solution that identifies differences between compliance obligations and defined control standards, across thousands of data inputs. Our POC focused on SEC regulations to start, prior to a production ready state. The application of ML to this problem is expected to provide a novel capability to Archer’s core platform whilst using Generative AI to differentiate Archer’s compliance and risk management offering.
At a glance
Archer Technologies LLC help organisations manage risk in the digital era, uniting stakeholders, integrating technologies and transforming risk into reward.
Challenge
Creating a Generative AI solution that addresses the core issues with the prior solution, including document ingestion, change management and reduced false positives.
Solution
Develop a Generative AI solution that identifies the differences between compliance obligations and defined control standards.
Services Used
Amazon Bedrock
AWS Lambda
Amazon S3
Outcomes
15x faster regulation sorting
100% compliance accuracy
5x reduction in human error
Business challenges
Revitalising their previous ML solution
In 2020, Archer developed a custom ML solution that automatically processed regulatory changes, making it easier for customers to keep control standards up to date.
External factors caused Archer to de-prioritise the project and focus on core business priorities. Today, Archer customers are requesting this solution to available once again.
Since the initial solution was developed, NLP techniques have advanced considerably, thanks to the capabilities of Generative AI. As a result, Archer would like to revisit the core capabilities of this solution and replace their custom-built NLP solution with Generative AI.
“Firemind delivered beyond our expectations, ensuring a robust, Generative AI focused infrastructure, that enabled super fast classification and compliance checks. We couldn't be happier with the outcome and look forward to pushing towards a production state.”
Julian Hirsch, Senior Engineer — Archer Technologies LLC
Solution
Building a PoC that redefines risk classification
We needed to allow Archer Technologies LLC to send documents via an AWS hosted user interface, to then be inferred by a process utilising LLMs on Amazon SageMaker and Amazon Bedrock. The output of the inference would then be stored on Amazon S3 for Archer Technologies LLC to extract RAG (red, amber, green) information for each submission.
There were 5 main stages to this project:
1. Firemind worked with Archer Technologies LLC to define a number of prompts for up to 10 controls. Archer Technologies LLC could then use this template to update further controls, ready for inference by the LLM. This will be in the format of a single column CSV file.
2. Archer Technologies LLC could then upload a zip file directly into an Amazon S3 bucket, using either the APIs, CLI, or AWS console provided.
3. AWS Lambda could then unzip the files, triggering step function processes.
4. The step functions could then clean the data, splitting compliance rows into chunks, running mapping methods and extracting defining information from the output. This information could then be mapped to an output CSV file, providing a RAG status, an explanation of the SOP’s compliance level and an explanation of why the LLM provided each RAG status.
5. Finally, the output could be saved in Amazon S3, where Archer Technologies LLC can access it via the APIs, CLI or AWS console provided.
High level UX
This project needed not only a robust architecture, but a reliable and easy to use front-end design. Our design team worked alongside Archer Technologies LLC's design department to make a solution that will simple to use and effective in it's use of prompts and data controls.
Faster compliance
Significant speed increase when looking at compliance level in RAG statuses, when compared to human resource.
Reducing human error
The PoC showed a quick advancement in accuracy and correct classification. Ensuring significant savings by automating the time needed to complete the checks.
Why Firemind
“It was really about finding an AWS Partner that understood our ethos and our values, and echoed the same."
Archer Technologies LLC and Firemind were brought together by Amazon Web Services (AWS) as a designated delivery partner of the AWS ML Fast Start Program. As an outcome from the ML Fast Start Program, Archer Technologies LLC had defined a need to develop a Generative AI solution that identifies differences between compliance obligations and defined control standards.
As specialists in ML and generative AI, Firemind was the best choice of partner.
1920%
Workflow speed increase
1,000 documents in 12.5 minutes compared to 4 hours by human operator
88%
Keyword accuracy
for leading categories within data modelling & training
Added value
We were also able to take all stakeholders on a journey concerning how Cloud Adoption and Machine Learning would benefit their customers and their business continuity, all in a 12 week timeline from start to finish.
Get in touch
Want to learn more?
Seen a specific case study or insight and want to learn more? Or thinking about your next project? Drop us a message below!