About
SCANOSS provides a fully open-source SCA (Software Composition Analysis) platform, enabling developers to create compliant code from the start. Their platform indexes and scans over 11TB of public open-source code from sources like GitHub, Bitbucket, and GitLab.
Challenge
Due to its vast dataset and need for extremely high IOPS (Input/Output Operations per second), SCANOSS needed to reduce scanning times while optimising storage and lowering costs. Their existing Azure-based setup was struggling to deliver both the speed and efficiency required.
Solution
Firemind partnered with SCANOSS to architect and test a new AWS-based storage solution capable of handling their intense high-transaction workloads. Instead of traditional database services, we implemented Amazon S3 combined with Amazon EC2 I3 instances, which are optimised for high IOPS, low-latency workloads. A 40GB sample dataset was migrated for performance benchmarking, reducing scan times from 79 minutes to around 30 minutes - a 65% time reduction - while delivering notable cost savings.
Following successful testing, the architecture was prepared for full migration to AWS. Additional AWS services, including Amazon VPC for secure networking and Amazon CloudFormation for infrastructure automation, ensured scalability and repeatability. Firemind worked closely with SCANOSS to train their team on optimised storage practices, ensuring confidence in maintaining high-performance scanning as they fully transition from Azure to AWS.
Services used
- Amazon VPC
- Amazon CloudFormation
- Amazon EC2 I3 Instances
- Amazon S3
Results
- 65% faster codebase scanning (79 mins → 30 mins)
- Significant cost savings from optimised storage and reduced processing times
- Improved retrieval speed for massive 11TB dataset
- Full migration readiness proven with high IOPS AWS architecture
- Developer enablement through AWS EC2 I3 training and knowledge transfer
