Data migration from on prem systems to Snowflake reduces time to insights by 10X

Sigmoid optimized Snowflake's performance by implementing best practices for data migration, resulting in a 10X improvement in the efficiency of data pipelines.

Business Challenge

The customer provides financial technology services to banks, credit unions, securities brokers, mortgage, insurance, leasing, and financing organizations, as well as retailers. They were facing challenges with their legacy infrastructure on Hadoop in system administration, security, and disaster recovery. They wanted to modernize the current infrastructure by migrating to Snowflake and better manage merchants' transactional data including funds, lending, repayment, and retrieval among others.

Sigmoid Solution

Sigmoid developed the architectural guidelines derived from proven methodology for managing data on Snowflake. Best practices were implemented on data ingestion to Snowflake, data processing using suitable data warehouses, correct use of transient or temporary tables, and modularizing queries to improve the performance of data pipelines. By splitting the execution environment for the cloud data warehouse, it was possible to create better experiences for users such as faster query resolution times.

Download case study

Business Impact

The customer was able to significantly improve the performance of their data pipelines, with gains of up to 10X. Data management best practices on Snowflake led to real-time query resolution by reducing the time taken to process queries from 7 seconds to 2 milliseconds. Additionally, there were cost savings in running queries related to end-of-the-day merchant data reconciliation.