Striim Team

Organizations across various industries require real-time access to data to drive decisions, enhance customer experiences, and streamline operations. A leading home improvement retailer recognized the need to modernize its data infrastructure in order to move data from legacy systems to the cloud and improve operational efficiency. To achieve these goals, the retailer partnered with Striim to support its data modernization and real-time integration efforts.

About the Retailer 

A leading home improvement retailer with thousands of stores across North America generates annual revenue exceeding $150 billion. Serving both DIY customers and professional contractors, the retailer offers a vast range of products for home improvement, construction, and gardening. Known for its customer-centric approach and expansive product offerings, the company has maintained its leadership position in the industry for decades.

Challenges 

The retailer’s legacy data infrastructure presented significant hurdles, preventing the company from achieving its modernization goals. These challenges stemmed from a complex and fragmented data environment, which included:

• Siloed Data Sources: The retailer’s on-premise databases were spread across various locations, creating silos that made it difficult to consolidate and manage data effectively.
• In-House and Third-Party Solutions: The retailer relied on a combination of in-house developed tools and third-party software. This patchwork of solutions led to inefficiencies, as different systems were not always compatible or easy to integrate.
• Complexity in Data Replication: Moving data between platforms, particularly from legacy systems to newer ones, was a time-consuming and resource-intensive process. This made it difficult for the company to support critical initiatives like supply chain optimization and migration to the cloud.
• Real-Time Data Limitations: The existing infrastructure lacked the ability to ingest and process data in real-time, making it hard for the retailer to stay agile and responsive to market demands.
• Scalability Challenges: As the company grew, its data volumes increased dramatically. The legacy systems were not built to handle this scale, creating bottlenecks and limiting the company’s ability to manage data efficiently.
• Multiple Teams Using Different Tools: Various departments, including migration and supply chain teams, used different tools and processes to manage their data. This lack of standardization added complexity and slowed down decision-making processes.

These issues underscored the need for a more efficient, scalable, and unified approach to managing the retailer’s data infrastructure.

Solution

To address the complexity and inefficiencies of its legacy data infrastructure, the retailer sought a robust platform that could simplify the migration process and provide real-time data integration across its operations. 

The goal was to consolidate data replication efforts and improve supply chain efficiency by utilizing modern cloud infrastructure. After evaluating options, the retailer partnered with Striim to leverage its real-time data streaming and low-code/no-code integration capabilities.

• Striim’s platform enabled the migration of data from legacy Oracle and PostgreSQL databases to Google BigQuery.
• Using Striim’s low-code/no-code capabilities, the retailer streamlined the migration process, reducing the burden on internal resources and cutting costs.
• Striim’s real-time data integration capabilities played a vital role in optimizing supply chain operations.
• Timely, pre-processed data delivered by Striim ensured that reporting and logistics systems could optimize operations, such as configuring truckloads based on store orders.
• The platform met real-time SLAs and performed data transformations and validations on the fly, further simplifying processes.

Outcome

After facing significant challenges with its legacy data infrastructure, the retailer partnered with Striim to completely transform its approach to data management and integration. By migrating critical on-premise databases to Google Cloud and unifying its replication and migration efforts into a single platform, the retailer achieved substantial improvements in operational efficiency, scalability, and agility. These enhancements have allowed the company to optimize its data infrastructure, enabling it to better respond to evolving market demands and maintain a competitive edge in the retail industry.

Key results include:

• Unified Data Platform: Through Striim, the retailer successfully consolidated its fragmented migration and replication processes into a single, unified platform. This eliminated the need for multiple tools and reduced the complexity of managing data across various systems, improving overall operational efficiency.
• Migration to Google Cloud: Critical on-premise databases were seamlessly migrated to Google Cloud, enhancing the retailer’s ability to scale operations and support large volumes of data with greater ease. The migration to the cloud infrastructure enabled the retailer to benefit from more flexible, scalable computing resources.
• Improved Scalability: The modernization effort significantly enhanced the retailer’s ability to handle growing data volumes. With improved scalability, the company can now manage and process vast amounts of data more efficiently, which is essential for its expanding operations and growing customer base.
• Real-Time Data Integration: Striim’s real-time data streaming capabilities allowed the retailer to ingest and process data in real time. This empowered the company to make quicker, data-driven decisions, enabling faster responses to market dynamics and customer demands.
• Operational Efficiency: By modernizing its data infrastructure and integrating real-time data streaming, the retailer was able to reduce operational costs. The transition to a microservices architecture also improved system performance and reliability, resulting in smoother workflows and a more streamlined supply chain.
• Cost-Effectiveness: By moving to a cloud-based infrastructure and consolidating its migration efforts, the retailer reduced its reliance on legacy systems and lowered resource allocation for maintenance, which resulted in significant cost savings.
• Positioned for Future Success: The retailer’s newly modernized, agile, and cost-effective data infrastructure positions the company for continued growth and success. With its scalable cloud environment and real-time data capabilities, the company is well-prepared to adapt to future industry changes and remain competitive.

A nonprofit educational healthcare organization is faced with the challenge of modernizing its critical systems while ensuring uninterrupted access to essential services. With Striim’s real-time data integration solution, the institution successfully transitioned to a cloud infrastructure, maintaining seamless operations and paving the way for future advancements.

About the Nonprofit Educational Healthcare Organization

This nonprofit educational healthcare organization is committed to providing students with the knowledge and skills needed to succeed in the medical field. Serving thousands of students, it offers a variety of programs designed to prepare individuals for careers in allied health. The institution prioritizes student success by delivering high-quality education, supported by a robust infrastructure that ensures access to essential resources and services. Through its mission-driven approach, the institution plays a vital role in meeting the growing demand for healthcare professionals.

Challenge

This non-profit educational healthcare organization is navigating a dual challenge: migrating its core Student Information System (SIS) to a modern Azure SQL Server infrastructure while maintaining seamless data integration with their on-premise SQL Server databases. With student data central to daily operations and long-term outcomes, real-time data replication between the cloud and legacy systems ensures continuity and accessibility across platforms.

However, while the SIS migration was a significant step forward, the institution’s on-premise SQL Server systems remained vital. These legacy systems were deeply embedded into the institution’s infrastructure, supporting critical applications for student services. The challenge was not just migrating to the cloud but ensuring that the on-premise systems, still housing essential services, could continue to operate seamlessly and in real time with the cloud-based SIS.

This setup presented several technical hurdles. The reliance on SQL-based integrations had already caused performance bottlenecks, particularly around the API-driven data capture required for student inquiries and real-time updates. 

Without a solution to ensure uninterrupted access to both systems, the institution risked compromising student satisfaction, potentially leading to operational delays, downtime, and an overall negative student experience. Thus, the migration needed to ensure minimal disruption while maintaining the integrity and availability of critical data.

Solution 

In response to this challenge, the institution sought a partner that could help them achieve their dual goals: enabling cloud migration while supporting continued access to legacy on-premise systems. After evaluating various options, they selected Striim for its real-time data integration and streaming capabilities.

Striim’s solution was particularly suited to address the institution’s unique needs. Through Striim’s platform, real-time data capture and integration between the cloud-based Azure SQL Server and on-premise SQL Server systems were facilitated with minimal latency, ensuring that both systems remained in sync at all times. This was crucial for guaranteeing uninterrupted access to student records, class schedules, and other key services.

A key component of the solution was Striim’s in-memory processing capability. By leveraging this technology, Striim was able to efficiently capture, process, and transform data in real-time, reducing the reliance on custom-built integration solutions. This not only reduced the institution’s costs but also simplified the entire process, minimizing the need for ongoing development and maintenance efforts. With Striim, the organization could confidently migrate its SIS to the cloud while maintaining seamless data flow between the cloud and legacy on-premise systems.

Moreover, the integration allowed the institution to maintain critical student-facing applications, such as portals for class registration and transcript requests, without experiencing downtime. This real-time synchronization provided a stable environment that improved the student experience during a period of significant technological transition.

Results 

The partnership between Striim and the nonprofit educational healthcare organization resulted in several tangible benefits that went beyond ensuring a smooth cloud migration. Striim’s real-time data integration not only ensured operational continuity but also created opportunities for future growth, enhancing the institution’s ability to leverage data for more advanced use cases.

Real-Time Data Access:
Striim’s platform enabled immediate access to student, faculty, scheduling information, eliminating delays that had previously hindered the institution’s ability to serve its students. This real-time access provided more responsive services, allowing students to receive up-to-date information at any time, enhancing their overall experience.

Improved Response Time:
The seamless integration of real-time data also improved the institution’s ability to respond quickly to inquiries from prospective students. As a result, response times to student inquiries were significantly shortened. This quicker response fostered better communication between prospective students and admissions staff, creating a more positive experience for applicants.

Increased Conversion Rates:
The operational efficiency gained through Striim’s data integration helped the institution streamline its processes, and can result in improved conversion rates for prospective students. With faster access to accurate, up-to-date information, administrative staff were better equipped to assist prospective students in their decision-making process, ultimately increasing enrollment rates.

Seamless Integration of Systems:
Striim’s real-time data streaming and in-memory processing ensured that critical systems across both the cloud and on-premise environments remained fully synchronized. This seamless integration was particularly important for student-facing and administrative functions. By maintaining up-to-date, synchronized data, the institution ensured that students and staff had continuous access to the information they needed without disruption.

Foundation for Future Initiatives:
Perhaps most importantly, the nonprofit educational healthcare organization’s new cloud-based infrastructure, empowered by Striim’s real-time data integration, provided a strong foundation for future innovations. With the flexibility of real-time data streaming and a scalable cloud environment, the institution is now well-positioned to explore advanced analytics and AI-driven insights. This can lead to further improvements in student services, operational efficiencies, and decision-making.

 

Morrisons, a leading UK-based supermarket chain, is modernizing its data infrastructure to support real-time insights and operational efficiency. By embracing advanced data integration capabilities, Morrisons is transitioning to a more agile, data-driven approach. This shift allows the company to optimize processes, enhance decision-making, and ultimately improve the overall customer experience across its stores and online platforms.

About Morrisons

Morrisons is one of the UK’s largest supermarket chains, with over 100 years of experience in the food retail industry. Proudly based in Yorkshire, it serves customers across the UK through a network of nearly 500 conveniently located supermarkets and various online home delivery channels. With a commitment to quality, Morrisons sources fresh produce directly from over 2,700 farmers and growers, ensuring customers receive the best products. Dedicated to sustainability and community engagement, Morrisons continually invests in innovative solutions to enhance operations and improve the shopping experience.

Challenge 

Morrisons set out to modernize its data infrastructure to achieve five key goals:

1. Elevating Customer Experience: Creating a better shopping experience for customers.
2. Loading to Google Cloud: Transitioning to Google Cloud and leveraging Looker for enhanced reporting capabilities.
3. Accessing Real-Time Data: Shifting from batch processing to real-time data access, enabling faster decision-making and improved operational efficiency.
4. Enhancing Picking Efficiency: Morrisons sought to streamline their online picking process by improving stock visibility across depots and warehouses. 
5. Improving On-Shelf Availability: Ensuring products are consistently in stock and accessible to customers.

To meet these goals, the team needed to move away from their legacy Oracle Exadata data warehouse and strategically align on Google Cloud. This involved transitioning their data to Google BigQuery as the new centralized data warehouse, which required not only propagating data but also ensuring real-time access for better decision-making and operational efficiency. Moreover, prior to this transition, Morrisons never had a centralized repository of real-time data, and only ever had batch snapshots delivered from its disparate systems.

“Retail is real-time. We have our online shop open 24/7, and we have products moving around our distribution network every minute of every day. It’s really important that we have a real-time view of how our business is operating,” shares Peter Laflin, Chief Data Officer at Morrisons. 

In order to accomplish this, Morrisons needed a tool that could connect their separate systems and seamlessly move data into Google Cloud. Striim was selected to ingest critical datasets, including the Retail Management System (RMS), which holds vast store transaction data and key reference tables, and the Warehouse Management Systems (WMS), which oversee operations across 14 distribution depots. The integration of these systems into BigQuery in real time provided critical visibility into product availability, stock levels, and core business metrics such as waste and shrinkage. Most importantly, Morrisons needed this mission-critical data delivered in real time. 

“We’ve moved from a world where we have batch-processing to a world where, within two minutes, we know what we sold and where we sold it,” shares Laflin. “That empowers senior leaders, colleagues in stores, colleagues across our logistics and manufacturing sites to understand where we are as a business right now. Real-time data is not a nice to have, real-time data is an absolute essential to run a business the scale and size of ours.” 

Morrisons sought to move away from their existing analytics suite and leverage Google Looker for their reporting and analytics needs. This meant they had to regenerate all existing reports that previously ran on the Exadata platform, aligning them with the new Google Cloud infrastructure. Striim played a critical role in centralizing their data in BigQuery and delivering it in real time, enabling Morrisons to power their reporting with fresh insights. This transformation is key to achieving their goal of a more agile, data-driven operation and supporting future business initiatives.

Solution 

Morrisons now leverages Striim to connect disparate systems and ingest critical datasets from their Oracle databases into Google Cloud, using BigQuery as their new centralized data warehouse. They required a solution that could seamlessly load data from multiple sources while providing real-time access through BigQuery, and Striim provides this. 

Striim plays a pivotal role in ingesting two core databases: the Retail Management System (RMS) and the Warehouse Management System (WMS). The RMS, a vast dataset containing store transaction tables and key reference data, requires efficient data transfer to minimize latency, and Striim ensures that this high volume of data is processed seamlessly.

Striim also ingests data from all 14 distribution depots, which are connected through 28 sources in the WMS. This integration provides real-time visibility into stock levels, enabling ‘live-pick’ decision-making by revealing what stock is available, where it is located, and at what time. Backed by real-time intelligence, this capability accelerates business processes that were previously reliant on periodic batch updates. As a result, Morrisons can optimize the replenishment process and ensure that shelves remain well-stocked, ultimately improving overall efficiency and increasing customer satisfaction.

Striim’s real-time data delivery powers Morrisons’ reporting transformation as they rebuild all reporting within Google Looker. By centralizing and accelerating the flow of data into BigQuery in real time, Striim enables faster, actionable insights that drive operational excellence and future business initiatives. “My team felt that Striim was the only tool that could deliver the requirements that we have,” shares Laflin.

Outcome 

By leveraging Striim to transition from batch processing to real-time data access, Morrisons has significantly enhanced their ability to track and manage three critical key performance indicators (KPIs): availability, waste, and shrinkage. With access to faster, real-time insights, executives can more effectively identify risks and implement strategies to mitigate them, ultimately leading to improved operational decision-making and better performance across the organization. This shift allows Morrisons to optimize their processes and drive positive outcomes related to these key metrics.

“Without Striim, we couldn’t create the real-time data that we then use to run the business,” shares Laflin. “It’s a very fundamental part of our architecture.”

The move towards real-time data has allowed Morrisons to identify that their shelf availability has notably improved, ensuring that products are consistently in stock and accessible to customers. Best-ever on-shelf availability in December 2024 boosted customer satisfaction, marking a significant milestone for Morrisons. As a result, they are beginning to uncover the full range of benefits that this transformation can bring, including enhanced inventory management and reduced waste.

From the customer perspective, better shelf availability translates into happier shoppers, as they can find the products they want when they visit stores. This improvement not only fosters customer loyalty but also positions Morrisons to compete more effectively in the marketplace, ultimately driving growth and enhancing overall customer satisfaction.

View Case Study

In today’s enterprise landscape, ensuring high availability, scalability, and disaster recovery is paramount for businesses relying on continuous data flow and analytics. Striim, a leading platform for real-time data integration and streaming analytics, offers multi-node deployments that significantly enhance redundancy while delivering enterprise-grade capabilities for mission-critical workloads. This blog explores how Striim’s multi-node architecture supports these objectives, providing enterprises with a robust solution for high availability, scalability, and disaster recovery both as a fully managed cloud service, or platform that can be deployed in your private cloud and on-premises environments.

This blog explores how Striim’s multi-node architecture supports these objectives, providing enterprises with a robust solution for high availability, scalability, and disaster recovery.

Multi-Node Architecture: A Foundation for Enterprise Resilience

At the heart of Striim’s mission-critical platform is its multi-node architecture. Multi-node deployments allow Striim to operate across several interconnected servers or nodes, each handling data processing, streaming, and analytics in tandem. This distributed architecture introduces redundancy, ensuring that even if one node fails, other nodes can continue operations seamlessly. This approach is essential for disaster recovery, high availability, and fault tolerance.

1. Increasing Redundancy and Supporting Scalability

Redundancy is vital in distributed systems because it ensures that multiple copies of data and processing capabilities exist across nodes. Striim’s multi-node deployment increases redundancy by replicating workloads and data across several nodes. This means that in the event of a failure, another node can immediately take over, minimizing downtime and preventing data loss.

Additionally, Striim supports horizontal scalability. As data volumes grow—whether due to business expansion, increasing IoT devices, or heightened customer interactions—additional nodes can be added to the cluster to distribute the processing load. This ensures that the system can handle increasing demand without performance degradation, maintaining the ability to process millions of events per second across a distributed cluster.

2. High Availability Through Node Redundancy and Failover Mechanisms

For business-critical workloads, any downtime or data loss can have serious consequences. Striim addresses this concern by delivering high availability (HA) through node redundancy and automatic failover mechanisms. In a multi-node deployment, each node holds redundant copies of data and processing logic, ensuring that if one node fails, another can take over instantly without interrupting data flow.

Striim’s built-in failover automatically shifts workloads from a failed node to a functioning one, maintaining continuous service for real-time applications. This is critical for systems that demand high uptime, such as financial transactions, customer-facing dashboards, or logistics monitoring. Furthermore, Striim guarantees exactly-once processing, ensuring data integrity during node transitions and preventing duplicate or missed data events.

To provide a simple, declarative construct for node management and failover, Striim offers Deployment Groups which represent a group of one or more nodes with its own application and resource configurations. You can deploy Striim Apps to a Deployment Group, and that Deployment Group governs the runtime and resilience of the application. 

3. Disaster Recovery with Multi-Region and Cross-Cloud Support

In addition to failover, Striim’s multi-node deployment enhances disaster recovery (DR) by replicating data and services across geographically distributed nodes or across clouds. Enterprises can configure active-active or active-passive DR setups to quickly recover from catastrophic failures. By distributing nodes across multiple regions or clouds, Striim ensures that if one region experiences an outage, another can take over seamlessly, ensuring business continuity.

Striim’s cross-cloud capabilities offer additional flexibility, allowing organizations to distribute their infrastructure across different cloud providers. This architecture ensures resilience even in the face of regional outages, ensuring rapid recovery and reducing the risk of data loss. Additionally, Striim’s Change Data Capture (CDC) ensures that data is continuously synchronized between nodes, keeping all data consistent and up-to-date across the entire system.

Integrating Multi-Node Capabilities with In-Memory Technology

To provide real-time data streaming and analytics efficiently, Striim relies heavily on in-memory technology. Striim’s architecture allows for data to be cached in an in-memory data grid, enabling rapid data access without the latency of disk I/O. However, ensuring all nodes can process this data without time-consuming remote calls requires a tightly integrated design.

Striim’s multi-node deployment ensures that all system components—data streaming, in-memory storage, and real-time analytics—operate in the same memory space. This eliminates the need for costly remote calls, allowing for rapid joins and analytics on streaming data. By leveraging in-memory processing across a distributed cluster, Striim ensures that the system remains both highly performant and scalable, even under high data loads.

Security Across Nodes and Clusters

As enterprises scale their data processing across multiple nodes and regions, maintaining security becomes increasingly important. Striim addresses this need by employing a holistic, role-based security model that spans the entire architecture. Whether it’s securing individual data streams, protecting sensitive data in motion, or managing access to management dashboards, Striim provides comprehensive security across all nodes and processes in both Striim Cloud and Striim’s on-premise Striim Platform.

This centralized approach to security simplifies the task of managing access controls, especially in distributed systems where data and processes are spread across multiple locations. Striim’s role-based model ensures that all security policies are consistently applied across the entire system, reducing the risk of vulnerabilities while maintaining compliance with industry regulations.

Conclusion: Simplifying Enterprise-Grade Data Streaming

Striim’s multi-node deployments provide enterprises with a powerful, scalable, and resilient platform for real-time data streaming and analytics. By increasing redundancy, ensuring high availability through failover mechanisms, and supporting disaster recovery with multi-region and cross-cloud configurations, Striim enables businesses to maintain continuous operations even in the face of unexpected failures.

With Striim, enterprises can focus on deriving insights from their data without the need to invest in complex infrastructures or develop intricate disaster recovery strategies. Striim’s platform takes care of the complexities of distributed processing, in-memory analytics, and security, ensuring that business-critical workloads run smoothly and efficiently at scale.

By offering a unified solution for real-time data integration and streaming analytics, Striim empowers businesses to meet the demands of today’s data-driven world while maintaining the resilience and agility necessary to thrive in a competitive environment.