How cloud data warehouses can give organizations a boost

We have conducted dozens of CDW transformations and have found that the best outcomes have these characteristics:

• Highly scalable: A well-designed CDW should be able to keep pace with evolving business demands. Consider a small company that initially deploys a relatively small CDW. As transaction volumes grow, it’s critical that the CDW effortlessly adjust to the new realities. Additionally, it’s becoming increasingly routine for organizations to employ CDWs from different vendors. Scalable architectures designed to integrate new services and effectively synchronize across these platforms represent the best way to provide sufficient performance in these kinds of heterogenous environments.
• Prepared for change: Transitioning to a new CDW usually means increasing your ecosystem’s complexity, reshaping your employee culture to adapt to the shift from on premises to cloud and imposing operational changes on your data consumers. In preparation, you should first assess your operating model well in advance to see whether it’s ready for change management and be primed to realign it to avoid potential disruption to business processes. Transparent communication channels – including extensive end-user training – are a big contributor to furthering adoption of the new technology.
• Cost aware: CDW pricing models can be very different from those presented by traditional on-premises data centers. For instance, most service providers offer hybrid pricing models based on various factors that include usage, performance and services. Since unexpectedly high on-demand CDW activity may dramatically increase your costs, it’s vital to prioritize your functional requirements and accurately forecast your anticipated workloads. It’s also smart to recognize that there are significant disparities between the cloud and on-premises cost models in terms of upfront and ongoing operational expenses. Finally, there can be large variations between how much time is needed to acquire on-premises infrastructure vs. deploying a CDW.
• Comprehensive security and governance: Each CDW vendor offers robust, built-in security functions and related advanced features, such as distributed denial of service protection, on-demand data masking and tokenization. Some cloud providers may also supply their customers with specifically tailored services aimed at satisfying regulatory compliance obligations, such as anti-money laundering monitoring. Despite these protections, the shared responsibility model imposed by CDW suppliers – whereby they assume the job of protecting their infrastructure and networks while you safeguard your resources – means that you’re never totally free of carrying out your security tasks. Additionally, if your organization operates different management tools and security protocols, it’s in danger of jeopardizing vital information. For example, the inherent lack of coordination among disparate clouds in a multi-vendor environment can magnify damage when an attack occurs. Adopting an enterprise-wide governance framework to orchestrate different services goes a long way toward boosting your overall security and regulatory compliance.

We blend well-defined processes and proven technologies to deliver our CDW approach. Our clients benefit from the resulting connected, streamlined and customized migration pathway. We follow a time-tested, multiphase approach when engaging with a client on an enterprise-grade CDW engagement. Major steps include:

Phase 1 – Discover and define

We kick things off by understanding your vision and intentions for the future target state. At this early date, we’re also assessing the current data landscape to identify gaps and optimization opportunities.

Phase 2 – Architecture design

For maximum clarity and consistency, we take the time to lay out the nomenclature that we’ll be using throughout the whole CDW undertaking. In preparation for later stages, we may also need to enhance the existing data model.

Phase 3 – Migration strategy

To guarantee the most streamlined experience, we define how we will transform your infrastructure and data, including testing the results. We also introduce plans to retire any impacted legacy systems.

Phase 4 – Data security 

We turn our attention to strengthening your information protection. We establish role-based data access controls, including tokenization and identity access management enhancement. We also determine how to most effectively store and safeguard any personally identifiable information while promoting privacy and regulatory compliance.

Phase 5 – Implement and deliver 

We’re now ready to develop and deploy the infrastructure that will underpin your CDW. Testing and validating performance and key functional domains are a major part of these responsibilities.

Phase 6 – Pilot programs

Defining a minimal viable product helps promote the new platform while gathering essential stakeholder feedback. We also migrate pilot program data sources.

Phase 7 – Migration 

We migrate your mass data sources and activate the consumption and semantic layers of the new CDW. With the replacement system in place, we can deactivate your legacy sources.

Phase 8 – Unlock next-generation capabilities 

Since CDWs evolve and improve over time, we actively seek ways to leverage your new data model to explore additional enterprise analytics possibilities. We’re also likely to enrich your existing reporting capabilities with real-time and streamlined processing features. This is also a good time to enable any advanced artificial intelligence or machine learning solutions you may want to incorporate.

Although each CDW mission is unique, applying a reliable methodology goes a long way toward delivering the kind of outcomes that our clients desire. Here are high-level summaries of three very different engagements:

Case study No. 1: using data to fuel business transformation

A credit union launched a strategic endeavor aimed at helping it become a more agile organization and better prepared to accelerate its growth. Leveraging data and deep insights driven by machine learning were to be major ingredients in this undertaking.

We configured a Snowflake implementation and associated user access services designed for seamless connectivity with other cloud-native resources. This also meant designing and implementing an integrated, cross-business domain data model, along with enabling automation via continuous integration/continuous deployment processes.

The client’s new, cost-optimized cloud environment – bolstered by assets such as cloud readiness assessments, leading practices from the client’s CDW vendor, and extensive dashboards and reports – was a major catalyst for the project’s positive outcome.

Case study No. 2: optimizing information management responsibilities

A financial services organization sought to streamline its data ingestion processes. The enhanced procedures would need to operate individually as well as in concert with others. The client also wanted to encourage standardization while strengthening its security protocols.

We designed and developed a scalable, multi-tenant platform that employed next-generation, metadata-driven technologies. The new approach featured data isolation with expanded information quality capabilities designed to quickly detect anomalies. We combined this with fine-grained security controls, including authenticating traffic and restricting data access based on business-driven policies for masking and data sharing.

The client greatly sped up the process of onboarding tenants and activating data management capabilities. Previously, setting up a new customer required between one and three business days to complete. The new CDW platform slashed this to less than an hour.

Case study No. 3: establishing an enterprise data hub 

A leading US bank with primary services in digital banking, mortgage loans and auto finance wanted to construct a single, central enterprise data hub. This would facilitate more consistent cross-channel customer identity while also improving timeliness and accuracy for critical enterprise responsibilities, including financial reporting and demand forecasting.

We began by performing a cloud readiness assessment, which included examining the client’s data infrastructure design, tools and governance obligations. This research was instrumental when we then created a cost-optimized cloud environment and migrated the information to it.

With the data now stored in the new platform, our client was able to model more than 3.5 million of its customers using deterministic and probabilistic identity resolution to help improve personalization. We also provided the client with statistical methods to enhance data quality and accuracy by more than 30% for its information management dashboards.

CDWs offer irrefutable advantages compared with on-premises installations. Enterprises that make the transition can put their systems into production more quickly, yet still reap significant rewards from the cloud’s scalability and data safeguards.

However, it’s important to never lose sight of the fundamental challenges that must be overcome when launching a CDW initiative. These include possible high costs due to incorrect configurations, a lack of trained staff wielding essential skills, daunting cultural adjustments for current employees, governance mandates and security vulnerabilities.

To improve your likelihood of success, it’s wise to select and then adhere to a proven methodology that recognizes the importance of delineating overall goals and objectives, establishing a solid roadmap and planning for continual evolution. We’ve found this to be the case for the dozens of successful CDW projects that we’ve conducted, and we’re happy to share what we’ve learned with you and discuss your own initiatives.

Summary

Adopting a cloud data warehouse initiative can bring benefits to organizations, but they must overcome an array of challenges to make the digital transformation successful.