Authors: Ornella Urso, Filippo Battaini, Giulio Raffaele
Integrating data silos and implementing an AI/ML platform remain key challenges for retailers in achieving contextual customer experience (CX) at scale. Globally, 54% of retailers plan to consolidate customer data and leverage advanced analytics.
A customer data platform (CDP) aggregates, analyzes, and activates real-time customer data from multiple sources. It automates data connections; extracts, transforms, and loads processes; stores data persistently; and leverages AI and ML analytics.
CDPs should involve all major stakeholders across an organization's functions, from marketing to the IT department. The CMO should work with the head of CX, chief privacy officer, chief digital officer, and CIO/CTO to better align data analysis with profitability goals.
Prioritize business use cases and estimate your ROI. Focus on identity management, customer consent, and privacy. Enhance CX capabilities through data-driven contextual customer experience, relying on consistent data quality and a scalable, flexible technology.
The worldwide customer data platform market is expected to grow around 50% faster than the marketing campaign management (MCM) market overall. Despite the economic slowdown caused by the COVID-19 pandemic, the CDP market is still expected to grow at a 10.3% compound annual growth rate (CAGR) to $1.2 billion in 2024, crossing the $1 billion mark in 2023, according to the latest IDC Market Forecast (Worldwide Customer Data Platform Forecast Update, 2020-2024: COVID-19 Cuts $3.0 Billion Over Five Years, IDC #US46704820, August 2020).
IDC's 2020 Global Retail Innovation Survey indicates that 54% of retailers plan to consolidate customer data and leverage advanced analytics.
This IDC PlanScape identifies the main drivers (why), foundational elements (what), key actors (who), and business outcomes (how) involved in enhancing customer experience that helps retail organizations better respond in real time to unpredictable and event-driven customer behavior.
"Nowadays, relationships with customers are more fluid. On one hand, customers are aware and conscious of data shared with preferred brands. On the other hand, retailers need an extended value chain to improve customer lifetime value," said Ornella Urso, senior research analyst, IDC Retail Insights. "Offering a frictionless, contextualized, and immersive shopping experience requires that each gear of the retail value chain – people, processes, and technologies – is aligned and synchronized with customer preferences. Knowing the customer is not just about cookies; it is about enhancing customer experience by engaging the customer based on solid trust. To do so, retailers need to break down existing data silos and implement customer data platforms to ingest, process, and analyze first-, second-, and third-party data (e.g., customer real-time contextual and historical data), relying on an identity management and regulatory compliance architecture."
The fast-changing COVID-19 context is leading more retailers into the next normal. Regardless of their maturity level, retailers must reach, engage, and retain customers. Integrating data silos and implementing an AI/ML platform remain key challenges for retailers in achieving contextual customer experience at scale.
Retailers often have to deal with large amounts of unstructured and fragmented data and information coming from different interfaces, as well as those coming from within and across their business functions. In most cases, information such as demographics, customer relationship management (CRM), loyalty program subscriptions, and outdated personal information continues to be disconnected or incorrect – and thus impossible to use at an optimal level. Risks associated with poor data quality extend throughout the retail value chain (such as marketing activity, delivery services, loyalty programs, promotions, and stock management), with consequences on business processes and decisions.
Data management capabilities support structured and comprehensive CX personalization initiatives and foster expansion into innovative CX personalization applications. However, it is important to distinguish CDPs from existing platforms such as data management platforms (DMPs) and CRM applications.
According to IDC, DMPs are designed to be advertising platforms that enable organizations to collect anonymous data (typically cookies and segmented audience IDs) and create groups of audiences for targeted communications through online advertising. DMPs store data for only up to 90 days, which equals the lifetime of a cookie. Meanwhile, CDPs can store structured and unstructured data from a wide range of internal and external sources on known and unknown contacts, including personally identifiable information (PII), and retain data for a longer period. This enables retailers to analyze customers' behavior and shape an accurate and contextualized customer profile in real time. Integrating a CDP with a DMP can retroactively link anonymized customers with contextualized customer profiles once they are identified, building the segment of one and delivering empathy at scale.
On the other hand, CRM applications serve to manage the entire life cycle of a customer – including the process of brand building, conversion of a prospect to a customer, and the servicing of a customer – and help an organization build and maintain successful relationships with customers (Worldwide Customer Relationship Management Applications Software Forecast, 2020-2024, IDC #US46534220, June 2020). Moreover, while CRM works only with known customers, CDP works with both known, recorded data and unknown data. This enables retailers to overcome the issue of dealing with anonymized data and collect customers' names and attributes.
A CDP acts as a customer data repository hub that ingests, connects, and aggregates data from other databases. Meanwhile, an enterprise data warehouse (EDW), a data lake (DL), or a Big Data platform (BDP) are mainly used only as data repositories. Through CDPs, retailers can exchange transaction data and customer profiles with EDWs, DLs, and BDPs.
By implementing a CDP, retailers can manage personal information as first-party data (information provided by the customer to the company), second-party data (data shared with and received by partners), and third-party data (from open data providers); structured and unstructured data; unknown and known data associated with records; and historical and real-time data (since a CDP stores both device ID or cookie ID).
CDPs aggregate data and convert it into the same language. By doing so, CDPs aim to create a single customer view for marketing and CX functions to define and coordinate a comprehensive engagement approach across all interfaces and interaction moments of the customer journey.
A customer data platform aggregates, analyzes, and activates real-time customer data from multiple sources. It automates data connections; extracts, transforms, and loads processes; stores data persistently; and leverages AI and ML analytics. As a result, a CDP is an essential part of an enterprise customer data ecosystem.
AI and ML analytics support retailers' ability to deliver personalized customer experiences by combining customer data with data from the extended value chain and providing insights on how to innovate with suppliers and partners. Retailers that focus on a real-time contextual customer journey model, following a Commerce Everywhere business model approach, aim to enable the definition, execution, and dynamic, real-time updates of customer journeys based on hyper-micro customer segmentation and contextual customer segmentation. They do so by leveraging AI to dynamically optimize customer journey options while fully automating the selection and delivery of individual customer offers across all available interfaces. Thus, data consolidation and data processing are fundamental steps that retail organizations are constantly facing to secure a connected and contextual customer journey.
Due to the complexity of data sources, IDC has developed a conceptual retail data architecture that aims to provide a holistic overview of possible data that retailers might collect or gain access to, and what constitutes the basis of such a retail customer data platform.
All three layers are fully embedded into an identity management architecture that captures and secures each customer consent moment, in view of regulatory compliance and personal privacy.
A CDP collects data from various acquisition points such as mobile apps, web, AR/VR, vocal assistants, stores, robots, humans, and connected products. It ingests and aggregates data from customers (personal data, CRM, payments, loyalty), products (order analysis, product attributes), inventory (EDW, order fulfillment, assortment), third parties (statistical and aggregated data), and second parties (DMP partners and suppliers). Identity management is central, handling unknown and regulatory data. AI/ML analytics drive customer data personalization, supporting marketing, customer analytics, loyalty teams, product managers, sales enablement, digital teams, and store associates.
Data layers in a customer data repository hub, such as a CDP, constitute a solid base of retailers’ data services. This includes data coming from products, customers, IoT commerce, connected stores, and third-party data accessed externally through dynamic partner ecosystems (peers, suppliers, IT vendors, etc.). When offered as an enterprise service to all customer-facing systems (as part of the retail commerce platform), historical customer data enables retailers to provide a continuous customer experience along retail.
The retail commerce platform integrates enterprise services (store operations, enterprise applications, supply chain, merchandise and assortment planning, price and promotion, finance and accounting, asset management, HCM, ERP), E2E security, customer experience services (contact discovery, customer journey personalization and loyalty, interface enablement), order fulfillment services (fulfillment optimization, networked KPI-based delivery execution, returns management), AI/ML analytics, commerce services (single commerce engine, order capture, order configuration, payment, delivery setup), content optimization services (content management, adaptive content distribution), development and integration services (API), data services (product and customer, IoT, external), and consumer services (store, mobile, web, connected product, robot, social, marketplace, ecosystem network).
In conclusion, the execution of CDPs enables retailers to collect and structure data into individual, centralized customer profiles; differentiate their offerings from competitors with a real-time contextual customer journey model; and, ultimately, improve customer satisfaction and increase customer lifetime value.
For retailers that have a companywide innovation culture or are planning to develop customer-centric leadership among C-level executives and managers, a customer data platform should involve all major stakeholders across the organization's functions, from marketing to the IT department. For retailers to get the full value from their CDP investments, it is critical that their data schema and specifications not be exclusively focused on a single department even if they will be initially deployed departmentally. Leaders from each department that produces or consumes customer data should be involved in the data specification process so that definitions and policies governing data sharing across interactions are designed on an enterprise framework. Use cases in each function should be considered so that access permissions, usage criteria, and data labels are clearly articulated and expansive enough to meet a growing portfolio of use cases around the business.
| Marketing | Sales/Commerce | Product | Service/Account Management | Support |
|---|---|---|---|---|
| Optimize return on ad spend (ROAS) | Automated lead distribution to direct and indirect interfaces | Optimize user experience and data valorization | Increase retention with enhanced churn analysis | Increase cross-sell/upsell opportunities |
| Increase online to offline conversion rates | Optimize offers, pricing, and discounting | Increase app installation, activation, and utilization | Greater insights for renewal planning and account expansion | Revenue-based resource allocation |
| Optimized customer relationship/ enhance loyalty programs |
Multidimensional scoring (by opportunity, account, offer, status, etc.) | Feature and enhancement request prioritization | Enhance and drive a contextualized customer engagement | Omni-channel customer support |
The innovation of retail business models necessarily passes through a deep and continued analysis of customer data. AI supports retailers in combining customer information with data coming from the extended value chain. It also provides them with insights on how to co-innovate with suppliers and partners in the direction of contextual CX personalization.
How you treat customer data is how you treat your customers. Compliance with current data privacy and security regulations is, therefore, a fundamental step that retailers should consider to effectively prevent the misuse of customer data and their identity management. Retail organizations that don't manage customer consent and data privacy will struggle in securing the end-to-end customer journey and in ensuring compliance and confidentiality. Focusing on identity management, organizations can secure customers' data, avoid data breaches, and deliver personalized experiences. This, in turn, enables retailers to become trusted partners with customers and to achieve mutual trust.
According to IDC’s 2020 Global Retail Innovation Survey, customer experience services remain a top priority for retailers that plan to invest in platform-enabling capabilities over the next 12 to 24 months. CX services enable the discovery of the customer context, real-time customer journey personalization (including loyalty), and customer interface enablement across voice, image, text, and AR. Companies can dynamically collect data from several consumer interfaces and data sets (CRM, DMP, EDW, DL, BDP, etc.), aggregating and transforming them by leveraging AI and delivering real-time contextual experiences. Therefore, having more customer data (both direct and contextual) and the adequate technology to appropriately employ that data helps retailers meet customer needs more effectively.
A higher maturity level of CX personalization is a differentiator for retailers that aim to engage new customers; it increases customer trust and drives loyalty. Over the past four years, the percentage of retailers moving toward real-time contextual customer journey models has increased from 2% in 2017 to 24% in 2020. Contextual and highly personalized interactions, enabled by AI and cognitive-driven technologies, can transform customer engagement, driving overall satisfaction and increasing trust. This means retailers should adequately invest in enabling technologies in relation to their current level of CX personalization and need to make the difference in the market.
Despite the great amounts of data that retailers are now able to collect daily across the enterprise’s functions and from third-party partners, 32% of global retailers are still struggling to gather and select useful data from existing and new data sources. Thus, it is fundamental for retailers to maintain comprehensive, current, consistent, and high-quality customer data assets, harvested from the enterprise, partners, and social media sources and made available to CRM. Through a CDP, retailers can consolidate data silos and gain access to a central repository for customer data, transactions, demographics, preferences, attributes, and loyalty data. Converting customer interactions into enduring and trusted relationships is an important step for retailers in becoming empathetic enterprises, retaining loyal customers, engaging new ones, and optimizing and increasing conversion rates.
CDPs enable retailers to better respond to the fast-changing market trends and business agility and resiliency requirements to compete in the next normal. Relying on the same foundations as the retail commerce platform, CDPs respond to the need to avoid data silos while merging online and offline data across interfaces and departments by leveraging AI/ML analytics. Retailers should pay attention to IT vendors’ specific retail business use cases that CDP enable retailers to achieve, as well as the technical requirements associated with the implementation of a CDP. These include speed, scalability, connectivity, architecture, operational costs, identity management, and regulatory compliance.
The next normal challenges retailers’ persisting legacy systems and data silos. It continues to be of paramount importance to architect data-driven customer experience, as reaching and retaining loyal customers remains a major goal for retail organizations. This is true well beyond the current contingencies. Retailers need to respond to customer needs at speed and scale, adapting their personalized offerings to customers’ requests.
Predictive analytics can effectively work only by ingesting and processing cleaned and precise data. This, in turn, requires a comprehensive and coherent CX strategy widely shared and fueled by each retail organization's functions across the different steps of the customer journey.
Retailers that are considering investing in incremental technologies such as CDPs should set clear objectives and steps for execution:
International Data Corporation (IDC) is the premier global provider of market intelligence, advisory services, and events for the information technology, telecommunications and consumer technology markets. IDC helps IT professionals, business executives, and the investment community make fact-based decisions on technology purchases and business strategy. More than 1,100 IDC analysts provide global, regional, and local expertise on technology and industry opportunities and trends in over 110 countries worldwide. For 50 years, IDC has provided strategic insights to help our clients achieve their key business objectives. IDC is a subsidiary of IDG, the world's leading technology media, research, and events company.
IDC Italy
Viale Monza, 14
20127 Milan, Italy
+39.02.28457.1
Twitter: @IDCItaly
idc-insights-community.com
www.idcitalia.com
This IDC research document was published as part of an IDC continuous intelligence service, providing written research, analyst interactions, telebriefings, and conferences. Visit www.idc.com to learn more about IDC subscription and consulting services. To view a list of IDC offices worldwide, visit www.idc.com/offices. Please contact the IDC Hotline at 800.343.4952, ext. 7988 (or +1.508.988.7988) or sales@idc.com for information on applying the price of this document toward the purchase of an IDC service or for information on additional copies or web rights. IDC and PlanScape are trademarks of International Data Group, Inc.
Copyright 2021 IDC. Reproduction is forbidden unless authorized. All rights reserved.