Downstream data

“I have given up that I will ever find an ideal trade promotion solution, ” J&J presentation, Consumer Goods Technology Event, June 2010

They crowded together in stifling heat at the Roosevelt Hotel.  A group of 42 consumer product executives gathered to discuss Trade Promotion Management (TPM) in the Consumer Goods Technology (CGT) Share Group. It is a hot topic that become hotter as the day progressed.

Turning up the heat

Procter & Gamble (P&G) kicked off the session sharing insights on their global TPM project.  The goal is to consolidate over 50 applications for 30 countries. The project streamlines, and improves global processes.  The project genesis? It started when multiple regions for P&G could not meet the internal Sarbanes Oxley assessment.  The focus of the presentation at the session was project management:  project ownership, decision making, and global process deliverables. 

P&G is operating at a different level than the industry.  When the room was asked how many companies are currently working on the implementation of a global TPM project, no hands were raised. It is proof that for most companies, TPM is still a VERY regional process with multiple systems. 

What was boiling?

To prepare for the session, the group was given a list of potential topics.  When push came to shove, the group wanted to discuss three topics:

-IT Architecture:  Is there is a preferred system for TPM?  The answer is no. In this discussion, the room came to life.  Energy radiated as the group shared stories, anger and frustration.

 In the industry, TPM deployments are all over the map. Companies have multiple systems, many implementations and cannot identify a clear technology leader.  Traditional Customer Relationship Management (CRM) approaches under-served the market in consumer products.  When asked what systems were currently deployed?  The most widely deployed system is Oracle followed by SAP.  (There were eight companies with Oracle architectures, five using SAP-based infrastructure, four using CAS TPM, and one with Microsoft-based custom solution.)   

Oracle and SAP users agreed on similarities between the two vendors:  slowdown of innovation, fragmented direction without consistent leadership, and the lack of an acceptable user interface.  Both sets of users discussed the need to build a front-end to overlay over the architecture to improve ease of use.  Then they laughed and raised a question.  “Why should they have to build an overlay architecture to improve ease of use?

Both SAP and Oracle customers expressed frustration on how to spur development and provide leadership to improve the situation. The group agreed that the Oracle and SAP stories had amazing similarities with no ideal solution for the industry for a global deployment. Companies using best-of-breed solutions expressed similar frustrations.

-Not alot of O in TPM.  While the subject of optimization in TPM had the highest interest for the group, fifteen years after market introduction, less than 10% of companies in the room use optimization techniques intrade promotion management . The issues include the lack of manpower, expectations (the recognition that the output is directional versus absolute), ownership and the lack of process clarity. 

There is no clear technology winner.  The solutions deployed for optimization are also all over the board.  One company had deployed Oracle/Demantra, one had deployed ProMax in Australia, one had deployed DemandTec, one had M-Factor, one had Synectics, one was working in pilots with CAS 8 and one had built a custom solution.  One participant stated that business process outsourcing was not an option because they could not outsource the chaos.

-Few use Shopper Insights: Of the group, less than 8% are using shopper insights to make decisions.  Of this group, eight are looking at shelf virtualization, three are optimizing based on store clusters and one company is using test and learn techniques. In short, very little is currently being done by the group to tie shopper insights to TPM decisions.

My take:

The market is still largely a Trade Promotion Management(TPM)-as opposed to a Trade Promotion Optimization(TPO)–market. What seems so simple is still a long way away.  Companies desperately want a better user interface to improve trade coordination.  They are equally frustrated by Oracle and SAP.  The lack of success in bringing this to market has stalled the adoption for new technologies. They are disappointed that no platform vendor has brought a usable solution to market that easily combines TPM and TPO.

However, there are several trends that will transform this market.  Retailers are pushing consumer products companies to change their processes from the outside-in.  The greatest change is happening in the account team structures where retailers are asking for price and trade deals together.  This pressure from retailers to the sales teams will gradually change the processes within consumer products companies, but the adoption of systems using shopper insights and predictive analytics will happen slowly.

There will evolve and Software as a Service (SaaS) will continue to grow: 

  • A Bake-off is Coming:  There will be a race between Oracle and SAP to right the ship and improve TPM usability.  Oracle will defend the front office using its surround ERP message, and SAP will push an integration value proposition.  Companies are frustrated with both vendors.  Advancement will only happen if these two large vendors have large ears and small mouths.  Current groups like ASUG for SAP and the Oracle Customer Advisory Group have been largely ineffective because of structure, membership and vendor commitment.  Now is the time to re-frame the discussion, to actively listen and leave the quota carrying sales person at the office.
  • Predictive Analytics that Use Downstream Data and Shopper Insights will happen:  The techniques by Softwre as a Service (SAAS) vendors like  Applied Predictive Technologies, M-Factor andDemandTec to use external data within test & learn scenarios will gain receptivity as more and more companies use gain ROI. This will be forced because the rate of adoption of these techniques by retailers using Software as a Service technologies like APT, DemandTec, Predictix, and Revionics is gaining steam in retail.  The battle lines are being drawn.
  • Redefinition of Forecasting:  There will be a traffic cop emerge to own baseline forecasting. This is a pain point for every company that I speak to.  The redefinition of these processes is not as easy as data integration.  It requires a steward or over-lay organization to model demand shaping factors together, share baseline demand and provide insight into the organization on where they are getting true value from demand shaping activities.  The group thought  that this role could be played by finance or supply chain; but not by marketing or sales.  Increasingly I see inquiries on how to improve baseline forecasting. The most progress against this goal is happening with companies deploying IRI, M-Factor, SAS Demand-driven Forecasting or Terra Technology MDS.  Four very different solutions deployed by four different groups with the same goal.

Time to condense?

It is time for the consumer products company to ask themselves some hard questions:

Why do they not know baseline demand?  Why do they over-predict trade deals? Why are they not better at pricing?  Why are they not using shopper insights?  Why are they not taking advantage of test and learn capabilities?  Can they afford to not be good at these processes when it represents 14-22% of revenue?

I think not.  The answers lie in change managementand restructuring reward systems.  They need to be enabled by technologies, but it is not a technology project.  It starts with an internal audit.  The sales teams are incented on sales volume. Marketing is driven by market share.  Supply chain is driven by production volume. Who is looking after the customer?  And, serving as the check and balance on the systems?  According to the group, the answer is no one.  90% of the group responded that they would like to see an organization that crosses sales, marketing and supply chain to serve as the traffic cop: the single point of truth.

Sales teams are getting the most pressure from retail for deals.  The retail pressure will get greater.  Retailers are better at pricing and using shopper insights that consumer products manufacturers.  The gap is growing.  The sales teams are trying to keep up.  They are using the most advanced tools, but there is no check and balance.  Companies struggle with how to tie this work to corporate demand planning.  It requires a rethinking of the process from the outside-in and challenging traditional paradigms and reward systems.   

All these issue contributed to the group getting lathered up to talk TPM.  But, before we throw the baby out with the bath waterand scald the technology vendors, I think that it is important to hold up a mirror and ask how do we organize to help technology companies better serve the true need.  How do we forge a bond to help consumer products companies redefine their processes to be more responsive and accurate for customers?  It may take a hot New York minute to hit the flash point to drive change; but it is clear, it is an industry issue.

Is this the Future of Downstream Data?

by Lora Cecere on March 29, 2010 · 5 comments

It started as simple sales reporting. It is no longer simple.  In 2005, there were five consumer product leaders actively using downstream data (retail inventory and sales data).  Today, over 80% of consumer products companies greater than 1 billion in revenue are redefining work processes to use it.  The space is murky. It is not easy, and it is evolving. However, the deeper teams explore the usage of the data, the more excited they get.  This has been FUN to watch.

One of the things that you get to do as an analyst is name things.  We get to put OUR stamp –usually a cool three-letter acronym—on new technologies, processes and trends.  (At least, we think that it is cool.)  I am one of the mother’s of the Demand Signal Repository (DSR) term. It was originally defined by Kara Romanow, now Executive Editor at Consumer Goods Technology (CGT). I was an early collaborator with Kara.  It took shape and form from my fingertips as I wrote about the usage of downstream data in consumer value networks.  I have followed the market for the past six years.  Kara and I like to joke, that we were both mothers in the genesis and market acceptance of the term DSR.

A DSR is a repository of demand information. Demand information comes in many different forms—orders, shipments, syndicated data, point of sale information, warehouse withdrawal information from retailers, customer panel groups—and needs to be used by many different roles within the organization—sales for account reporting, marketing for promotional/new item acceptance and market share analysis, supply chain for forecasting and out-of-stock sensing, and R&D for new product launch insights.  The problem is that everyone wants it in a different form—frequency, granularity, attributes—and the data processes of cleaning, harmonizing and synchronizing are messy requiring a strong understanding of the data.

Help me Get it Right

So, as one of the mother’s—some would even disdainfully say “muther“—of the term DSR, I have been thinking about the evolution of this market and the market drivers, and I wanted to get some community input.  I am writing a report on market evolution and wanted to get community input on the evolution of the technologies.  (One of the exciting aspects of Altimeter Group’s research model is open research with the community.)  So, I welcome your feedback, do you think that these are the right trends?

Right Trends?

2010 will be the year of predictive analytics.  The DSR is not the end state.  The value of downstream data is the USAGE of the data into new business processes/work streams.  Whether it is Vision Chain’s new sensing of out-of-stocks, or Terra Technology’swork on short life cycle product sensing.  2010 is the year of predictive analytics.  Look for new applications to evolve.  I predict exciting launches in the area of price and promotion compliance, market basket analysis, shopping patterns, damage, and category analytics.  I find the convergence of loyalty data, point of sale information and geo mapping technologies very exciting to give live representations of market out-of-stocks, customer demand, and near real-time sensing of market trends.

In parallel, we will see market convergence of the technologies.  There are just too many sales reporting applications in the market, and predictive analytics vendors will need a database structure to enable insights from harmonized, disparate data sources (E.g. Nielsen’s TDLinkx product, orders, shipments, syndicated data, point of sale, warehouse inventory levels, retail inventory levels, etc).

It won’t just be about modern trade.  Hopefully, by now, it is clear to everyone that there has been a step change in data sharing by retailers in North America and Europe.  No, not everyone is sharing data; but, the data that is being shared gets more significant, and better quality, each year.  Customers that are working at getting the data now have  70% of North American and 30% of the European grocery markets.  (The secret is knowing how to ask for it.)

As companies use downstream data for modern trade, they will use the techniques to build demand networks to tackle the challenge of emerging markets.  The bullwhip effect of distributor relationships is just too painful. Consider the differences in table 1 for the food industry.

Table 1:  Bullwhip effect

Supply Chain Type Demand Latency from Shelf to Order Order Cycle Time Manufacturing Cycle
Modern trade for warehouse distribution 10-14 days 3 days 10-20 days
Emerging markets 40-48 days 1 day 30-40 days
Food service 24-35 days 2-3 days 30-40 days

 

Leaders will build distributor networks (similar to Anheuser’s Budnet) as part of the infrastructure to capture market share in the evolving markets.

Differentiation will come from enrichment and the design of the information layer.  To serve multiple roles and to enable new discovery, the secret sauce is the design of the information layer.  It is not a traditional Master Data Management (MDM) problem. Instead, it requires flexible data assembly and quick data parsing.  I believe that we will see the use of Search Engine Optimization (SEO) technologies like Endeca evolve to help companies solve this problem. (I saw some interesting evolutions of this concept in India past week.)   In parallel, new content—store demographics, in-store shopping data, panel data—will evolve.  These two elements will enable the true POWER from the new predictive analytics.

SAAS, License, Cloud.Initial forays into the usage of downstream data will be deployment neutral:  a true toss-up between license and Software as Service (SaaS) models.  However, as data enrichment and advanced predictive analytics evolve, the DSR will come behind the firewall. Similarly, as social media turns into social commerce, an information layer will evolve in the cloud for the value chain.  This value chain information layer will enhance not replace the enterprise DSR.

Figure 1: Downstream Data Evolution

Social Commerce will Power the Tipping Point.  As power shifts from the retailer to the shopper, social media technologies will power social commerce business processes (the ability to buy, return, shop on mobile devices based on reviews, price and inventory levels). Channels will blur as we enter the hype cycle of social commerce.  To power these applications Point of Presence (where is the shopper) will combine with Point of Sale (POS) to track the success of social CRM. The amplitude of the hype cycle will be immense—a mini dot.com era—but, it will be the tipping point for the usage of downstream data and the design of outside-in value network.

 Evolution. Do I have it Right?

So, what do you think? Do I have it right? Will it look like figure 1?  Let me know your thoughts.  Your input will serve as foundational input to refine the models for the report that I am writing on downstream data technologies in August.

Until then….  The Supply Chain Shaman is happy to be back from a very fruitful trip to India, and will be busy with research and advisory calls this week on the east coast of the United States.