Data is the foundation on which all modern businesses are built and the importance of accurate and timely data has never been more important to inform critical decision making. It is therefore curious that data migration (the process of moving data from one system to another) is often neglected and resigned to the bottom of the priority list during ERP projects. Ask any seasoned ERPer what caused the biggest headache on their last project and you are sure to be told one thing: data migration.
Reasons for project failure
Explore related questions
The foundation for the systems that run your business is the data they store. If you neglect this foundation, you risk your business, and this is never more evident than when you embark on improvements in your business processes and therefore need to upgrade your systems. Managing the move of the data from your old systems to new is a critical part of the business change, and ensuring their accuracy, completeness and correctness should be a given. While the improvements in the user experience in the new system may be eye catching and the changes in business processes may deliver big benefits, if the data in the system is not complete and correct then it is bound to fail. It is just as important to focus resources and effort on getting the data right as it is on the user interface or business change. The changes are driven by the business and so should be managed by the business.
Data migration (DM) projects are notorious for overrunning on both time and budget, although senior management of organisations undertaking major change programmes may not be aware of this reputation. Or at least that is how it seems to DM professionals who are regularly called in to rescue programmes that are delayed because the data isn’t ready. So why is it that DM, which appears simple to those not experienced in doing it, causes so many problems?
“Data migration (DM) projects are notorious for overrunning on both time and budget, although senior management of organisations undertaking major change programmes may not be aware of this reputation”
It often relates to the lack of understanding among the senior management who therefore don’t allow sufficient time, resource or expertise to do it properly. Among the more common issues to be encountered in failed projects are:
• No clear ownership of the data
• Unclear acceptance criteria/inability to reconcile and sign off results
• Lack of senior support for the DM project
• Starting too late waiting for decisions in main programme
• Overly optimistic programme plans after no involvement of migration experts in planning
So let’s run through these and see why they cause the projects to fail.
Too many executive teams think that the IT department owns their data, and even worse too many IT departments agree with them. They would never consider that the maintenance department owned the machinery in the factory or the computers in the offices, but they are happy to delegate ownership of the data that drives their business to the IT department that manages it for them. This lack of ownership results in them not accepting responsibility for the quality or cleanliness of the data in their existing systems, which are fundamental to any successful migration. This is the basis of the successful implementation of a new system and the business change that has justified it. They then try to blame the IT department for the failure, when they are the only people able to sort the issues out.
This has been further complicated by the trend for large organisations to set up shared service centres (SSCs) to carry out the transactional processing for their systems. This may get the IT department off the hook, however just because the SSC physically updates the data this is only done on instructions from the business, so the ultimate responsibility for the accuracy, completeness and correctness of the data still lies with the business that owns them.
When, late in the day, the problems with the data become apparent, this lack of clarity of ownership leads to the different groups playing the blame game rather than collaborating to fix them and so extending the inevitable delay. The ownership should be established at the very beginning of the programme. While all the stakeholders, IT, the SSC, and the systems integrator have a role to play in getting the data in a fit state to be loaded to the new system, the ultimate authority and responsibility must lie with the business.
This leads onto the next issue where clear and measurable criteria for the acceptance of the migrated data have not been established. Most programmes will have a series of phases with a quality gate at the end of each to identify if the deliverables of the phase are complete so that the programme can move on to the next phase, confident that it is doing so on firm foundations. The state of the data must be one of the deliverables that is included in this check and the users of the new system must make plain what is considered acceptable in terms of accuracy, completeness and correctness of the data at each stage. If this isn’t done then it is not until the very late stages of testing, if at all, that any attention is paid to whether the new system, when populated with data from the migration, will actually work.
The acceptability criteria for the data must be clear and measurable, but it is never possible to achieve 100 percent correctness or a complete history of data, so the business must work out what the cost of incomplete or incorrect data is. Then, an informed decision can be made as to how much resource should be committed to fixing data quality problems. Without clear criteria as to what is acceptable it can be difficult to establish exactly what is required to correct the situation, and again the disputes and finger pointing that ensue just delay the corrections even more. However, you would be surprised how often this fundamental step is overlooked in setting up the governance for a programme, usually with the result described above.
Now, if you have established ownership and acceptance criteria you should hopefully be able to plan and resource your DM project with confidence. However this does not guarantee success as, if you do not have support from your executive management, you will never get the support of the business to address the issues you find with the data. It is critical that the business management, who are the owners of the data, are clear that they will be held accountable by their executive management for the success of the migration and therefore the programme as a whole. Executive management’s support should be visible to the whole organisation; without this business as usual, priorities will override the data migration requirements in decision making throughout.
Without the appropriate motivation by executive management, the business will not find the time to help identify the causes of the data quality problems and change their processes to avoid compounding the problems or help the corrective action required to fix the existing incorrect data. The best way of fixing problems with incorrect or incomplete data is in the existing system, as it is a familiar environment for the staff and they will remain fixed going forward rather than having to be corrected each time the data is processed. To do this you need the support of the business users, but they have a business as usual job to do so are not necessarily too keen on taking on the additional workload of correcting historical data. This will only happen if it is clear that they are being measured on achieving this alongside their normal targets, hopefully with some of their normal targets relaxed a little to allow for this additional workload, or additional resource being provided.
“You need the support of the business users,but they have a business as usual job to do so are not necessarily too keen on taking on the additional workload of correcting historical data”
Another common problem for the DM project is that it is not kicked off early enough in the programme. It is not possible to complete the necessary analysis, design and processing to deliver good quality data if you wait until the new system has been selected and the design completed. Especially as the new system design and configuration is likely to change right up to the final testing phase; even if you wait until it seems reasonably stable, you may have to cope with changes that cause rework at the last minute. It is more important to have the capability to react to these changes than to wait until you believe the design is fixed.
In many cases where the DM element of the programme has been delayed until the design was ready, the result is that fatal quality problems with the data in the legacy systems aren’t discovered until it is too late to correct them in time for the planned implementation date. This leaves the programme with the choice of delaying the implementation or continuing and attempting to correct the data in the new system. If the latter has to be attempted it means that the users are put in a position where they not only have to cope with an unfamiliar system and processes but also try and correct data issues that may be months, if not years, old. This inevitably tends to colour the early experience of the new system by the users and they may never overcome the negative impression they receive.
“This leaves the programme with the choice of delaying the implementation or continuing and attempting to correct the data in the new system”
Indeed with the advance of cloud based systems, and the reduction in the time necessary to implement them, it is even more important to kick off the DM project even before a vendor has been selected. Or possibly you could consider an ongoing data quality programme so that when it is decided to make such a change there is less likelihood of issues with the data.
It may seem that you can’t start to design your migration until you have a clear view of the target system design, but in most cases the logical view of the data in the new system will not be altered greatly by the actual software selected. The mapping from the logical view to the physical requirements of the new system can be a step undertaken later in the schedule once the design has begun to settle down.
It would seem that in a desire to satisfy the wishes of the customer for an aggressive delivery date, the programme plans developed during the selection process for a new system can often be described as optimistic at best. This may be due to a lack of expertise on the part of the planners; in most cases no DM specialist is engaged until after the overall programme timescales have been developed, and so the timeline and resources allocated to DM are commonly inadequate. Being faced with a plan that has two or three test loads before the live cutover and an elapsed time of a few months is a common experience and in more cases than not the first task of the newly appointed DM project manager is to argue either for a replanning exercise and a delay to the implementation date or a significant increase in the resources allocated to the task.
The initial planning phase is usually carried out before a number of key decisions about the migration have been made, such as:
• What tools are to be used
• How much historical data should be migrated
• What the acceptance criteria is for a successful migration
• What the archive strategy will be for data not migrated
• What the cutover strategy will be
In this case it is not surprising that the initial plans are inaccurate and need to be reworked once these decisions have been made. Programme planners and, dare I say it, salespeople are always more likely to err on the side of optimism in their initial plans and it is up to the business to take a view as to how optimistic they are being and so decide on a realistic target date. The strategy and tactics for the migration need to be put in place at this stage so that the planning is based on firm foundations. Thinking that your existing IT department will be able to carry out a successful migration using spreadsheets, with little or no business support, is a recipe for disaster. Make sure you have engaged the right people, can provide them with the right tools and resources, and that you can plan confidently.
The key learnings in all this are, I believe:
• Ensure the ownership of the data lies clearly with senior business management
• Establish clear and measurable acceptance criteria for the migration
• Get the C-level management to visibly support the migration
• Start your DM work as soon as you decide you are going to implement a new system
• Engage DM professionals early to inform the planning process and give the migration a fast start
• Employ a professional toolset; don’t just use spreadsheets etc.
• And finally expect it take at least 50 percent longer than you think it should
If you pay attention to these learnings and find a good team, you are well on the way to a successful implementation.
Case Study
Is poor master data management holding you back?
Irrespective of your system of record, if the data contained within it is poorly managed, inaccessible or cumbersome, it is unlikely you will reap the rewards promised by your investment in ERP. Many customers develop manual work-arounds and tasks like simple approvals can lead to errors and duplication of effort.
Embedding master data management (MDM) tools can deliver significant benefits including improved accuracy, more efficient workflows and generally improving the user experience for employees.
One company that has taken MDM to the next level is Upfield – the largest plant-based product company in the world that operates in 95 countries and runs SAP S/4HANA as the backbone of its global operations.
The Challenges
Complexity and variety of processes and business scenarios Upfield’s standard process of master data governance was clear but lacked the supporting tools required to make the process efficient. Everything from request to approval, communications and processing was manual and relied on extensive email flows. Lack of automation exposed Upfield to human error and required many master data adjustments, decreasing overall data quality.
Upfield also wanted to cover all accompanying objects required for purchasing, costing, manufacturing, distribution and sales at least to a level where they could track if essential tasks were completed or not.
Material master. Upfield needed to ensure that all production items (raw, packaging, semi-finished and finished goods) were under control and delivered on time and in full.
Vendors. Upfield needed to streamline the overall onboarding process of vendors, creating a workflow eliminating the need to use multiple systems.
Customers. Upfield needed to improve new customer creation and master data management. The process of approval requests required changes to ensure that the business could actively participate in the process.
Lack of process harmoniation between regions. Upfield runs businesses on six continents, using separate SAP systems, running a slightly different data model and requiring a specific set of rules. Each region has a slightly different organisational structure which impacts ownership of both process steps and data.
Definition of roles and responsibilities. Upfield discovered many gaps in roles and responsibilities in the MDM process. Finding an owner for each data object and each master data view or field became a challenge that needed to be resolved for the process to work.
SOLUTION OVERVIEW
Upfield looked at numerous MDM and governance solutions. They eventually selected Maextro after a successful pilot project. The initial Maextro objects delivered to Upfield were material, customer and vendor (business partner). The project was managed and delivered by the Upfield master data team with support from Bluestonex.
Material master. Encompassing the creation, extend and change functionality of four material types across all Upfield plants and sales orgs. There was extensive use of the Maextro rule functionality and high governance of process.
Business partner. Maextro’s Business Partner object was deployed for the maintenance of both vendor and customer business partners in Upfield’s S/4HANA environment. The workflow capabilities of Maextro were used extensively to direct tasks to different users based on geography, departmental functions and vendor types. Additionally, multiple linked customers could be processed together, within a single Maextro request.
From the end user perspective, the look and feel of the Maextro Business Partner screens is consistent with screens for other Maextro objects (such as Material Master), thus insulating the user from the complexity of the SAP standard BP transaction.
User experience. The user experience was further simplified using SAP Screen Personas Slipstream UI5 engine apps covering create request, approve request, task list, data log/audit. Maextro’s core UI apps meant zero development was required and worked ‘out of the box’, using data and screens defined in the back-end ABAP system.
The results. Transparency and ease of use were flagged as obvious benefits. Additionally, the fact that users now need provide only the data they actually own rather than having to query multiple teams to collect all of the data required. The significant decrease in offline communication was also noted as a big advantage.
The business rules adopted by Maextro not only improve the quality and consistency of the data, but also to reduce manual effort. In many tasks, users simply review pre-populated data and only occasionally adjust where required.
Business partner master data. The implementation of Maextro was a huge success and more than 300 potential users of the system participated in the training provided. The number of e-mail communications between users of the system decreased significantly. Data quality improved due to the way Maextro enforces Upfield’s business rules. And the time needed to create a customer account has also been shortened and the requestor now has full visibility of progress.
Skills & knowledge transfer. Upfield benefitted from the quality of knowledge transfer and support received from the Bluestonex team, which greatly helped in understanding how Maextro works and how it can be shaped to requirements. The tool is very flexible and opens up vast opportunities for process simplification and improvements in the future.
“Bluestonex has been incredibly supportive and agile in getting us over the line for three major data objects – product, customer and vendor. Their passion, knowledge and experience has been immensely appreciated by our Associates, who have learnt a lot and gained new skills and capabilities. We are now building on this good foundation to deliver further process and UX improvements for our business.”
Jaroslaw Chrupek, Global director master data & head of finance shared services
