When IT projects fail on launch, it’s rarely down to the technology. A project can look healthy in testing and still fall apart when the business moves onto the new system. In recent project rescues, we’ve seen the same cause: not enough testing.
The advice is to test properly with real users. Have them run the same process in the old and new systems and compare the outputs, including documents. For finance or other operational migrations, run parallel runs with the same data in both systems. Check any variance and understand why it’s there.
That testing also builds user adoption, because people who help test feel part of the project. Get the essentials right first and add the extras later. You’ll then know what your variance looks like and what to do about it, which gives you a smoother go live and a better start afterwards.
Quick takeaways
- Go live is where many projects fail. A good result in testing doesn’t prove the business is ready to run on the new system.
- Insufficient testing is the most common cause we see in project rescues.
- Test with real users. Have them run the same process in the old and new systems and compare the outputs.
- Check your documents. Confirm the new system produces the same documents as the old one. Any changes should be deliberate improvements.
- Run parallel runs for finance and operational migrations. Enter the same data in both systems and compare the results.
- Investigate every variance. Find out why it exists and decide whether you’re happy with it.
- Essentials first, extras later. Add the extras after go live, once the basics are working.
Everything looked fine in testing, so what went wrong?
Often, not enough testing. It’s that simple.
Most teams say, “Yes, we test.” But do you really test? Do you sit down with the users while they run the same process in the old system and then in the new one? And do you get the same outputs?
Test properly: same process, same outputs
A proper test compares like with like.
- Users run the same process in the old system and the new system.
- You compare the results.
- You check that the new system produces the same documents as the old one.
There may be differences, and some are intended. You might be changing systems because you need better or more consistent documentation. The point is that you know what has changed and why. You don’t find out on go live day.
Parallel runs: a safety net before go live
A parallel run means entering the same data in the old system and the new system, then comparing the results. It’s especially useful for a finance migration or another operational system migration. Our testers and data migration specialists help clients plan these runs.
Ask yourself:
- Did you enter the same data in both systems?
- Did you get the output you expected from each?
- If the outputs differed, did you check the variance (the difference between the two results)?
- Do you know why it’s there, and are you happy with it?
How testing helps user adoption
This isn’t only about catching errors. When users take part in testing and parallel runs, they’re more engaged. They buy into the project and become part of making sure adoption doesn’t fail.
After go live, those same people keep using the system as it was designed. They also feed back on any tweaks the design needs.
Why IT projects fail at go live when the extras come first
Think about any big event, whether that’s Glastonbury, a system change or a wedding. The essentials have to be in place first. For a wedding you need a venue, guests, food and tables.
But when you plan a big event, some things never make the final cut, and nobody notices. Fret about the bells and whistles early on, and you can miss the essentials. Then you’ve got a failed event. In a systems project, that’s a failed go live.
How to avoid a failed go live: the order we work in
- Get the essentials right first.
- Test with real users, using the same process in the old and new systems.
- Compare outputs, including documents.
- Run parallel runs for finance and operational migrations.
- Check every variance and decide how to act on it.
- Add the bells and whistles afterwards.
- Keep users involved after go live so their feedback shapes any design tweaks.
Get the basics right every time and you can’t go too wrong. You know what your variance looks like and how to act on it. That gives you a more successful go live and a stronger start to user adoption.
If you’re partway through a project and worried about go live, our project rescue support is a sensible place to start.
Is your project ready to go live?
- Essentials agreed. You know the core processes the business needs on day one, and the extras are parked until after go live.
- Real users tested it. Users ran the same process in the old system and the new system.
- Outputs match. You compared the results from both systems.
- Documents match. The new system produces the same documents as the old one, apart from changes you chose to make.
- Parallel runs done. For a finance or operational migration, you entered the same data in both systems.
- Variances checked. You know why each difference exists and you’re happy with it.
- Users are on board. The people who tested it feel part of the project and know how the system is designed to be used.
- Feedback route planned. Users know how to report tweaks once you’ve gone live.
- Extras scheduled. The bells and whistles have a place on the plan for after go live.
If you can’t tick every box, that’s a sign to pause and look at the gaps. It’s much cheaper to find them now than on go live day.
Frequently asked questions
In the project rescues we’ve picked up recently, the common cause was not enough testing. Everything looked fine in testing, then fell apart when the business moved onto the new system. Testing with real users, comparing outputs and running parallel runs helps you find problems before go live.
A parallel run means entering the same data in your old system and your new system, then comparing the outputs. It’s useful for finance migrations and other operational system migrations. If the results differ, you check the variance and work out why before you rely on the new system.
Sit down with the users and have them run the same process in the old system and the new system. Compare the outputs, including any documents produced. Expect some intended differences, such as more consistent documentation, but make sure you understand each one rather than discovering it after go live.
Check the variance. Find out why the outputs differ and decide whether you’re happy with the difference. Some differences are deliberate changes. Others point to a problem in the data, setup or process. Either way, you want to understand them before go live, not after.
Users who take part in testing and parallel runs are more engaged and feel more ownership of the project. They are part of making sure adoption doesn’t fail. After go live, they keep using the system as designed and can feed back on design tweaks.
The essentials, meaning the core processes your business needs to run on day one. Which ones those are depends on your business. Think of a wedding: you need the venue, guests, food and tables before anything else. Add the extras once the basics are working and you understand your variance.
We’ve picked up project rescues recently, and not enough testing was a common cause. Whether a particular project can be recovered depends on its situation, so we’d need to look at the details. Reviewing how testing, parallel runs and essentials were handled is a sensible starting point.
About Mark Pullar
Mark Pullar is the founder of All My Systems, a Microsoft Dynamics 365 consultancy based in Hebden Bridge, West Yorkshire. He has 25 years’ experience in IT and senior management, holds an MSc in Information Systems Management, and is a Microsoft Certified Professional in Dynamics 365. Mark set up All My Systems in 2014 to bring proper business systems to small and medium sized companies, and he shares practical lessons on productivity and running a business along the way. Read more about Mark’s story and how All My Systems began.