This Phoenix will never raise again

There is a saying, hindsight is 20/20: looking back at a problem, it is obvious now what the solution was supposed to be right from the beginning. Knowing what we know now, we would never have made any mistakes.

Here is a story about an infamous “digital transformation” project that went horribly sideways. Unlike its namesake, the Phoenix project is not going to rise from the ashes. However, the government seems intent on repeating this failed process from square one.

Can you identify the recurrent patterns which led to this fiasco?

  • 2008 Canadian Prime Minister Steven Harper decides to reduce the cost of running the government by making it more efficient. 

  • 2009 The PM’s cabinet approves C$310 million funding for a new project, Phoenix. It will replace the homegrown payroll system and consolidate payroll processing for the whole federal government. That's about 101 departments and 300,000 employees. C$310 million sounds like a lot of money. However, the government estimated savings of C$78 million a year.

  • 2011 IBM wins the contract and decides to use PeopleSoft software, owned by Oracle.

  • 2014 The Federal Government decides to take over training and execution in order to reduce the cost of the project.

  • May 2015 IBM recommends delaying the launch due to critical problems.

  • November 2015 Justin Trudeau becomes the new Prime Minister of Canada.

  • February 2016 The Phoenix system is launched for 34 departments including about 120,000 employees. At this time, 2,700 people are laid off, in line with project objectives.

  • April 2016 Reports emerge that thousands of public servants are experiencing payroll problems. These employees can’t pay their loans or mortgages and some just run out of money. 

  • June 2016 The federal government opens another pay center to deal with the problems. Meanwhile, several unions representing federal employees take the Federal Government to court over unpaid wages 

  • July 2016 After an emergency committee meeting, it is estimated that another C$20 million will be needed to fix all the problems, by the end of October 2016

  • May 2017 The government announces that the total cost of fixing this system will increase to C$400 million. (Not C$40 million, which would have been twice the previous estimate -- C$400 million.)

  • November 2017 The estimate to fix the system jumps to C$540 million. The union calls for the project to be scrapped. The government declines and hires an additional 1,000 employees to handle the backlog problems (You’ll recall that much of the cost savings were to be achieved by cutting salaries, not adding them.)
    Public Services Minister Carla Qualtrough says all options will be considered over the long term, including replacing Phoenix. But she said the government has to “forge ahead” with its fixes to ensure that Canada’s public servants are paid: 'Stabilizing the system remains our top priority.'

  • May 2018 The Auditor General concludes that there were three government executives responsible for delivering the project, that the Deputy Minister 'was responsible for ensuring that a governance and oversight mechanism to manage the project was in place, documented, and maintained, and that the project was managed according to its complexity and risk.' At this point, there is a backlog of 600,000 unsolved payroll cases.

  • July 2018 A report by Canada’s Standing Senate Committee concludes that so far, the project has cost C$954 million. By 2023, costs could rise to C$2.2 billion.

  • May 2019 The federal government decides to replace the Phoenix project and three companies - Ceridian, SAP SE, Workday Inc. - will compete to provide a replacement.

  • September 2021 (Present Day). The federal government awards an 8 year, $16.9 million contract to Ceridian. The company will run a pilot project to move 4 departments to a cloud-based HR software platform. (An almost irrelevant note: in July 2019, Ceridian hired Gianluca Cairo. Mr. Cairo was then Chief of Staff to Minister Navdeep Bains of the Federal Government.)

As you might expect, there is lots of blame to go around. It is for others to spend the time on that.

Over the years I’ve worked on many projects. These were complex projects, which had major impacts on companies’ operations. This is what I learned.

Certain recurrent patterns tell you if a project will run into major problems.

  • While preparing the requirements, the customer never talks with the (potential) vendor and creates unrealistic expectations. Many requirements in an RFP are categorized  as 'must-have' without proper context. It forces vendors to 'creatively' describe how their solution will fit the requirements.

  • The requirement document isn’t detailed enough to allow for properly estimating the resources required for the project.

  • The complexity of the project is not fully appreciated by the vendor. As an example, in this case, for these 300,000 federal employees, there are 80,000 rules which govern their employment and payroll.

  • The vendors always underestimate the effort required. Instead, they are focused on winning the project with a 'competitive' bid. Delivering it is a problem for the delivery team, not for the sales team.

  • The customer rarely has an experienced team ready to manage the project.

  • The technology is chosen before all the analysis is done.

  • Bad news is ignored and everyone hopes that it will never come up.

  • Training, deployment and support are always an afterthought.

  • Diligent testing is not seen as a critical part of the project.

I am sure that from your experience you can list many more examples.

Knowing these recurrent patterns, how can a company strategically run a successful project?

  • Build a team which has the experience to execute on the project

  • Spend more time on requirements and planning. With a better roadmap, the implementation is always easier and shorter.

  • Never choose technology before you know and validate the requirements.

  • Always communicate. Bad news will only get worse, the longer you try to ignore the problems.

And finally, more money never fixes a bad project. That's the recurrent pattern.

For those who like to do their own research, I recommend checking out these resources:

Previous
Previous

Facebook keeps amplifying the bad for its own good

Next
Next

Alexa loves you more