Another Australian Government IT Catastrophe Looms

Introduction

People who have followed my work over the past year know that I am deeply involved in a grass-roots campaign to eliminate and provide redress for a program in the National Disability Insurance Scheme that is generally referred to as “RoboNDIS’.

RoboNDIS refers to the business systems by which the National Disability Insurance Agency (NDIA) makes decisions about who can participate in the scheme and how much support each participant gets. We are referring to the entire business system – the processes, organisational frameworks, people and technology that is used. We are running this campaign because RoboNDIS has caused immense harm over several years, to participants who have been denied the supports they need, to those who battle on behalf of participants for the correct decisions to be made, for the clinicians who are forced over and over again to provide reports that are ignored, and for the NDIA personnel who want to do the right thing, but are prevented from doing so by perverse internal rules, performance measures and prejudiced managers.

There is extensive evidence that RoboNDIS exists and is an intentionally created regime intended to attack the vulnerable in pursuit of “government savings”. Australia is still working through the fallout from a similar scheme called RoboDebt, in which unlawful algorithms were used to create false debts and unjustly enrich the government. It was a clear example of a regime intended to attack the vulnerable in pursuit of “government savings”.  People whose role in RoboDebt has been closely examined by a Royal Commission have, sometimes in parallel, held key roles in RoboNDIS. It will take a Royal Commission to fully understand exactly what has been done within the RoboNDIS arena, and to identify the people involved and the role they played.

There is a current external review looking at how the NDIS has evolved over the past ten years, and there is clear evidence that the review sees the scheme as having lost its way. This is a good start to turning the tables, but it does not have the powers of a Royal Commission to examine the scheme in forensic detail and expose the depth and breadth of wrong-doing that has been our experience.

Its name is "PACE". Be afraid!

About four years ago, the NDIA commissioned a new business system to enable access and planning decisions. One might imagine that this new business system would be delivering better ways of working that eliminate the harm. But, for at least three of those years, the development of the new system was conducted under the same leadership and management structure as the one we call RoboNDIS. 

More and more people are becoming aware of this “New NDIS System” called PACE.

As an IT Professional of more than 45 years standing, including at least 35 years working with the highest levels of technology and business management in many major Australian and international organisations, the little I know about PACE quite literally terrifies me.

My concern starts with discovery that the acronym PACE literally stands for Participant Alternative Cloud Environment. That’s a bit like calling a new electric car “EMF” for “Electro-Motive Force”. It’s not just meaningless, but it betrays a disturbing level of focus on the underlying technology, rather than its purpose. 

What do I know about PACE? Frankly, very little. There was a superficial announcement a year ago – on 18 August 2022. Subsequently, there has been some vague communication from the NDIA about the trial in Tasmania. In the past few weeks, there have been several superficial announcements about preparing for the new system. But even these are terrifying.

Some of my colleagues in the RoboNDIS Campaign know more than I do. I expect that they will add their insight and perspective to the discussion this paper is intended to stimulate.

Pace Announcement

We need to look at the hidden messages in the 18 August 2022 announcement. Following each paragraph (in italics), I offer some perspective in italics.

• PACE is being designed to be more user-friendly and make it easier for NDIA staff and partners to do their job, giving them more time to deliver a quality experience for participants and providers.

User friendly might be a minimum aspiration, but will it be accessible? What are the design goals? Vague and superficial statements like this inspire no confidence when considered against a backdrop of the appalling browser and phone based efforts delivered to date.

• Our design and build of PACE have been strongly influenced through our ongoing consultation with participants, staff, partners and providers.

I have been a member of a thing called “Participant First” for quite some time. Like everything else I have seen from the NDIA, this has been superficial, bordering on lip service to the concept of engagement.

• It has also been influenced by the outcomes of prior consultation with participants.

But none of that consultation has been presented as a basis for developing expectations.

• NDIA staff have now started testing features of the PACE system. This process will continue over the coming months.

So the system was ready for NDIA testing a year ago. Typically, major initiatives provide dashboards against which progress can be observed. Where were they? 

• We are also continuing our engagement with staff, partners, participants and providers on the new system and its processes and how we want it work.

Co-Design is the appropriate model for developing new, highly complex business systems with integrate automated and manual processes which involve many different human communities. “Engagement” doesn’t signal that there is any real co-design. Indeed, the experience of people involved in Participant First is far, far from co-design. It hardly even rates as “Show and Tell”.

• In addition, we have started planning for a real time test of the PACE system. We expect to start real time testing in Tasmania by the end of 2022.

I’m sure that everybody is interested in the progressive testing program and outcomes. But we don’t see any of that.

• We will provide further information about the testing process in the coming months.

We are still waiting. We’ve seen We’re improving the way we deliver the NDIS | NDIS, which talks vaguely about what the NDIA “plans” to do. We don’t see results!

• The real time test will involve our NDIA and Partners in the Community staff using the new system with current and prospective participants, providers and the wider disability community.

Again, we await results. What is published to date is meaningless high level marketing gibberish that provides only a vague insight to the way people responded to the system during testing. That said, even these superficial results reflect a disturbing lack of acceptance that the system is even barely adequate.

• Participants and providers will be supported by NDIA staff and our Partners in the Community through every step as we test the PACE system.

Well, we would hope so, but if it’s like the standard of help we have seen to date, it won’t be worth anything.

• We will be able to update and improve the PACE system based on the experience and feedback received during User Acceptance Testing, ongoing consultation and engagement and the testing in Tasmania.

Exactly what is this supposed to mean? Is there a robust change management framework that serves to ensure that changes to the system are properly considered, co-designed and tested? How could anybody be confident that the system will be “updated and improved”? We have to remember that this system has been in gestation for four years and in its live use test for more than a year. How much change did it need, and why?

• This will make sure PACE works as designed, our training and support is appropriate, and the new system delivers an improved participant experience.

Again, the question goes to the design. Who is responsible for the system design? Was it co-designed, with the right level of continuing engagement from ALL stakeholders, including a diverse selection of participants and nominees? The low level of satisfaction and the nature of the gaps are strong indicators that if there was co-design, it was, at best, superficial. It probably did not have any significant engagement of real people with real workplace roles that could positively influence the design, let alone the roll-out plans.

• We will make sure our national roll out of the new system is fit for purpose and incorporates feedback from everyone who engages with us, our partners and the Scheme.

By what mechanism is this to be achieved. At this very late stage, it should already be possible to understand how feedback is taken up and actioned.

• We will continue to engage our stakeholders as we develop our plans for Tasmania and be deeply involved in guiding and advising the NDIA in our national roll out approach.

How can anybody have any confidence that stakeholders are engaged appropriately. We don’t even know who is representing us, let alone knowing the channels by which we can provide feedback.

Participant Readiness for PACE National rollout

A videoconference presentation was delivered to a broad audience of NDIS participants, providers and others associated with the scheme on 27 July. To say that the presentation was vague and superficial would be to give it excessive credit. It was a very credible effort at using minimal information to create an illusion of significant progress. We know some significant things will change, but we still have no idea how they will be different in practice. One has to wonder how so little could have taken four years to deliver!

Notwithstanding, there were some interesting snippets. For example, on page 12 of the presentation, we see the following:

We have 3 focus areas to get ready for national expansion:

• Make our new computer system easier for NDIA staff and NDIS partners to use.
• Make our new provider portal and ways of working simpler for NDIS providers.
• Help everyone get ready for the change to our new computer system.

What about the poor participants and their overworked unpaid supports? They didn’t rate a mention. Just who is the target for this new, supposedly much improved NDIS Business System that introduces major changes to process that are briefly glossed over with no detail? Surely it should be focused on the participants first and everybody else second.

And by the way, the use of language in the overview of the new planning pathway uses words like “discussion”, “opportunity”, “Explain”. All these are substantially absent from current experience of many participants. What do they really mean in the context of an NDIS which will doubtless continue to harbour attitudes that harm vulnerable people. A new technology platform will do nothing to change the way people behave. But then again, we still see that planners – people who have no relevant training and qualifications” are going to make decisions about reasonable and necessary supports. Yes, they are going to explain them, but what happens when their explanation is based on a profound lack of understanding of the participant needs? How will that problem be resolved?

PACE Working Groups

The NDIA convenes a program called “Participant First”, which is a weak facsimile of a co-design arrangement, in which a variety of proposed changes to NDIS operational activities are discussed. I have participate in this group and, while contributing to the best of my ability, have now given up on it, because the junior personnel running the program are clearly unable to engage at any level within the NDIA that has influence and control over the direction the agency is taking, and because they lack the expertise to take on board the 45 years of experience I have tried to share with them. 

One activity within the Participant First program has been a series called “PACE Participant Working Groups”. Through a colleague involved in the RoboNDIS Campaign, I have gained access to material provided to participants in this “PACE Participant Working Group”.

I now refer to the slide pack provided for the first meeting of this working group. On page 7, titled “E. Purpose of this working group”, we see 5 points, including:

• Your role is to help us communicate NDIS policy, not to change it
• Give your feedback:
• what’s good
• what needs improving
• what should be deleted
• what is missing

Do those two points seem contradictory? Do they say, “we will ignore your most important feedback, but you can give us the superficial stuff”? That’s what it looks like to me.  Is this an attempt to make up some facsimile of co-design after the fact? It doesn’t fill me with anything but contempt for an agency that barely manages to give lip service to the proven effective methods for building high quality business systems in the 21st century. 

In reality, what I observe in respect of PACE is less adequate than what I experienced when I was tasked to rescue a massively important aspect of the UK National EftPos system, which was being established by the Bank of England. The technology team had eschewed even the most primitive aspects of participant engagement and built an incredibly complex system for inter-bank settlement that might have been technically elegant, but which had zero hope of actually functioning in the real world.

I could go on. It would be pointless. As a world-leading expert in governance of information technology, I already have enough insight to underpin advice to the NDIA Board, that it should stop PACE in its tracks, and conduct an explicit assessment of the project against the International Standard for Governance of IT. The result will doubtless horrify everybody.

And that’s before we even consider the preliminary recommendations of the NDIS Review.

The NDIS (PACE) test in Tasmania

I have spent significant time reviewing the report titled The NDIS test in Tasmania: insights from our evaluation. This report is fascinating for what it says, and fails to say.

Overall, the report has the feel of a marketing statement, putting substantial positive spin on a project that outsiders already expect to be problematic.

On page 4, we see this quite astonishing paragraph: “Overall, NDIA staff and NDIS partners using the new computer system have a positive view of the future operating system. However, their consistent feedback was that several critical features, processes, system guidance, change, communication and training activities or products, were needed prior to any further roll out of our new computer system”.

Read that again: several critical features, processes, system guidance, change, communication and training activities or products, were needed prior to any further roll out. This system was supposed to be ready! Clearly, it was not. The missing elements are actually very familiar to experts in governance of IT. They are frequently missing from projects in which there is an excessive focus on the technology aspect and a significant failure to address the context in which the technology will operate. These conditions caused the Australian Customs to shut down the National Supply Chain for 3 weeks in 2005, and led to massive problems with the Queensland Health Payroll system several years later.

This fault alone could be catastrophic. Not only does it signal a severe lack of readiness, it points to a massive lack of competence in respect of change management, and reinforces suspicions that there has been no serious Co-Design. One thing that co-design ensures, along with effective governance, is that the non-technical aspects of systems are at least given due attention during the design phases. In reality, the non-technical aspects should be thoroughly addressed at every stage.

The test may well have shown that urgent changes (it is inappropriate to call them “enhancements”) can be delivered. But that statement likely gives no consideration whatsoever to the challenges of managing a mass population through what could easily become an extremely fluid and inconsistent system. Particularly for participants and unpaid supports, the experience of working with a system that is not entirely adequate (why else are there so many “enhancements”) and is frequently changing, will become an unbearable nightmare. Remember, a substantial portion of the people who will be trying to use this are people with disabilities. An equally large portion of users will be people with little to no experience of dealing with complex interactive systems. Both groups may be able to learn how to use the system over time, but how will they cope with it changing frequently?

Statistics presented in the report are interesting. Of 16,375 participants in Tasmania, 1,614 had new plans approved during the test and 171 participants joined the scheme. But only 400 participants, family and “carers” responded to the surveys. It appears that only 155 responses came from actual participants.  Of 500 registered providers, only 150 responded ot the surveys. This is not an encouraging level of engagement, and leaves wide open the possibility that many issues will emerge in the full roll-out.

We see that more than 140 people participated in focus groups from December 2022 to March 2023. We also see that over 340 people participated in other Tasmanian and national focus groups and engagement sessions. Seriously, in a population of over 16,000 participants, that’s not a compelling level of engagement – especially when the quite low rate of response to surveys is taken into account.

Despite the relatively low participant feedback rate, the feedback reported is compelling:

• Only 72% of participants said their experience in the Tasmania test was good or very good.
• Only 55% of participants said their experience in the test was better, or a lot better than the last time they talked to the NDIA.
• Only 59% of participants said the information available was helpful or very helpful.
• Only 44% of participants said they knew we were testing a new computer system and improved processes in Tasmania

When I say compelling, I mean in the context of confirming a major failure. The converse of the above statistics are awful. 28% did not have a good experience. 45% thought the new system no better than the old. 41% found material provided not helpful. 56% didn’t realise they were working with a new system. Further, we see numerous statements alluding to significant weaknesses in the system AND the rollout, such as:

• “The NDIS app is a great way to manage my daughter’s plan on the go. It just would help to know when new features are added, perhaps through a pop-up message.”
• “More information on the changes for participants need to be provided in review meetings (plan meetings) or earlier.”
• “Partners delivering Community Connections plans need more training on the new computer system and improved processes.”
• “The purpose or intent of a community connections plan essentially …needs clarity.”
• “More easy read resources are needed.”
• “We are confused about participant-endorsed providers and what they need to do… Needs to be simplified as it's too complicated.”
• Participants would also like more information about how to use the new my NDIS participant portal and app. Participants said a portal testing environment as well as training support would be helpful.

Quite simply, none of these statements should have been prompted by the Tasmania trial. The gaps should have been identified and resolved at a far earlier stage.

I won’t go into as much detail in respect of the feedback from providers, as it’s generally quite similar to that from participants. The satisfaction levels are way too low. Two comments stand out:

• “an opportunity for change management to be more effective could focus on addressing the impacts on the provider organisations, relationships between the provider and the participants”.
• “There was a lack of understanding of what plan managers really need. Much of the change management approach did not cover specifics to plan managers including the claiming process.”

Looking at the responses from NDIA Staff and Partners, we see no metric giving a 50% or above positive score EXCEPT the ones that relate to the need for change. Clearly, these people understand how significantly awful the old system is. The lack of high scores in any other aspect of measurement speaks volumes to how poorly the new system seems to achieve its purpose of delivering a better way of working. Buty having an unsatisfactory system is no justification for replacing it with an equally unsatisfactory system. Nobody side-grades their worn out car for another that is also worn out, unless they have motives other than having a better car!

According to the Introduction to the report, the new system is supposed to deliver an immense upgrade to the operational NDIS environment. Clearly, it has not done that for a substantial body of participants in the test.

The final sentence in chapter 8 of the report is significant: “NDIS partners thought we could improve how we explain the work we expect them (to) do, supported by training resources”. It has been clear for several years now that the NDIA has done very poorly in explaining the work that people are supposed to do, resulting in self-admitted deficiencies and inconsistencies in practice. To see a statement that NDIS Partners still need an uplift in the effectiveness of resources that should prepare people for working with the NDIS is particularly damning!

And then we arrive at Chapter ten – “What we are going to do next”. Again, speaking as a global thought leader in governance of IT, with 45 years of experience in helping organisations achieve higher performance through effective use of information technology, I can only say that the entire content of this chapter fills me with rage at the steaming incompetence of the NDIA in respect of exactly what this project was supposed to do – to deliver massive improvement in the way that NDIA performs its core function of administering the NDIS Act by providing reasonable and necessary supports to Australians who live with disability.

Take these three opening points:

• We’ll work closely with the disability community to refine and prioritise the practice and process we should start with.

Should have been happening from the beginning of the project – at least 2 years ago. It’s way too late to start this now.

• We will expand our Disability Representative and Carer Organisation advisory group to include provider peak bodies to help us prepare to introduce our new computer system nationally

Have I already mentioned that it cannot be only about a computer system? I think so. The NDIA is rolling out a BUSINESS SYSTEM, which has complex elements other than computer systems. They include people, practices, policies, controls, structures and myriad other elements that enable a business to function smoothly. It is already clear that there has been far too little attention to these elements, and history demonstrates over and over again that such failure almost always leads to catastrophic problems in operation.

• Prioritise further improvements to the new computer system based on provider and participant feedback

Of course, this should be a standard practice. But it cannot be limited to improving the computer system. It must address the entire Business System.

The very fact that the document continually addresses computer system context rather than business system context makes it impossible to believe that the system is fit for purpose at this point in time, and raises the very vexing question of whether it can ever be made fit for purpose.

In Conclusion

In July 2018, IEEE Spectrum reported that: “An Australian Senate committee published a 146-page report assessing the government’s progress toward its goal of becoming ‘one of the top three digital governments in the world…that other nations can look to for guidance and inspiration,’ by 2025. Given what is in the report, other nations may want to look elsewhere for their inspiration”.

The situation has not improved. Australia enjoys an ignoble reputation as a World Leader in Government IT Failures.

Who can forget that remarkable time in 2005 when Australian Customs shut down the National Supply Chain.

Who can ignore the fortune still being wasted on the “Personally Controlled Electronic Health Record”, a chameleon that has had many names and forms and continues to defy success.

What about the Australian Apprenticeship Management System?

At the end of July, just a month ago, we saw that “The government has written off a $191 million investment sunk into building a new entitlements calculation engine (ECE) for Centrelink, with “nothing to show for it" in terms of a working system”. If that’s not clear, that was money wasted on a failed system intended to do, pretty much exactly what PACE has to do, except that the context of PACE is way more complex! And, being a Services (yes, that is an oxymoron too), Australia project, it has shared significant elements of top level leadership and management with the NDIA over time. Apparently, sources had said that the project had effectively been cancelled because the ECE did not work as intended. It probably lacked co-design too, as co-design should ensure that systems do actually work as intended, even if it takes longer and requires more people. 

The ECE project had also been caught up in a scandal on how the contracts were awarded, which had invited further scrutiny. Again, these contracts were awarded under the same ministerial oversight as applied to PACE. Why would we ever think that PACE could have been immune from corrupt purchasing as well as perverse functional design.

The signs are unmistakable. Even without the extensive change that will be recommended by the NDIS Review, PACE can only be seen as another failure for the Australian Government. It should be shut down, immediately.

The NDIS Review appears to be pitching an entirely new operating model for the NDIS. A new operating model that can only be achieved by wholesale change, not just in enabling technology, but in the mindset, practices and rules of the National Disability Insurance Agency. 

PACE should not rise like a phoenix from its own ashes. Rather, the Australian government should establish an entirely new leadership and governance framework for the ongoing development of new NDIS Business Systems.

The Governance arrangements must be designed using the global standard developed in Australia. It has been eschewed by every federal and state government to date, but analysis of every government failure and the few successes (especially in New South Wales) demonstrates its relevance. The governance arrangements must include expertise in this standard, expertise in the disability and professional communities that the new business systems are to serve, expertise in large scale, technology-enabled business change, expertise in compliance, safety and security, and expertise in design and delivery of 21st century government business systems.

In my experience, there is only one person in Australia who has the competence and track record required to lead this initiative. He was the absolute force behind the most successful digital transformation in government in Australia.

His name is @Victor Dominello. I hope that @Bill Shorten and @Anthony Albanese are smart enough to secure his help for at least three years, despite political differences.

What do you think?