Another Australian Government IT Catastrophe Looms
Introduction
People who have followed my work over the past year know that I am deeply involved in a grass-roots campaign to eliminate and provide redress for a program in the National Disability Insurance Scheme that is generally referred to as “RoboNDIS’.
RoboNDIS refers to the business systems by which the National Disability Insurance Agency (NDIA) makes decisions about who can participate in the scheme and how much support each participant gets. We are referring to the entire business system – the processes, organisational frameworks, people and technology that is used. We are running this campaign because RoboNDIS has caused immense harm over several years, to participants who have been denied the supports they need, to those who battle on behalf of participants for the correct decisions to be made, for the clinicians who are forced over and over again to provide reports that are ignored, and for the NDIA personnel who want to do the right thing, but are prevented from doing so by perverse internal rules, performance measures and prejudiced managers.
There is extensive evidence that RoboNDIS exists and is an intentionally created regime intended to attack the vulnerable in pursuit of “government savings”. Australia is still working through the fallout from a similar scheme called RoboDebt, in which unlawful algorithms were used to create false debts and unjustly enrich the government. It was a clear example of a regime intended to attack the vulnerable in pursuit of “government savings”. People whose role in RoboDebt has been closely examined by a Royal Commission have, sometimes in parallel, held key roles in RoboNDIS. It will take a Royal Commission to fully understand exactly what has been done within the RoboNDIS arena, and to identify the people involved and the role they played.
There is a current external review looking at how the NDIS has evolved over the past ten years, and there is clear evidence that the review sees the scheme as having lost its way. This is a good start to turning the tables, but it does not have the powers of a Royal Commission to examine the scheme in forensic detail and expose the depth and breadth of wrong-doing that has been our experience.
Its name is "PACE". Be afraid!
About four years ago, the NDIA commissioned a new business system to enable access and planning decisions. One might imagine that this new business system would be delivering better ways of working that eliminate the harm. But, for at least three of those years, the development of the new system was conducted under the same leadership and management structure as the one we call RoboNDIS.
More and more people are becoming aware of this “New NDIS System” called PACE.
As an IT Professional of more than 45 years standing, including at least 35 years working with the highest levels of technology and business management in many major Australian and international organisations, the little I know about PACE quite literally terrifies me.
My concern starts with discovery that the acronym PACE literally stands for Participant Alternative Cloud Environment. That’s a bit like calling a new electric car “EMF” for “Electro-Motive Force”. It’s not just meaningless, but it betrays a disturbing level of focus on the underlying technology, rather than its purpose.
What do I know about PACE? Frankly, very little. There was a superficial announcement a year ago – on 18 August 2022. Subsequently, there has been some vague communication from the NDIA about the trial in Tasmania. In the past few weeks, there have been several superficial announcements about preparing for the new system. But even these are terrifying.
Some of my colleagues in the RoboNDIS Campaign know more than I do. I expect that they will add their insight and perspective to the discussion this paper is intended to stimulate.
Pace Announcement
We need to look at the hidden messages in the 18 August 2022 announcement. Following each paragraph (in italics), I offer some perspective in italics.
User friendly might be a minimum aspiration, but will it be accessible? What are the design goals? Vague and superficial statements like this inspire no confidence when considered against a backdrop of the appalling browser and phone based efforts delivered to date.
I have been a member of a thing called “Participant First” for quite some time. Like everything else I have seen from the NDIA, this has been superficial, bordering on lip service to the concept of engagement.
But none of that consultation has been presented as a basis for developing expectations.
So the system was ready for NDIA testing a year ago. Typically, major initiatives provide dashboards against which progress can be observed. Where were they?
Co-Design is the appropriate model for developing new, highly complex business systems with integrate automated and manual processes which involve many different human communities. “Engagement” doesn’t signal that there is any real co-design. Indeed, the experience of people involved in Participant First is far, far from co-design. It hardly even rates as “Show and Tell”.
I’m sure that everybody is interested in the progressive testing program and outcomes. But we don’t see any of that.
We are still waiting. We’ve seen We’re improving the way we deliver the NDIS | NDIS, which talks vaguely about what the NDIA “plans” to do. We don’t see results!
Again, we await results. What is published to date is meaningless high level marketing gibberish that provides only a vague insight to the way people responded to the system during testing. That said, even these superficial results reflect a disturbing lack of acceptance that the system is even barely adequate.
Well, we would hope so, but if it’s like the standard of help we have seen to date, it won’t be worth anything.
Exactly what is this supposed to mean? Is there a robust change management framework that serves to ensure that changes to the system are properly considered, co-designed and tested? How could anybody be confident that the system will be “updated and improved”? We have to remember that this system has been in gestation for four years and in its live use test for more than a year. How much change did it need, and why?
Again, the question goes to the design. Who is responsible for the system design? Was it co-designed, with the right level of continuing engagement from ALL stakeholders, including a diverse selection of participants and nominees? The low level of satisfaction and the nature of the gaps are strong indicators that if there was co-design, it was, at best, superficial. It probably did not have any significant engagement of real people with real workplace roles that could positively influence the design, let alone the roll-out plans.
By what mechanism is this to be achieved. At this very late stage, it should already be possible to understand how feedback is taken up and actioned.
How can anybody have any confidence that stakeholders are engaged appropriately. We don’t even know who is representing us, let alone knowing the channels by which we can provide feedback.
Participant Readiness for PACE National rollout
A videoconference presentation was delivered to a broad audience of NDIS participants, providers and others associated with the scheme on 27 July. To say that the presentation was vague and superficial would be to give it excessive credit. It was a very credible effort at using minimal information to create an illusion of significant progress. We know some significant things will change, but we still have no idea how they will be different in practice. One has to wonder how so little could have taken four years to deliver!
Notwithstanding, there were some interesting snippets. For example, on page 12 of the presentation, we see the following:
We have 3 focus areas to get ready for national expansion:
What about the poor participants and their overworked unpaid supports? They didn’t rate a mention. Just who is the target for this new, supposedly much improved NDIS Business System that introduces major changes to process that are briefly glossed over with no detail? Surely it should be focused on the participants first and everybody else second.
And by the way, the use of language in the overview of the new planning pathway uses words like “discussion”, “opportunity”, “Explain”. All these are substantially absent from current experience of many participants. What do they really mean in the context of an NDIS which will doubtless continue to harbour attitudes that harm vulnerable people. A new technology platform will do nothing to change the way people behave. But then again, we still see that planners – people who have no relevant training and qualifications” are going to make decisions about reasonable and necessary supports. Yes, they are going to explain them, but what happens when their explanation is based on a profound lack of understanding of the participant needs? How will that problem be resolved?
PACE Working Groups
The NDIA convenes a program called “Participant First”, which is a weak facsimile of a co-design arrangement, in which a variety of proposed changes to NDIS operational activities are discussed. I have participate in this group and, while contributing to the best of my ability, have now given up on it, because the junior personnel running the program are clearly unable to engage at any level within the NDIA that has influence and control over the direction the agency is taking, and because they lack the expertise to take on board the 45 years of experience I have tried to share with them.
One activity within the Participant First program has been a series called “PACE Participant Working Groups”. Through a colleague involved in the RoboNDIS Campaign, I have gained access to material provided to participants in this “PACE Participant Working Group”.
Recommended by LinkedIn
I now refer to the slide pack provided for the first meeting of this working group. On page 7, titled “E. Purpose of this working group”, we see 5 points, including:
Do those two points seem contradictory? Do they say, “we will ignore your most important feedback, but you can give us the superficial stuff”? That’s what it looks like to me. Is this an attempt to make up some facsimile of co-design after the fact? It doesn’t fill me with anything but contempt for an agency that barely manages to give lip service to the proven effective methods for building high quality business systems in the 21st century.
In reality, what I observe in respect of PACE is less adequate than what I experienced when I was tasked to rescue a massively important aspect of the UK National EftPos system, which was being established by the Bank of England. The technology team had eschewed even the most primitive aspects of participant engagement and built an incredibly complex system for inter-bank settlement that might have been technically elegant, but which had zero hope of actually functioning in the real world.
I could go on. It would be pointless. As a world-leading expert in governance of information technology, I already have enough insight to underpin advice to the NDIA Board, that it should stop PACE in its tracks, and conduct an explicit assessment of the project against the International Standard for Governance of IT. The result will doubtless horrify everybody.
And that’s before we even consider the preliminary recommendations of the NDIS Review.
The NDIS (PACE) test in Tasmania
I have spent significant time reviewing the report titled The NDIS test in Tasmania: insights from our evaluation. This report is fascinating for what it says, and fails to say.
Overall, the report has the feel of a marketing statement, putting substantial positive spin on a project that outsiders already expect to be problematic.
On page 4, we see this quite astonishing paragraph: “Overall, NDIA staff and NDIS partners using the new computer system have a positive view of the future operating system. However, their consistent feedback was that several critical features, processes, system guidance, change, communication and training activities or products, were needed prior to any further roll out of our new computer system”.
Read that again: several critical features, processes, system guidance, change, communication and training activities or products, were needed prior to any further roll out. This system was supposed to be ready! Clearly, it was not. The missing elements are actually very familiar to experts in governance of IT. They are frequently missing from projects in which there is an excessive focus on the technology aspect and a significant failure to address the context in which the technology will operate. These conditions caused the Australian Customs to shut down the National Supply Chain for 3 weeks in 2005, and led to massive problems with the Queensland Health Payroll system several years later.
This fault alone could be catastrophic. Not only does it signal a severe lack of readiness, it points to a massive lack of competence in respect of change management, and reinforces suspicions that there has been no serious Co-Design. One thing that co-design ensures, along with effective governance, is that the non-technical aspects of systems are at least given due attention during the design phases. In reality, the non-technical aspects should be thoroughly addressed at every stage.
The test may well have shown that urgent changes (it is inappropriate to call them “enhancements”) can be delivered. But that statement likely gives no consideration whatsoever to the challenges of managing a mass population through what could easily become an extremely fluid and inconsistent system. Particularly for participants and unpaid supports, the experience of working with a system that is not entirely adequate (why else are there so many “enhancements”) and is frequently changing, will become an unbearable nightmare. Remember, a substantial portion of the people who will be trying to use this are people with disabilities. An equally large portion of users will be people with little to no experience of dealing with complex interactive systems. Both groups may be able to learn how to use the system over time, but how will they cope with it changing frequently?
Statistics presented in the report are interesting. Of 16,375 participants in Tasmania, 1,614 had new plans approved during the test and 171 participants joined the scheme. But only 400 participants, family and “carers” responded to the surveys. It appears that only 155 responses came from actual participants. Of 500 registered providers, only 150 responded ot the surveys. This is not an encouraging level of engagement, and leaves wide open the possibility that many issues will emerge in the full roll-out.
We see that more than 140 people participated in focus groups from December 2022 to March 2023. We also see that over 340 people participated in other Tasmanian and national focus groups and engagement sessions. Seriously, in a population of over 16,000 participants, that’s not a compelling level of engagement – especially when the quite low rate of response to surveys is taken into account.
Despite the relatively low participant feedback rate, the feedback reported is compelling:
When I say compelling, I mean in the context of confirming a major failure. The converse of the above statistics are awful. 28% did not have a good experience. 45% thought the new system no better than the old. 41% found material provided not helpful. 56% didn’t realise they were working with a new system. Further, we see numerous statements alluding to significant weaknesses in the system AND the rollout, such as:
Quite simply, none of these statements should have been prompted by the Tasmania trial. The gaps should have been identified and resolved at a far earlier stage.
I won’t go into as much detail in respect of the feedback from providers, as it’s generally quite similar to that from participants. The satisfaction levels are way too low. Two comments stand out:
Looking at the responses from NDIA Staff and Partners, we see no metric giving a 50% or above positive score EXCEPT the ones that relate to the need for change. Clearly, these people understand how significantly awful the old system is. The lack of high scores in any other aspect of measurement speaks volumes to how poorly the new system seems to achieve its purpose of delivering a better way of working. Buty having an unsatisfactory system is no justification for replacing it with an equally unsatisfactory system. Nobody side-grades their worn out car for another that is also worn out, unless they have motives other than having a better car!
According to the Introduction to the report, the new system is supposed to deliver an immense upgrade to the operational NDIS environment. Clearly, it has not done that for a substantial body of participants in the test.
The final sentence in chapter 8 of the report is significant: “NDIS partners thought we could improve how we explain the work we expect them (to) do, supported by training resources”. It has been clear for several years now that the NDIA has done very poorly in explaining the work that people are supposed to do, resulting in self-admitted deficiencies and inconsistencies in practice. To see a statement that NDIS Partners still need an uplift in the effectiveness of resources that should prepare people for working with the NDIS is particularly damning!
And then we arrive at Chapter ten – “What we are going to do next”. Again, speaking as a global thought leader in governance of IT, with 45 years of experience in helping organisations achieve higher performance through effective use of information technology, I can only say that the entire content of this chapter fills me with rage at the steaming incompetence of the NDIA in respect of exactly what this project was supposed to do – to deliver massive improvement in the way that NDIA performs its core function of administering the NDIS Act by providing reasonable and necessary supports to Australians who live with disability.
Take these three opening points:
Should have been happening from the beginning of the project – at least 2 years ago. It’s way too late to start this now.
Have I already mentioned that it cannot be only about a computer system? I think so. The NDIA is rolling out a BUSINESS SYSTEM, which has complex elements other than computer systems. They include people, practices, policies, controls, structures and myriad other elements that enable a business to function smoothly. It is already clear that there has been far too little attention to these elements, and history demonstrates over and over again that such failure almost always leads to catastrophic problems in operation.
Of course, this should be a standard practice. But it cannot be limited to improving the computer system. It must address the entire Business System.
The very fact that the document continually addresses computer system context rather than business system context makes it impossible to believe that the system is fit for purpose at this point in time, and raises the very vexing question of whether it can ever be made fit for purpose.
In Conclusion
In July 2018, IEEE Spectrum reported that: “An Australian Senate committee published a 146-page report assessing the government’s progress toward its goal of becoming ‘one of the top three digital governments in the world…that other nations can look to for guidance and inspiration,’ by 2025. Given what is in the report, other nations may want to look elsewhere for their inspiration”.
The situation has not improved. Australia enjoys an ignoble reputation as a World Leader in Government IT Failures.
Who can forget that remarkable time in 2005 when Australian Customs shut down the National Supply Chain.
Who can ignore the fortune still being wasted on the “Personally Controlled Electronic Health Record”, a chameleon that has had many names and forms and continues to defy success.
What about the Australian Apprenticeship Management System?
At the end of July, just a month ago, we saw that “The government has written off a $191 million investment sunk into building a new entitlements calculation engine (ECE) for Centrelink, with “nothing to show for it" in terms of a working system”. If that’s not clear, that was money wasted on a failed system intended to do, pretty much exactly what PACE has to do, except that the context of PACE is way more complex! And, being a Services (yes, that is an oxymoron too), Australia project, it has shared significant elements of top level leadership and management with the NDIA over time. Apparently, sources had said that the project had effectively been cancelled because the ECE did not work as intended. It probably lacked co-design too, as co-design should ensure that systems do actually work as intended, even if it takes longer and requires more people.
The ECE project had also been caught up in a scandal on how the contracts were awarded, which had invited further scrutiny. Again, these contracts were awarded under the same ministerial oversight as applied to PACE. Why would we ever think that PACE could have been immune from corrupt purchasing as well as perverse functional design.
The signs are unmistakable. Even without the extensive change that will be recommended by the NDIS Review, PACE can only be seen as another failure for the Australian Government. It should be shut down, immediately.
The NDIS Review appears to be pitching an entirely new operating model for the NDIS. A new operating model that can only be achieved by wholesale change, not just in enabling technology, but in the mindset, practices and rules of the National Disability Insurance Agency.
PACE should not rise like a phoenix from its own ashes. Rather, the Australian government should establish an entirely new leadership and governance framework for the ongoing development of new NDIS Business Systems.
The Governance arrangements must be designed using the global standard developed in Australia. It has been eschewed by every federal and state government to date, but analysis of every government failure and the few successes (especially in New South Wales) demonstrates its relevance. The governance arrangements must include expertise in this standard, expertise in the disability and professional communities that the new business systems are to serve, expertise in large scale, technology-enabled business change, expertise in compliance, safety and security, and expertise in design and delivery of 21st century government business systems.
In my experience, there is only one person in Australia who has the competence and track record required to lead this initiative. He was the absolute force behind the most successful digital transformation in government in Australia.
His name is @Victor Dominello. I hope that @Bill Shorten and @Anthony Albanese are smart enough to secure his help for at least three years, despite political differences.
What do you think?
Digital Specialist at Various Non Profit organizations
1ySorry, haven't had time to read the above yet. I am looking for the name of the system the NDIS use to record participant information, as I may supoena further information... thank you
Community Engagement champion, developing ways to better address the identified needs of our community, particularly our Older Persons and People with Disability
1yAnything started under the former SloMo "It's not my job" regime will be laced with problems and disrespect.
Advocate : Educator : Assistance Animal Specialist : Lived Experience
1yHear hear.
Digital Executive, Author and Advisor - Available for Consulting, Advisory Work and Interim CxO Services
1yHi Mark, thanks for tagging me in this article. I do not have direct experience with the NDIS, so my comments have limited specific domain context. I do however know a little about IT systems inside of government and how decision-making happens. The systems supporting disability service delivery to recipients are going to be complex, particularly from an experiential perspective. Good governance of these complex systems requires a holistic approach as you have pointed out, because great outcomes require all stakeholder needs to be met - funders, service providers, family carers and of course those in need of disability services, who we all know are extremely vulnerable in our communities. Without going into detail, there are some important governance tools that the Government can employ to ensure transparency and trust are maintained. This includes a system program board based on skills and representation from the diverse stakeholders. It starts there and then requires detailed implementation planning with lots of assurance checkpoints that are publicly accessible. The complex nature of the NDIS requires that systems such as PACE are based on continuous assessment, adjustment and effective change management.
Specialist in Digital Government, Transformation, Emerging Technologies, AI and Cyber Security
1yMark - While there is certainly no doubt about the number and scale of major ICT enabled change programs that have failed spectacularly to deliver any benefit - let alone some percentage of that promised in the original business case - it isn't entirely accurate to say that there has been a "...long and painful history of wasting taxpayer funds on so-called IT Initiatives ...". The evidence is that this experience has been achieved more directly through the policies adopted by the Coalition Government over the last decade. And that evidence has been provided by no less than that same government. In late 2013, the new Abbot Coalition Government commissioned AGIMO/Finance to undertake an independent audit of major ICT projects undertaken during the previous three years (ie under Labor's stewardship). The expectations was clearly that such an audit would uncover useful political ammunition to embarrass the Labor opposition. Unfortunately for those Ministers that commissioned it, the report found no major flaws in governance and that overall value for money was reasonable. The audit also demonstrated how effective management, oversight and assurance mechanisms contributed to this. It took an FOI request to force its release!