Article
When planning to adopt artificial intelligence-based tools in the health service, several questions and assessments are of key importance for a well-structured process and safe introduction.
Illustration: Espen Friberg
The Healthcare Personnel Commission highlights artificial intelligence as one of the technologies that can help improve services in the Norwegian health sector and make them more efficient (1) . Hundreds of tools based on artificial intelligence (AI) are already available on the market (2) , but few will be adopted for use in clinical practice due to various obstacles and challenges. Although AI-based tools show great potential, most of them are still at the development stage or undergoing clinical trials. The process from laboratory to hospital bed, from development to implementation requires time and resources, and many tools and solutions are dropped underway.
When planning to use AI-based tools in the healthcare services, it is vital to understand that there are significant differences between ordinary software and AI products. This is of key importance in realising the innovative potential of the technology and managing risk in an appropriate manner. To support enterprises that want to adopt the technology, DNV (formerly Det Norske Veritas ) published a report in January 2023 with suggestions as to how to plan and structure implementation processes for AI-based tools (3) . The report identifies key questions that need to be answered, and provides an overview of necessary assessments.
Ordinary software and artificial intelligence
Ordinary software and artificial intelligence
There are many similarities between AI-based tools and conventional software, but AI tools have a number of unique and challenging features that make them more difficult to implement. An important difference between AI products and other IT products is that traditional software uses code while AI products use code + data. This affects how solutions are evaluated, procured and managed. For example, stipulating requirements in respect of safety, performance and uptime, as is customary with traditional software, will no longer be adequate. The extra complexity of AI-based tools, where data is a key component of the product, demands a more thorough assessment and evaluation.
The extra complexity of AI-based tools, where data is a key component of the product, demands a more thorough assessment and evaluation
AI-based tools must be trained and tested on data. To reduce the risk of error, it is important to have good routines ensuring that the dataset on which the tool is based is representative of the demographic group or patient population for which the tool will be used. The tools must be adapted to the logic and rules that are necessary for the completion of a task. Normally, a considerable body of data is needed to train AI-based tools in order to obtain good predictions and classifications. More training data – both greater volume and diversity – can generally, but not always, give greater precision. This affects both the procurement and management of AI-based tools. What data are used and are they relevant for the planned use? How has the tool been 'trained'? Have demographic differences, for example, been taken into account, and how will the tool be further trained after procurement? These are some of the many important questions that make procurement, implementation and management a complicated process.
The three phases of the implementation process
The three phases of the implementation process
Prior to procurement
Prior to procurement
Prior to procurement, enterprises should spend time clearly identifying and formulating the problem that this particular procurement is intended to solve. The end user is key, because without their involvement and ownership, it is difficult to ensure acceptance and use of the solution. Clarification of whether it is possible and appropriate to solve the problem with an AI-based tool is essential. When the problem has been identified, enterprises must check whether there are products on the market that can solve it. If not, an option may be to start the process of developing a product independently or together with a research partner. In this connection, it is vital to clarify whether it is potentially a product that can benefit patients at other institutions, and whether it can be classified as medical equipment. If so, a plan should be devised to obtain CE marking.
Procurement
Procurement
If a decision is made to purchase an AI-based tool, the user must evaluate potential suppliers and products. What documentation is there of clinical effect and safety? Is the product compatible with local data and infrastructure? Is further confirmation or validation required to demonstrate that the tool is appropriate for the patient population? Should a small pilot project be carried out prior to full implementation? Do the potential advantages of the tool outweigh the anticipated costs? Testing and trialling different solutions at an early stage of the project can provide good understanding of both the solutions and the problem to be solved.
Following procurement
Following procurement
The product must be fully implemented in the users' technical infrastructure and integrated in the clinical workflow, preferably with the support of an interdisciplinary team. It is good practice to ensure that throughout the entire lifecycle of the AI-based tool, there is a plan in place for monitoring performance, as this can change over time in line with changes in the real-world environment, so-called model drift, and for identifying adverse events and new risks.
Development and implementation
Development and implementation
Legal and regulatory assessments
Legal and regulatory assessments
When adopting AI-based tools in clinical practice, challenges linked to data quality and quantity, confidence in the models and complex legal and regulatory requirements must be addressed in collaboration with all the involved parties.
It is vital to find a balance between using and protecting the data. The identification of applicable statutes and regulations for access to data can pose a challenge for many AI projects. The Norwegian national coordination project for the adoption of AI in the health sector, a multi-agency coordination project in which all agencies with a supervisory responsibility for a relevant statute or regulation participate, has established a webpage in order to provide a better overview of the applicable body of regulations (4) .
Another legal challenge is dealing with intellectual property rights. In this connection, technology transfer organisations at universities and hospitals, such as VIS (5) or Inven2 (6) , play a key role.
Health data and algorithms
Health data and algorithms
Developers of AI models find the various challenges posed by health data, such as accessibility, data sharing, quality, bias etc., difficult to handle. Poor training data can impact the entire value chain negatively, and this may trigger a domino effect: poor data can lead to poor models, leading in turn to poor performance when using own data. Therefore it is vital to secure access to reliable and representative high-quality datasets. Moreover, it is wise to develop and employ a strategy for data management and governance to facilitate the development and implementation of AI technology.
The National Strategy for Artificial Intelligence (7) affirms that the use of AI in Norway must be responsible and reliable, building on ethical principles, respect for privacy and good digital security. In the case of the health service, it will be essential to address any bias in the datasets used to create the models, and to secure equal access to this technology. Meanwhile, new digital solutions can contribute to equitable medical care regardless of geographical location. Professional accountability requirements apply, and legal responsibility for adverse events or injuries must be clarified (8) . Decision-makers and administrators must have an overarching approach to the regulation of access to medical data, including secondary use of data and consent, anonymisation and pseudonymisation, as well as intellectual property rights and any use of third-party data.
Integration
Integration
To ensure that tools based on artificial intelligence are accepted and actually used by end users – who may be health personnel or patients – it is important to integrate the tools adequately into the enterprise's infrastructure and clinical workflow. They must support the daily work, not lead to extra work. This can be a challenge, particularly when AI-based tools must be integrated with older systems that were not originally designed for artificial intelligence. Therefore, it is important at an early stage to involve an interdisciplinary team composed of end-users, IT technicians responsible for integration in the existing workflow and infrastructure, and suppliers.
To ensure that tools based on artificial intelligence are accepted and actually used by end user, it is important to integrate the tools adequately into the enterprise's infrastructure and clinical workflow
Another solution may be to procure a platform that integrates everything from the start. Regardless of what is chosen, it is essential to ensure interoperability and technical integration. A monitoring plan with clear roles for all involved parties should be devised. The performance and results of AI-based tools must be monitored in order to detect model drift and mitigate the risk of diminishing predictive ability and increased uncertainty. AI-based tools require more stringent suitability testing to ensure coverage of data and infrastructure needs.
Management and governance
Management and governance
Competent management is one of the main criteria for the successful introduction of this technology in the health service. Management must undertake a number of strategic prioritisations linked to AI implementation, including a plan for putting into effect changes in the enterprise and work processes – i.e. change management. It is also necessary to develop and implement suitable governance tools and management systems to provide good internal control of the technology and to ensure that the tool is adapted to the relevant patient population and infrastructure. The organisation should devise a plan for dealing with the risk of adverse events associated with the use of AI-based tools, carry out a health economic analysis to assess their value in relation to clinical results, put in place a tendering process to select the tools, and secure the necessary funding and resources prior to committing to implementing the use of AI.
Culture and receptiveness
Culture and receptiveness
Receptiveness and trust are vital in creating a robust culture around the use of artificial intelligence in the health service. Expectation management is an element of this. If managers and end users avoid overestimating or underestimating the capability of the technology and what it will entail, the introduction of AI will be smoother. The health service should find clinicians who are interested in artificial intelligence and can help promote acceptance of and receptiveness to the technology. The results of suitability testing and validation of AI-based tools should be presented and made available to end users. This will give them better insight into how the technology functions and what it can contribute.
Competence
Competence
The availability of competence is important for the successful implementation of AI in the health service. Managers must have sufficient expertise to provide the necessary support and resources and be able to undertake good prioritisations. Employees must know enough about the technology to perform their roles in a confident and responsible manner. This can be achieved by offering relevant training programmes, allocating sufficient resources for employees to enjoy continuing professional development, and implementing research and innovation projects.
Overarching approach
Overarching approach
Implementing AI systems is an iterative process requiring suitable systems that can manage data as well as processes for good integration of the systems in both workflow and infrastructure. Moreover, knowledge of the relevant regulations, competent managers and skills development for both health personnel and patients are essential. An overarching approach ensures that we are better equipped for the successful and safe introduction of this technology.