Translartion. Region: Russians Fedetion –
Source: Moscow Government – Government of Moscow –
Moscow scientists have developed new method testing of artificial intelligence (AI) systems in healthcare. It will allow for faster and more accurate assessment of their reliability. This was reported by Moscow’s chief radiologist Yuri Vasiliev.
“Moscow has been a leader in the use of artificial intelligence in medicine for many years, and now we have taken another important step forward. Scientists from the Center for Diagnostics and Telemedicine have conducted a large-scale study and found a new and reliable way to test AI systems in this area. Until now, there was no clear answer to the question of how many studies are needed to objectively test a neural network. We had to test on huge samples, which required a lot of time and resources. Now we know exactly how many studies are needed to evaluate the accuracy of AI, and when further increasing the sample does not change the result. This discovery will allow developers to quickly adapt their technologies, and doctors to use them more effectively in their work,” said Yuri Vasiliev.
He added that now the capital’s specialists will be able to implement AI services even faster, being confident in their reliability. The method has already proven its effectiveness in radiation diagnostics, but it can also be used in other areas of medicine. This approach will help make artificial intelligence an even more accurate and safe tool for doctors and patients, which means it will improve the quality of diagnostics and speed up the detection of diseases at early stages.
Scientists analyzed more than two million test variants and proved that at least 400 studies are needed to objectively assess the accuracy of binary classification algorithms (for example, identifying pathologies in images). Of these, at least 10 percent should belong to each of the classes, that is, have the noted signs of pathologies. Further increase in the sample does not change the result, which makes this method the most effective. The study was conducted using radiation diagnostics as an example, but this approach can be applied in other areas where AI works on the “yes or no” principle. This discovery will allow faster testing and implementation of artificial intelligence in medicine, increasing its accuracy and reliability.
“Classical methods of testing artificial intelligence did not give an exact answer to the question of how many studies are needed to objectively verify its accuracy. The complexity of the problems that AI solves in medicine is constantly growing, so scientists from the Center for Diagnostics and Telemedicine proposed an alternative approach. They analyzed more than two million combinations of test sample parameters and 25 thousand images, studied the behavior of diagnostic metrics and proved that at least 400 studies are needed to obtain a stable result. The minimum share of each class should be at least 10 percent, that is, 40 studies, and a further increase in the sample does not affect the final accuracy. The data obtained does not depend on the type of images or a specific neural network, which makes the method universal. The study was conducted using radiation diagnostics as an example, but this approach can be scaled to other medical AI systems with binary classification, which will be the next stage of scientific work,” added Yuri Vasiliev.
Article “An Empirical Method for Calculating Sample Size for Testing Artificial Intelligence Algorithms” has already received a positive review from academicians of the Russian Academy of Sciences and other involved parties and has become the winner of the AI Journey competition.
Since 2020, the Diagnostics and Telemedicine Center has been conducting the world’s largest prospective clinical study – an experiment on the implementation of computer vision for the analysis of medical images. Scientists have developed unique methods for a comprehensive assessment of the quality and maturity of AI technologies. In addition, they have substantiated specific methods and scenarios for the use of such technologies in the work of the radiation diagnostic service.
Center for diagnostics and telemedicine Moscow Department of Health— a leading scientific and practical organization in the structure of the social development complex of the Moscow City Hall. It was founded in 1996. The center specializes in the implementation of artificial intelligence technologies in medicine, the development of radiation diagnostics, the organization of the work of departments in medical institutions, conducting scientific research, and educating health workers.
The project is in line with the goals and objectives of the Moscow healthcare development strategy until 2030 and is aimed at improving the quality and accessibility of medical care for residents of the capital.
Get the latest news quicklyofficial telegram channel the city of Moscow.
Please note: This information is raw content directly from the source of the information. It is exactly what the source states and does not reflect the position of MIL-OSI or its clients.
Please Note; This Information is Raw Content Directly from the Information Source. It is access to What the Source Is Stating and Does Not Reflect
https: //vv.mos.ru/nevs/ite/152926073/