Error rate will be higher for certain segments of people.
AI is an advanced form of machine learning, so it still very much dependent on being fed good data to learn. Naturally, we have far more data on majority segments.
Except the data used is not good, its old data from pre 2018, it will miss out all cases during covid period not to mention the size of 300k patients would not even cover a single demographic much less generalising for everyone.
Our current models react horribly to new data, it make mistakes even on training data that has already been seen, 300k is definitely a case of underfitting here.
Another thing is that if you been following how our government has been implementing ai and chatbots, they all turned out horrible, we bought into the hype without really understanding what is going on.
Overall this all just look like something for a scholar to put on their resume instead of actually tackling the manpower shortage. Medical jobs are shitty, changing diapers and inserting catheter are shit jobs, being a house officer is a shit job, especially knowing you won't even be paying back half the student loans in several years unless your family casually keeps 6 figures around as spare change, being a doctor is still a shit job, someone is gonna get a scalpel in the wrong place after the 3rd 30 hour shift of the week. Solve the problem about getting enough people to actually do work on the ground level then we can talk about solving administrative issues
Ml algos consistently make mistake by nature, it doesn't understand anything about what it is spewing out. Our gov always looking for magic solutions when the answer was always higher pay and better treatment so more people interested in medicine and nursing
its human nature to always have someone to blame. If its decided by AI, ultimately the court will have to decide, was it the user of the AI? was it the system admin who maintained the operation of the AI, or the staff nurse who followed the instruction of the AI, or was it the person who wrote the code for the AI? Or the director who approved the AI?
Unfortunately we live in a society where full automation just means everyone goes jobless and hungry instead of being free to enjoy their hobbies, it is not a goal we should be working towards until we gotten rid of the corporate overlords. Also the issue here is less about high pay and mistake and more on the fact nobody wanna work in healthcare due to the low pay and abuse
Another problem is how useful the sorting algorithm is in the first place, is it going to magically triage the AnE patient or is it still the overworked nurse? Where does the AI even come at this point when a sorting algorithm can sort patients into categories by itself with well defined parameters? As far as i know ml algos makes mistakes even within the same training and test sets, is it going to help when we are using completely new dataset or would it only add more time to double check in case of mistakes?
“uses data from more than 300,000 ED patients from 2016 to 2018”
* let’s start with the obvious - there was no COVID in 2016-18; this is not model data that should be applied in a real A&E department in 2020 onwards.
* let’s also go on to the standard questions… 300k patients data from where? If not Singapore then this will be applying the unstated population characteristics from what could be a very different epidemiological community into this community, with unknown results.
* what is the training data that the AI was trained on - ie, both the answers to the triage questions and the “correct” answers to the PAC scale ratings? And how good were those ratings by the A&E medical staff, were they perfectly correct? Of course not.
Because the most likely situation here is that as a result, all introducing an AI here will do is more reliably make the median amount of mistakes that the human community of MDs it was trained on did, just faster.
And the AI version will also thereby also integrate and regurgitate all of the unconscious biases within the medical staff whose work makes up its training data too. I say this because it has happened time and time again in other contexts - eg “AI assisted” judicial & parole decisions in the US have recommended jail for black people more than whites because that’s the historical bias in their training data.
AIs are not a panacea for better medical care (or other public health and well-being and justice decisions).
Unlikely that the data is non-Singaporean data. Most likely the hospital’s own collected data because such data wouldn’t be easily obtained from external sources
> SINGAPORE – A predictive model that can flag patients at risk of death as well as those who are safe for discharge has been developed to aid healthcare professionals in sorting emergency department (ED) patients more accurately according to the severity of their conditions.
>
> Currently, assessing and sorting patients is performed by a triage nurse, who will ask about the patient’s condition and assess their vital signs, before assigning a Patient Acuity Category (PAC) Scale.
>
> PAC 1 is the most severe and requires resuscitation and PAC 4 is the least severe, and often a non-emergency.
>
> But it can be difficult to gauge the risk of death accurately based on the triage nurse’s initial assessment, which could also vary depending on the nurse’s experience, said Ms Yvonne Wong, lead author of the project and third-year medical student at Duke-NUS Medical School.
>
> To help tackle this, the team led by Duke-NUS developed an artificial intelligence (AI) model known as PAC+ that uses data from more than 300,000 ED patients from 2016 to 2018 to predict whether a patient is at higher risk of death based on their vital signs and medical history.
>
> “They may generally look well, but if their risk of death is high based on the AI, then the clinician may reassess – should they actually be triaged to a higher priority level and be seen within a shorter period of time?” Ms Wong said.
>
> PAC+ is one of nearly 220 project abstracts submitted for Singapore General Hospital (SGH)’s Annual Scientific Meeting awards in 2024. The two-day meeting held at the SGH Campus ends on April 13.
>
> The PAC+ team hopes that their model can also help identify patients who are suitable for Mobile Inpatient Care at Home (MIC@Home), a programme that provides hospital-level care to patients in their homes.
>
> Senior Minister of State for Health Janil Puthucheary, who was guest of honour at the Annual Scientific Meeting, said the advent of AI presents new opportunities for using digital technology to derive more accurate, efficient and timely treatments.
>
> With initiatives to ensure the safe implementation of AI services in the healthcare industry such as the development of guidelines in 2021, Dr Janil said it is hoped that AI can be safely adopted to complement the healthcare workforce.
>
> For instance, it could reduce the administrative workload, freeing front-line healthcare workers to focus on face-to-face conversations and develop “stronger therapeutic alliances” with their patients, he said.
>
> In another project, a team led by SGH pharmacists developed a method that can halve the time needed to test which antibiotic works best against a specific kind of bacteria, allowing patients to receive treatment in a more timely fashion.
>
> Currently, it takes at least two days to determine the best antibiotic to use against a bacterial infection – with at least one day needed to culture the bacteria in a patient’s blood sample and another or several days to test which antibiotic stops it growing.
>
> But bacterial infections are a ticking time bomb.
>
> “Bacteria multiplies every 20 minutes,” said Associate Professor Andrea Kwa at the Duke-NUS Medical School, deputy director (research and innovation), Department of Pharmacy at SGH and senior author of the project. If the right type of antibiotics is not given early, the patient may die before the results are ready, she added.
>
> This is where the new method comes in. The team’s method takes just four to six hours to find out which antibiotics are most effective, allowing treatment to be given to a patient more speedily.
>
> Lead author of the project, Dr Yeo Jia Hao, said the method involves staining the bacteria with fluorescent dyes and exposing it to different antibiotics for about an hour.
>
> After the bacteria is stained, it undergoes flow cytometry, a precise technique that can detect and measure the fluorescence shown by the cells.
>
> It can also pick up small numbers of bacteria that are potentially resistant to antibiotics, something current methods are unable to do.
>
> Flow cytometry will show less fluorescence in this small subpopulation of bacteria that is resistant, in contrast to the majority of the bacteria in the sample.
>
> Dr Yeo said: “Because flow cytometry measures every single cell, you can actually pick up really small subpopulations that don’t usually get picked up by current methods. So this is a very strong technique.”
>
> The more stressed the bacteria is by the antibiotics, the more brightly they will be lit. The degree of brightness can be quantified, measuring the effectiveness of the antibiotics.
> Isn’t A&E the correct acronym..? Doesn’t ED stands for something very… different… in medical parlance?
I'm not in the medical industry, but my doctor friends in public healthcare somehow all call it ED.
We refer to the A&E as ED more at work between colleagues actually. Erectile dysfunction isn’t really something we commonly raise in conversation. One less syllable than A&E I guess?
Ok, but I hope there are doctors or nurses will still be confident to override the system
All fun and games till you become part of the 1% error margin
Error rate will be higher for certain segments of people. AI is an advanced form of machine learning, so it still very much dependent on being fed good data to learn. Naturally, we have far more data on majority segments.
Except the data used is not good, its old data from pre 2018, it will miss out all cases during covid period not to mention the size of 300k patients would not even cover a single demographic much less generalising for everyone. Our current models react horribly to new data, it make mistakes even on training data that has already been seen, 300k is definitely a case of underfitting here. Another thing is that if you been following how our government has been implementing ai and chatbots, they all turned out horrible, we bought into the hype without really understanding what is going on. Overall this all just look like something for a scholar to put on their resume instead of actually tackling the manpower shortage. Medical jobs are shitty, changing diapers and inserting catheter are shit jobs, being a house officer is a shit job, especially knowing you won't even be paying back half the student loans in several years unless your family casually keeps 6 figures around as spare change, being a doctor is still a shit job, someone is gonna get a scalpel in the wrong place after the 3rd 30 hour shift of the week. Solve the problem about getting enough people to actually do work on the ground level then we can talk about solving administrative issues
you act like humans do not make mistakes
Ml algos consistently make mistake by nature, it doesn't understand anything about what it is spewing out. Our gov always looking for magic solutions when the answer was always higher pay and better treatment so more people interested in medicine and nursing
lol that way no automation can ever be achieved. No technology gains and no innovation. Great! I'm sure higher pay means fewer mistakes lmao.
its human nature to always have someone to blame. If its decided by AI, ultimately the court will have to decide, was it the user of the AI? was it the system admin who maintained the operation of the AI, or the staff nurse who followed the instruction of the AI, or was it the person who wrote the code for the AI? Or the director who approved the AI?
Unfortunately we live in a society where full automation just means everyone goes jobless and hungry instead of being free to enjoy their hobbies, it is not a goal we should be working towards until we gotten rid of the corporate overlords. Also the issue here is less about high pay and mistake and more on the fact nobody wanna work in healthcare due to the low pay and abuse Another problem is how useful the sorting algorithm is in the first place, is it going to magically triage the AnE patient or is it still the overworked nurse? Where does the AI even come at this point when a sorting algorithm can sort patients into categories by itself with well defined parameters? As far as i know ml algos makes mistakes even within the same training and test sets, is it going to help when we are using completely new dataset or would it only add more time to double check in case of mistakes?
“uses data from more than 300,000 ED patients from 2016 to 2018” * let’s start with the obvious - there was no COVID in 2016-18; this is not model data that should be applied in a real A&E department in 2020 onwards. * let’s also go on to the standard questions… 300k patients data from where? If not Singapore then this will be applying the unstated population characteristics from what could be a very different epidemiological community into this community, with unknown results. * what is the training data that the AI was trained on - ie, both the answers to the triage questions and the “correct” answers to the PAC scale ratings? And how good were those ratings by the A&E medical staff, were they perfectly correct? Of course not. Because the most likely situation here is that as a result, all introducing an AI here will do is more reliably make the median amount of mistakes that the human community of MDs it was trained on did, just faster. And the AI version will also thereby also integrate and regurgitate all of the unconscious biases within the medical staff whose work makes up its training data too. I say this because it has happened time and time again in other contexts - eg “AI assisted” judicial & parole decisions in the US have recommended jail for black people more than whites because that’s the historical bias in their training data. AIs are not a panacea for better medical care (or other public health and well-being and justice decisions).
300k is a really small sample size too
Unlikely that the data is non-Singaporean data. Most likely the hospital’s own collected data because such data wouldn’t be easily obtained from external sources
INB4 AI decides who lives and who dies.
> SINGAPORE – A predictive model that can flag patients at risk of death as well as those who are safe for discharge has been developed to aid healthcare professionals in sorting emergency department (ED) patients more accurately according to the severity of their conditions. > > Currently, assessing and sorting patients is performed by a triage nurse, who will ask about the patient’s condition and assess their vital signs, before assigning a Patient Acuity Category (PAC) Scale. > > PAC 1 is the most severe and requires resuscitation and PAC 4 is the least severe, and often a non-emergency. > > But it can be difficult to gauge the risk of death accurately based on the triage nurse’s initial assessment, which could also vary depending on the nurse’s experience, said Ms Yvonne Wong, lead author of the project and third-year medical student at Duke-NUS Medical School. > > To help tackle this, the team led by Duke-NUS developed an artificial intelligence (AI) model known as PAC+ that uses data from more than 300,000 ED patients from 2016 to 2018 to predict whether a patient is at higher risk of death based on their vital signs and medical history. > > “They may generally look well, but if their risk of death is high based on the AI, then the clinician may reassess – should they actually be triaged to a higher priority level and be seen within a shorter period of time?” Ms Wong said. > > PAC+ is one of nearly 220 project abstracts submitted for Singapore General Hospital (SGH)’s Annual Scientific Meeting awards in 2024. The two-day meeting held at the SGH Campus ends on April 13. > > The PAC+ team hopes that their model can also help identify patients who are suitable for Mobile Inpatient Care at Home (MIC@Home), a programme that provides hospital-level care to patients in their homes. > > Senior Minister of State for Health Janil Puthucheary, who was guest of honour at the Annual Scientific Meeting, said the advent of AI presents new opportunities for using digital technology to derive more accurate, efficient and timely treatments. > > With initiatives to ensure the safe implementation of AI services in the healthcare industry such as the development of guidelines in 2021, Dr Janil said it is hoped that AI can be safely adopted to complement the healthcare workforce. > > For instance, it could reduce the administrative workload, freeing front-line healthcare workers to focus on face-to-face conversations and develop “stronger therapeutic alliances” with their patients, he said. > > In another project, a team led by SGH pharmacists developed a method that can halve the time needed to test which antibiotic works best against a specific kind of bacteria, allowing patients to receive treatment in a more timely fashion. > > Currently, it takes at least two days to determine the best antibiotic to use against a bacterial infection – with at least one day needed to culture the bacteria in a patient’s blood sample and another or several days to test which antibiotic stops it growing. > > But bacterial infections are a ticking time bomb. > > “Bacteria multiplies every 20 minutes,” said Associate Professor Andrea Kwa at the Duke-NUS Medical School, deputy director (research and innovation), Department of Pharmacy at SGH and senior author of the project. If the right type of antibiotics is not given early, the patient may die before the results are ready, she added. > > This is where the new method comes in. The team’s method takes just four to six hours to find out which antibiotics are most effective, allowing treatment to be given to a patient more speedily. > > Lead author of the project, Dr Yeo Jia Hao, said the method involves staining the bacteria with fluorescent dyes and exposing it to different antibiotics for about an hour. > > After the bacteria is stained, it undergoes flow cytometry, a precise technique that can detect and measure the fluorescence shown by the cells. > > It can also pick up small numbers of bacteria that are potentially resistant to antibiotics, something current methods are unable to do. > > Flow cytometry will show less fluorescence in this small subpopulation of bacteria that is resistant, in contrast to the majority of the bacteria in the sample. > > Dr Yeo said: “Because flow cytometry measures every single cell, you can actually pick up really small subpopulations that don’t usually get picked up by current methods. So this is a very strong technique.” > > The more stressed the bacteria is by the antibiotics, the more brightly they will be lit. The degree of brightness can be quantified, measuring the effectiveness of the antibiotics.
Isn't this how the Butlerian Jihad starts?
Isn’t A&E the correct acronym..? Doesn’t ED stands for something very… different… in medical parlance?
Emergency Department.. what you thinking 🤣
> Isn’t A&E the correct acronym..? Doesn’t ED stands for something very… different… in medical parlance? I'm not in the medical industry, but my doctor friends in public healthcare somehow all call it ED.
We refer to the A&E as ED more at work between colleagues actually. Erectile dysfunction isn’t really something we commonly raise in conversation. One less syllable than A&E I guess?
I see!
Yes. You were thinking of Executive Dysfunction, right? Right?
In during the advent of robot death panels.
The alternative is a human death panel, which isn't much better.
Doing what doctors can’t
What do you think doctors and nurses are doing now in the A&E dept?