AI competitions don ’t produce useful models

By LUKE OAKDEN-RAYNER A huge new CT brain dataset was released the other day, with the goal of training models to detect intracranial haemorrhage. So far, it looks pretty good, although I haven’t dug into it in detail yet (and the devil is often in the detail). The dataset has been released for a competition, which obviously lead to the usual friendly rivalry on Twitter: Of course, this lead to cynicism from the usual suspects as well. And the conversation continued from there, with thoughts ranging from “but since there is a hold out test set, how can you overfit?” to “the proposed solutions are never intended to be applied directly” (the latter from a previous competition winner). As the discussion progressed, I realised that while we “all know” that competition results are more than a bit dubious in a clinical sense, I’ve never really seen a compelling explanation for why this is so. Hopefully that is what this post is, an explanation for why competitions are not really about building useful AI systems. DISCLAIMER: I originally wrote this post expecting it to be read by my usual readers, who know my general positions on a range of issues. Instead, it was spread widely on Twitter and HackerNews, and it is pretty clear that I didn’t provide enough context for a number of statements made. I am going to write a follow-up to clarify several things, but as a quick response to sev...
Source: The Health Care Blog - Category: Consumer Health News Authors: Tags: Health Tech AI Luke Oakden-Rayner Source Type: blogs

Related Links:

New J. Chem., 2019, Accepted Manuscript DOI: 10.1039/C9NJ04551A, PaperYitong Xie, Di Wei, Tian Hu, Yajing Hou, Yuanyuan Lin, Huaizhen He, Cheng Wang Uncaria rhynchophylla (UR) is a widely used traditional Chinese herb for hyperpyrexia, epilepsy, preeclampsia, and hypertension. In this paper, the potential anti-pseudo-allergic components in UR were explored. Six alkaloids in... The content of this RSS Feed (c) The Royal Society of Chemistry
Source: RSC - New J. Chem. latest articles - Category: Chemistry Authors: Source Type: research
INTRODUCTION: Orbitocranial wooden foreign body (OWF) penetrations are rare but challenging occurrences that may violate the intracranial space resulting in brain damage and hemorrhagic, as well as infectious, complications. Moreover, there is a specific s...
Source: SafetyLit - Category: International Medicine & Public Health Tags: Economics of Injury and Safety, PTSD, Injury Outcomes Source Type: news
The most common treatment against migraine is unsafe for people at risk of cardiovascular disease and stroke, but a new drug against migraine has shown promise in a large-scale clinical trial, offering hope of an alternative treatment.
Source: CNN.com - Health - Category: Consumer Health News Source Type: news
ConclusionOur results provide a preliminary explanation of the basic pharmacological effects of JWZHS against RTIs and characterize the potential underlying mechanisms. These findings provide a foundation for experimental research and eventual clinical application.
Source: European Journal of Integrative Medicine - Category: Complementary Medicine Source Type: research
Publication date: January 2020Source: American Heart Journal, Volume 219Author(s): Salvatore Cassese, Gjin Ndrepepa, Robert A. Byrne, Karl-Ludwig Laugwitz, Heribert Schunkert, Massimiliano Fusaro, Fernando Alfonso, Adnan KastratiBackgroundRandomized trials did not consistently support superiority of ticagrelor, as monotherapy or in combination with aspirin, in terms of efficacy or safety, in patients with atherosclerotic artery disease.MethodsMedline, EMBASE, the Cochrane Central Register of Controlled Trials, and scientific session abstracts were searched for trials of patients with coronary or peripheral artery disease (...
Source: American Heart Journal - Category: Cardiology Source Type: research
This study measured the residual stress in full-scale laser cladded rails where the residual stresses were measured in the cladding layer, heat affected zone (HAZ) and substrate zone of the railhead. A new sample preparation strategy was developed to quantify the residual stresses in the full-scale rails with high spatial resolution. Higher compressive residual stress was found in the cladding layer, which may have resulted from the martensitic transformation occurred in that region. Tensile stresses occurred in the HAZ to a depth of 4 mm, which might be mainly caused by thermal contraction and volumetric change in ...
Source: Journal of Materials Processing Technology - Category: Materials Science Source Type: research
ConclusionAll collected data were observational, which precluded making strong recommendations. Furthermore, all CIs were wide, increasing the uncertainty of the observed effects. As such, evidence was insufficient to recommend rhythm or rate control as the first-line therapy for AF in older adults. As AF is particularly prevalent in older people, more randomized controlled trials are needed in this population.
Source: Drugs and Aging - Category: Geriatrics Source Type: research
AbstractA strong relationship exists between tumor and inflammation, which is the hot point in cancer research. Inflammation can promote the occurrence and development of cancer by promoting blood vessel growth, cancer cell proliferation, and tumor invasiveness, negatively regulating immune response, and changing the efficacy of certain anti-tumor drugs. It has been demonstrated that there are a large number of inflammatory factors and inflammatory cells in the tumor microenvironment, and tumor-promoting immunity and anti-tumor immunity exist simultaneously in the tumor microenvironment. The typical relationship between ch...
Source: EPMA Journal - Category: International Medicine & Public Health Source Type: research
This release appeared last week:Statewide Network for Clinical Data Sharing Reduces Healthcare Costs by an Estimated $160-195 Million AnnuallyPosted on November 12, 2019 by NYeC Staff FOR IMMEDIATE RELEASE: November 12, 2019 New analysis estimates potential savings of $1 billion once the Statewide Health Information Network for New York is fully leveraged ALBANY, NY – Today the New York Health Collaborative (NYeC) announced that use of the Statewide Health Information Network for New York (SHIN-NY) is reducing unnecessary healthcare spending in New York State by $160-$195 million annually—including significant ...
Source: Australian Health Information Technology - Category: Information Technology Authors: Source Type: blogs
es MA Abstract The sympathetic nervous system (SNS) regulates body functions in normal and pathological conditions and is characterized by the presence of a neuroplastic phenomenon, termed ganglionic long-term potentiation (gLTP). In hypertension, either in spontaneously hypertensive rats (SHR) or in humans, sympathetic hyperfunction, such as elevated SNS outflow and changes in synaptic plasticity have been described. Because enhanced SNS outflow is detected in the hypertensive stage and, more importantly, in the prehypertensive phase of SHR, here we explored whether synaptic plasticity, particularly gLTP, was mod...
Source: Neural Plasticity - Category: Neurology Authors: Tags: Neural Plast Source Type: research
More News: Academia | Australia Health | Blogging | Brain | Child Development | Epidemiology | Health | Health Management | Hemorrhagic Stroke | Hypertension | International Medicine & Public Health | Internet | Learning | Neurology | Pneumonia | Pneumothorax | Radiology | Statistics | Stroke | Training | Universities & Medical Training | X-Ray