Shocking study reveals brain scan results look different if they are viewed on a Mac"

Discussion in 'Current Events' started by atlanticza, Jun 19, 2012.

  1. atlanticza macrumors 6502a


    Jul 18, 2008
    Cape Town
    From the UK's Daily Mail:

    Software used to analyse medical brain images gives significantly different results if it’s run on a Apple Mac or PC, according to new research.

    The findings by a team of German researchers has raised concerns
    about potential problems with software analysis.

    Data from 30 brain scans was fed into the computer using a package called FreeSurfer.

    Read more here
  2. iJohnHenry macrumors P6


    Mar 22, 2008
    On tenterhooks
    I will hold my shock until it is determined if this is better, or worse, in the treatment of patients. ;)
  3. Rodimus Prime macrumors G4

    Rodimus Prime

    Oct 9, 2006
    The fact that a update in OSX caused such a huge change is an issue. The other one can be correct for but an update to an OS should never cause such a massive change. The question comes is what the hell is Apple screwing with during the updates.
  4. chown33 macrumors 604

    Aug 9, 2009
    Sailing beyond the sunset
    Following links through to the original data, we read:
    FreeSurfer is a popular software package to measure cortical thickness and volume of neuroanatomical structures. However, little if any is known about measurement reliability across various data processing conditions. Using a set of 30 anatomical T1-weighted 3T MRI scans, we investigated the effects of data processing variables such as FreeSurfer version (v4.3.1, v4.5.0, and v5.0.0), workstation (Macintosh and Hewlett-Packard), and Macintosh operating system version (OSX 10.5 and OSX 10.6). Significant differences were revealed between FreeSurfer version v5.0.0 and the two earlier versions. These differences were on average 8.8±6.6% (range 1.3-64.0%) (volume) and 2.8±1.3% (1.1-7.7%) (cortical thickness). About a factor two smaller differences were detected between Macintosh and Hewlett-Packard workstations and between OSX 10.5 and OSX 10.6. The observed differences are similar in magnitude as effect sizes reported in accuracy evaluations and neurodegenerative studies.The main conclusion is that in the context of an ongoing study, users are discouraged to update to a new major release of either FreeSurfer or operating system or to switch to a different type of workstation without repeating the analysis; results thus give a quantitative support to successive recommendations stated by FreeSurfer developers over the years. Moreover, in view of the large and significant cross-version differences, it is concluded that formal assessment of the accuracy of FreeSurfer is desirable.
    Underlines added for emphasis.

    Summary: the largest differences were seen between FreeSurfer version v5.0.0 and its two earlier versions. Differences about half that size (factor of two smaller) were seen between Mac OS X 10.5 and 10.6 and H-P workstations (the H-P OS version or versions is unspecified in the article abstract, turns out to be CentOS 5.3 in complete article).

    So I agree with the conclusion of the abstract: a formal assessment of FreeSurfer's accuracy is called for. Speaking as a developer, I do wonder what the FreeSurfer developers might have done that lead to this degree of difference.

    And frankly, my shock is reserved for the apparent complete lack of formal accuracy assessment in medical software, free or not.

    Complete free pubmed article link:
  5. Mousse macrumors 68000


    Apr 7, 2008
    Flea Bottom, King's Landing
    Maybe it's just me, but... version 4.2.1 on the PC and version 5.0 on the Mac? Different results. And this was unexpected?:confused: Maybe if the ran the same version #'s the difference would be less newsworthy.:rolleyes:
  6. Dave00 macrumors 6502a

    Dec 2, 2003
    Nothing to see here. You can't compare results between different versions of the software - says so right on the software web site. There are many variables you can adjust on an MRI, CT, or PET scan that make them look very different based on the same data set - different thresholds, etc. That is probably all you're seeing.

    Move along. :)
  7. AnimaLeo macrumors 6502

    Sep 2, 2009
    For the benefit of you American's: The Daily Mail is not considered a 'serious' newspaper and is often filled with 'shocking' headlines just designed to make you buy the paper.
  8. niuniu macrumors 68020


    Mar 29, 2009
    A man of the people. The right sort of people.
    Too many variables.. not ready to be news yet..
  9. chrono1081 macrumors 604


    Jan 26, 2008
    Isla Nublar
    Thank you for that. I was not aware. So its kind of like our Weekly World News? (Although I'd assume not quite as extreme).
  10. AnimaLeo macrumors 6502

    Sep 2, 2009
    I'm not too sure I'm afraid. I think it might be a bit like FOX news. So the news is kinda skewed from a right-wing perspective, very anti-immergration etc etc.
  11. andiwm2003 macrumors 601


    Mar 29, 2004
    Boston, MA
    That article is lousy and written by a moron who doesn't know what he is talking about. It's a common problem that scientific analysis software can give different results between versions unless it is validated properly. So nothing new here and nothing that has anything to do with the OS per se.

    I'm just surprised that analysis software for brain scan doesn't seem to be certified by the FDA or similar authorities. That would ensure consitency.
  12. rhett7660, Jun 21, 2012
    Last edited: Jun 21, 2012

    rhett7660 macrumors G4


    Jan 9, 2008
    Sunny, Southern California
    Maybe it is because they are running two different versions of the software? That could never ever have been the issue? :rolleyes: It could never be the developer of the software screwing things up could it?
  13. Huntn macrumors P6


    May 5, 2008
    The Misty Mountains
    Ah, like the the Enquirer in the States?... :)
  14. sammich macrumors 601


    Sep 26, 2006
    Hmm, maybe that's why House and his team diagnosed people so well: because they were using Mac's.
  15. iJohnHenry macrumors P6


    Mar 22, 2008
    On tenterhooks
    Well, if they are wrong in their diagnosis, they can always cut open the resulting corpse, and learn from that. :p
  16. Dr McKay macrumors 68040

    Dr McKay

    Aug 11, 2010
    You lie, I'm staying in my Bunker before the immigrants flood in and destroy the country, shortly before the UK becomes completely submerged in water after Global Warming.
  17. eRondeau macrumors 65816


    Mar 3, 2004
    Canada's South Coast
    People BUY newspapers???
  18. AnimaLeo macrumors 6502

    Sep 2, 2009
    I've never bought one in my life. But I'm only 20.
  19. whooleytoo macrumors 604


    Aug 2, 2002
    Cork, Ireland.
    "Extra Extra! Software has bugs. Even critical medical software."

    The title is just a bit sensationalist; though the underlying problem is quite worrying. Medical software - and the underlying frameworks and libraries it relies upon - have bugs.


    Ooooh..... look at you with your "bunker"! I bet you can afford that because of ripping off the social welfare system, or being a nasty tax dodger; laughing all the way to the bank! ;)
  20. Carlanga macrumors 604


    Nov 5, 2009
    Here is the full official paper on it:

    and it's conclusion:

    The general conclusion from the present study and the practical advice it occasions is that users of FreeSurfer should exercise caution and restraint before applying a major upgrade in either the FreeSurfer (in particular) or OS version or to switch to a different type of workstation in the context of an ongoing study. This may be a truism and consistent with sound methodology for scientific experimentation and therefore a matter of common sense. However, the numerous questions about this issue posted by the user community seem to demonstrate the opposite and the results presented here reliably quantify the possible consequences. The message of caution applies not only to FreeSurfer but likely may be generalised to other intricate processing packages in the field of neuroimaging. The packages become more and more complex and therefore it is difficult to keep a check on propagation effects resulting from (small) modifications regarding one of the underlying algorithms.

    An additional message inferred from the present study is that authors reporting on results obtained with FreeSurfer are highly recommended to provide not only the version of FreeSurfer that was used but also details on the OS version and workstation.

    Finally, given the large and significant differences between the latest version v5.0.0 and earlier versions, it is concluded that an assessment of the accuracy of FreeSurfer is desirable.
  21. iJohnHenry macrumors P6


    Mar 22, 2008
    On tenterhooks
    Some QA, or Model Office, would be appropriate, given that people's lives are at risk.
  22. Tsuchiya macrumors 68020


    Jun 7, 2008
    AnimaLeo is just being polite. The Daily Mail is is a pile of **** that should be avoided.
  23. bartelby macrumors Core


    Jun 16, 2004
    Ammended for accuracy.:)
  24. rbrian macrumors 6502a


    Jul 24, 2011
    Aberdeen, Scotland
    This website gives you a sample of the kind of headlines you'll find in the Daily (hate)Mail:

    I'm pretty sure the writers use this, and then concoct a mostly made-up story around the headline.
  25. iJohnHenry macrumors P6


    Mar 22, 2008
    On tenterhooks
    Adverts drive the news, not the other way around.

    Same with TV.

Share This Page