Having had a quick read-through, there are a number of glaringly obvious problems – the paper would merit a fail at GCSE level, let alone any higher.
Aside from apparent failure to introduce blinding, which pretty much invalidates the results (such as they are) by itself, there are two major problems relating to the control group size. In the first place, it’s not big enough to say anything accurately – 10 rats isn’t enough to draw conclusions about development of tumours, since that’s something rats living to old-age (in lab settings; doesn’t normally happen in the wild) are overwhelmingly prone to. Moreover, they compare 9 sets of ‘treated’ rats to just one control group, giving them 9 chances to find one group which has higher mortality. When you account for that, their finding is actually the opposite of what they claim: seven out of eight groups showed no effect.
Kj
One has to be open for the possibility that this is a problem with RR-maize, and/or Roundup, even if organic ag. isn’t the answer and we love up-and-coming tech. The thrashing has been mostly about unavailable data-sets it seems.
The control group is too small. The work does raise concerns, but they need to redo it properly. There goes another two years dissecting rat tumours. It’s a pity they didn’t seek advice on experimental design before they started.
I dislike what seems to be data mining in the paper. “The maximum difference in males was 5
times more deaths occurring during the 17th month in the group consuming 11% GM maize, and in females 6 times greater mortality during the 21st month on the 22% GM maize diet with and without R.” When they redo this, they should say in advance what numbers they’re going to look at.
I don’t follow Dave’s remark about mortality. In Figure 1, mortality in female rates does seem to be lower in the control group compared with all doses.
Of course the work should be blinded, and it’s a concern that they don’t say it was. But I wouldn’t ignore it just because of that: there’s little potential for lack of blinding to introduce major errors.
James S
Surely there needs to be a mechanism as well to make it convincing. But there doesn’t appear to be any / much in the way of correlation between dose and tumours (indeed a brief skim looks like it could be negative).
Dave
PaulB>
Now I re-read it, I don’t follow my remark about mortality either. It has the look of a sentence I was writing whilst switching back and forth to the source doc, and got confused. Frankly, I can’t be bothered to go back and re-read that turgid tripe – the paper, I mean, although probably the same could be said of my comment – and find out if what I think I meant has any validity.
john77
Despite the appalling experimental design which may be deliberate (after reading the Abstract I felt that they were looking for data to support their conclusion, so I am not as confident as PaulB that they did not seek advice on experimental design) there seems to a case to answer for female rats. However, the lower incidence for rats fed the highest concentration of GMO maize is a prima facie argument for the results being unreliable. Also one should remember that a serious group of researchers found that high dosages of dihydrogen monoxide produced tumours in rats.
That said, there is, IMHO, a significant difference between GMOs which simply replicate a couple of centuries of selective breeding to combine disease-resistance with good yields and Monsanto’s moving outside that range to produce something mimicking but unrelated to natural crops that are resistant to its proprietary herbicides. IF, and it is a big IF, this experiment can be replicated under scientific conditions it will raise doubts over Monsanto’s GMOs but that would not apply to the”green revolution” high-yield rice that has been in use in India for a generation (unless someone can tell me that the Indian rat population has died out).
Yes, the paper contains a number of obvious flaws, which raise suspicions about its provenance, not least because it lacks any discussion of its own limitations, which is often a bad sign.
I’ve not much to add to what’s already been discussed save for the usual reminder that rats are not humans and so it doesn’t automatically follow that an effect seen in rats will necessarily occur in human to the same degree, or even at all.
I’d also add that an 11-22% GM maize diet strikes me as a hell of lot of corn dogs for a human, so unless you’re from a culture in which its absolute staple part of the diet, its questionable as to whether the consumption levels in the paper are realistic.
As a (former) scientist working in the GM field, a quick scan causes me to make the following comments:
– Populations not big enough in the groups so stats may not be meaningful.
– (Apparent) effect is so marked that it is extremely surprising that it hasn’t been reported before (always a first for everything though I suppose).
– They seem to have accurately reported that female sprague dawley rats get more tumours than male ones, but that’s irrelevant to their point. That species is very predisposed to getting tumours (hence their usefulness in such studies).
– I find it irritating that the controls aren’t in the figures so you can’t “eyeball” the difference that is supposed to be so significant.
I have a positive view of GMO in agriculture FWIW, but this feels like a stitch up, or at best confirmation bias at work.
Do it again with more animals, publish all the data.
DMS
Sorry, didn’t answer teh question posed in the title of this post from our host.
A: “Yes, but it shouldn’t”
A quick Google reveals the probable intended outcome is already happening right on schedule
“France’s Jose Bove, vice-chairman of the European Parliament’s commission for agriculture and known as an opponent of GM, called for an immediate suspension of all EU cultivation and import authorisations of GM crops.” (because of the paper)
If the effect described was so dramatic then I would have expected some epidemiological discussion. Surely we’d have seen some effect in humans by now? Where is it?
JamesV
@PaulB, it’s true lack of blinding won’t influence the assessment of mortality, but it will affect tumor measurements.
… In females, all treated groups died 2–3 times more than controls…..
Were they resuscitating them? Is there reincarnation for rats?
dearieme
“parse”: ugly, inaccurate Americanism. Why don’t you just write in American throughout? Thus “I’m not competent to parse this paper” would become
“I have not yet developed the skill set to enable transition to parsing reports of a study such as this”. You’re welcome.
I am grateful for the authors for publishing this paper, as it provides a fine case study for teaching a statistics class about poor design, analysis and reporting. I shall start using it immediately.
So they’re testing mice who have drunk water with trace amounts of RoundUp in them against mice who have eaten maize treated to cope with RoundUp but which aren’t designed to have RoundUp in their genetic make up.
So I’m assuming that they have tested that RoundUp gets incorporated into the maize. Otherwise they’re testing apples against oranges.
MrPotarto
I do enjoy the irony that when someone complains of “incorrect” American usage of a word, the best place to find the British usage is Merriam-Webster:
parse verb (parsed, parsing) tr & intr
1 grammar to analyse (a sentence) grammatically; to give the part of speech of and explain the grammatical role of (a word).
2 computational linguistics to analyse (a sentence, text, etc) into syntactic components and test its grammaticality against a given grammar.
3 computing to analyse (a string of input symbols) in terms of the computing language being used.
ETYMOLOGY: 16c: from Latin pars orationis part of speech.
Note that unlike Merriam-Webster it helpfully gives the computing usage, and thoughtfully omits the American generalization of the word’s meaning so as not to annoy dearieme.
dearieme
I don’t think that the Americans are exactly generalising: they’re just on the whole too ignorant to know that the word they seek is “construe”. What Tim’s excuse is, God knows.
Tim adds: Tim’s excuse is that he’s writing a lot for US outlets at present. And there’s a pernicious effect to that. Their spell checkers are training me…..
Mark Lynas (@mark_lynas) has been busy trashing this paper on twitter.
Science media centre has come out with this. They poke a lot of holes.
http://www.sciencemediacentre.org/pages/press_releases/12-09-19_gm_maize_rats_tumours.htm
Having had a quick read-through, there are a number of glaringly obvious problems – the paper would merit a fail at GCSE level, let alone any higher.
Aside from apparent failure to introduce blinding, which pretty much invalidates the results (such as they are) by itself, there are two major problems relating to the control group size. In the first place, it’s not big enough to say anything accurately – 10 rats isn’t enough to draw conclusions about development of tumours, since that’s something rats living to old-age (in lab settings; doesn’t normally happen in the wild) are overwhelmingly prone to. Moreover, they compare 9 sets of ‘treated’ rats to just one control group, giving them 9 chances to find one group which has higher mortality. When you account for that, their finding is actually the opposite of what they claim: seven out of eight groups showed no effect.
One has to be open for the possibility that this is a problem with RR-maize, and/or Roundup, even if organic ag. isn’t the answer and we love up-and-coming tech. The thrashing has been mostly about unavailable data-sets it seems.
The control group is too small. The work does raise concerns, but they need to redo it properly. There goes another two years dissecting rat tumours. It’s a pity they didn’t seek advice on experimental design before they started.
I dislike what seems to be data mining in the paper. “The maximum difference in males was 5
times more deaths occurring during the 17th month in the group consuming 11% GM maize, and in females 6 times greater mortality during the 21st month on the 22% GM maize diet with and without R.” When they redo this, they should say in advance what numbers they’re going to look at.
I don’t follow Dave’s remark about mortality. In Figure 1, mortality in female rates does seem to be lower in the control group compared with all doses.
Of course the work should be blinded, and it’s a concern that they don’t say it was. But I wouldn’t ignore it just because of that: there’s little potential for lack of blinding to introduce major errors.
Surely there needs to be a mechanism as well to make it convincing. But there doesn’t appear to be any / much in the way of correlation between dose and tumours (indeed a brief skim looks like it could be negative).
PaulB>
Now I re-read it, I don’t follow my remark about mortality either. It has the look of a sentence I was writing whilst switching back and forth to the source doc, and got confused. Frankly, I can’t be bothered to go back and re-read that turgid tripe – the paper, I mean, although probably the same could be said of my comment – and find out if what I think I meant has any validity.
Despite the appalling experimental design which may be deliberate (after reading the Abstract I felt that they were looking for data to support their conclusion, so I am not as confident as PaulB that they did not seek advice on experimental design) there seems to a case to answer for female rats. However, the lower incidence for rats fed the highest concentration of GMO maize is a prima facie argument for the results being unreliable. Also one should remember that a serious group of researchers found that high dosages of dihydrogen monoxide produced tumours in rats.
That said, there is, IMHO, a significant difference between GMOs which simply replicate a couple of centuries of selective breeding to combine disease-resistance with good yields and Monsanto’s moving outside that range to produce something mimicking but unrelated to natural crops that are resistant to its proprietary herbicides. IF, and it is a big IF, this experiment can be replicated under scientific conditions it will raise doubts over Monsanto’s GMOs but that would not apply to the”green revolution” high-yield rice that has been in use in India for a generation (unless someone can tell me that the Indian rat population has died out).
Yes, the paper contains a number of obvious flaws, which raise suspicions about its provenance, not least because it lacks any discussion of its own limitations, which is often a bad sign.
I’ve not much to add to what’s already been discussed save for the usual reminder that rats are not humans and so it doesn’t automatically follow that an effect seen in rats will necessarily occur in human to the same degree, or even at all.
I’d also add that an 11-22% GM maize diet strikes me as a hell of lot of corn dogs for a human, so unless you’re from a culture in which its absolute staple part of the diet, its questionable as to whether the consumption levels in the paper are realistic.
That said, a quick Google of the paper’s lead author did turn up a rather interesting interview – one which cast doubts on the paper’s assertion that there are no competing interests at play here 0 http://www.digital-athanor.com/PRISM_ESCAPE/article_usb312.html?id_article=18
As a (former) scientist working in the GM field, a quick scan causes me to make the following comments:
– Populations not big enough in the groups so stats may not be meaningful.
– (Apparent) effect is so marked that it is extremely surprising that it hasn’t been reported before (always a first for everything though I suppose).
– They seem to have accurately reported that female sprague dawley rats get more tumours than male ones, but that’s irrelevant to their point. That species is very predisposed to getting tumours (hence their usefulness in such studies).
– I find it irritating that the controls aren’t in the figures so you can’t “eyeball” the difference that is supposed to be so significant.
I have a positive view of GMO in agriculture FWIW, but this feels like a stitch up, or at best confirmation bias at work.
Do it again with more animals, publish all the data.
Sorry, didn’t answer teh question posed in the title of this post from our host.
A: “Yes, but it shouldn’t”
A quick Google reveals the probable intended outcome is already happening right on schedule
“France’s Jose Bove, vice-chairman of the European Parliament’s commission for agriculture and known as an opponent of GM, called for an immediate suspension of all EU cultivation and import authorisations of GM crops.” (because of the paper)
If the effect described was so dramatic then I would have expected some epidemiological discussion. Surely we’d have seen some effect in humans by now? Where is it?
@PaulB, it’s true lack of blinding won’t influence the assessment of mortality, but it will affect tumor measurements.
Excellent point ProgContra (#12)
No expert but I liked this senstence
… In females, all treated groups died 2–3 times more than controls…..
Were they resuscitating them? Is there reincarnation for rats?
“parse”: ugly, inaccurate Americanism. Why don’t you just write in American throughout? Thus “I’m not competent to parse this paper” would become
“I have not yet developed the skill set to enable transition to parsing reports of a study such as this”. You’re welcome.
David Spiegelhalter:
So they’re testing mice who have drunk water with trace amounts of RoundUp in them against mice who have eaten maize treated to cope with RoundUp but which aren’t designed to have RoundUp in their genetic make up.
So I’m assuming that they have tested that RoundUp gets incorporated into the maize. Otherwise they’re testing apples against oranges.
I do enjoy the irony that when someone complains of “incorrect” American usage of a word, the best place to find the British usage is Merriam-Webster:
http://www.merriam-webster.com/dictionary/parse
Potarto: see use 1. That’s the right one.
It’s the dictionary’s job also to record a wrong one if it’s common: that’s number 2 and Tim’s, as he would know if he’d had a decent schooling.
The best online dictionary is Chambers
It gives:
Note that unlike Merriam-Webster it helpfully gives the computing usage, and thoughtfully omits the American generalization of the word’s meaning so as not to annoy dearieme.
I don’t think that the Americans are exactly generalising: they’re just on the whole too ignorant to know that the word they seek is “construe”. What Tim’s excuse is, God knows.
Tim adds: Tim’s excuse is that he’s writing a lot for US outlets at present. And there’s a pernicious effect to that. Their spell checkers are training me…..
Whoa that link shud have warning. Gross pics!!