“Loman et al reflects the past, not the present” says Life Technologies/ Ion Torrent in a slide set accompanying a response, published yesterday, to the recent paper by Nick Loman et al, “Performance comparison of benchtop high-throughput sequencing platforms” (Loman et al, 2012). See also my coverage of this paper in my previous blog post.
It is a critique I have read and heard more often: the data used for the analyses in the Loman et al paper is already old, as the technologies have now improved. This is of course true, particularly so for Ion Torrent. However true, it is not a fair critique. Researchers, and Nick Loman and yours truly are not an exception, are bound by the ‘publish or perish’ mantra. We are dependent on publishing peer-reviewed articles for obtaining grants, establishing our reputation, and for getting our next job. Peer review takes time: “Right now the time lag between finishing a paper, and the relevant worldwide research community seeing it, is between 6 months and 2 years.” (source). Nick’s paper was ‘Received 19 December 2011″, “Accepted 30 March 2012” and finally “Published online 22 April 2012”. This is actually quite fast, taking into consideration the authors developed numerous new tools for the analyses (see the github repository accompanying the paper).
Sure, we can use a blog to circumvent the time delay, and publish a finding immediately, something Nick is actively doing (as am I through this blog). But, sometimes we need to go the peer-review route, for the reasons explained above.
It is therefore unavoidable that articles, like the one from Nick Loman, contain ‘old’ data. Heck, there are still lot’s of papers coming out based on 454 GS FLX and Illumina GA II(x) data. In addition, there must be many groups analysing IonTorrent/454/Illumina data they obtained using the same ‘generation’ of kits as were used for the Loman et al paper. These people will absolutely want to know about the different error types and accuracy levels. At the very least, the data presented in the paper give an overview of the relative performance at the time of studying, which might reflect on today’s performance.
I appreciate Ion Torrent people jumping on the occasion, requesting a sample from the strain used for the paper, sequencing with their latest chemistry, and redoing some of the analyses, all in less than three weeks. The results look promising, although they need to be quality checked by the community (bloggers like us, I guess 🙂 ). But don’t blame the messenger for taking the established route. Let’s rather congratulate Nick Loman et al for a job well done and a well deserved publication in Nature Biotechnology!
Agree? Disagree? Feel free to drop me a comment below!