Why don't I see publications criticising other publications?
up vote
44
down vote
favorite
This is about technology research specifically.
Often when reading a publication I'll be thinking to myself: "Yeah, you've got that thing, but what about this and that limitation you brush off and this and that assumption which seems pretty poorly justified?" After starting my PhD I got pretty shocked just how common that seems to be.
The only critiques of other publications I ever see are either in literature review sections of papers suggesting alternative methods, or mentioned in surveys. In either case those tend to be not very in-depth.
Why is this the case? It seems to me like something that would be pretty damaging to research in general, as then you see a lot of papers with built-in assumptions and limitations propagated by their predecessors.
publications technology
|
show 5 more comments
up vote
44
down vote
favorite
This is about technology research specifically.
Often when reading a publication I'll be thinking to myself: "Yeah, you've got that thing, but what about this and that limitation you brush off and this and that assumption which seems pretty poorly justified?" After starting my PhD I got pretty shocked just how common that seems to be.
The only critiques of other publications I ever see are either in literature review sections of papers suggesting alternative methods, or mentioned in surveys. In either case those tend to be not very in-depth.
Why is this the case? It seems to me like something that would be pretty damaging to research in general, as then you see a lot of papers with built-in assumptions and limitations propagated by their predecessors.
publications technology
42
2-3 decades ago a professor use to publish few research articles in his entire career now I have seen people publishing 10s of articles during PhD. The quality of published article is getting low day by day. There is are multiple reasons but most important reason in my opinion is Publish or perish culture imposed on academia.
– MBK
2 days ago
8
@Trilarion not a very good one imo - there's all sorts of reasons one paper could have fewer citations than another, and I think poor quality is reasonably far down the list
– llama
2 days ago
2
Maybe that's field-dependent, but I see that a lot in political science. A major way to bolster the "relevance" of a publication is to situate it within a debate. Bonus points if you criticize "conventional wisdom". So I'm not.sure about the premise of this question.
– henning
2 days ago
2
Actually there is a lot of published criticism on papers of different fields. See for instance PubPeer.
– Scientist
yesterday
1
Having written such an article myself: they do exist, but getting the balance between constructive criticism and pure negativity right is tricky. Articles that cite our paper also often uncritically cite the paper we are refuting.
– Konrad Rudolph
yesterday
|
show 5 more comments
up vote
44
down vote
favorite
up vote
44
down vote
favorite
This is about technology research specifically.
Often when reading a publication I'll be thinking to myself: "Yeah, you've got that thing, but what about this and that limitation you brush off and this and that assumption which seems pretty poorly justified?" After starting my PhD I got pretty shocked just how common that seems to be.
The only critiques of other publications I ever see are either in literature review sections of papers suggesting alternative methods, or mentioned in surveys. In either case those tend to be not very in-depth.
Why is this the case? It seems to me like something that would be pretty damaging to research in general, as then you see a lot of papers with built-in assumptions and limitations propagated by their predecessors.
publications technology
This is about technology research specifically.
Often when reading a publication I'll be thinking to myself: "Yeah, you've got that thing, but what about this and that limitation you brush off and this and that assumption which seems pretty poorly justified?" After starting my PhD I got pretty shocked just how common that seems to be.
The only critiques of other publications I ever see are either in literature review sections of papers suggesting alternative methods, or mentioned in surveys. In either case those tend to be not very in-depth.
Why is this the case? It seems to me like something that would be pretty damaging to research in general, as then you see a lot of papers with built-in assumptions and limitations propagated by their predecessors.
publications technology
publications technology
asked 2 days ago
KubaFYI
39435
39435
42
2-3 decades ago a professor use to publish few research articles in his entire career now I have seen people publishing 10s of articles during PhD. The quality of published article is getting low day by day. There is are multiple reasons but most important reason in my opinion is Publish or perish culture imposed on academia.
– MBK
2 days ago
8
@Trilarion not a very good one imo - there's all sorts of reasons one paper could have fewer citations than another, and I think poor quality is reasonably far down the list
– llama
2 days ago
2
Maybe that's field-dependent, but I see that a lot in political science. A major way to bolster the "relevance" of a publication is to situate it within a debate. Bonus points if you criticize "conventional wisdom". So I'm not.sure about the premise of this question.
– henning
2 days ago
2
Actually there is a lot of published criticism on papers of different fields. See for instance PubPeer.
– Scientist
yesterday
1
Having written such an article myself: they do exist, but getting the balance between constructive criticism and pure negativity right is tricky. Articles that cite our paper also often uncritically cite the paper we are refuting.
– Konrad Rudolph
yesterday
|
show 5 more comments
42
2-3 decades ago a professor use to publish few research articles in his entire career now I have seen people publishing 10s of articles during PhD. The quality of published article is getting low day by day. There is are multiple reasons but most important reason in my opinion is Publish or perish culture imposed on academia.
– MBK
2 days ago
8
@Trilarion not a very good one imo - there's all sorts of reasons one paper could have fewer citations than another, and I think poor quality is reasonably far down the list
– llama
2 days ago
2
Maybe that's field-dependent, but I see that a lot in political science. A major way to bolster the "relevance" of a publication is to situate it within a debate. Bonus points if you criticize "conventional wisdom". So I'm not.sure about the premise of this question.
– henning
2 days ago
2
Actually there is a lot of published criticism on papers of different fields. See for instance PubPeer.
– Scientist
yesterday
1
Having written such an article myself: they do exist, but getting the balance between constructive criticism and pure negativity right is tricky. Articles that cite our paper also often uncritically cite the paper we are refuting.
– Konrad Rudolph
yesterday
42
42
2-3 decades ago a professor use to publish few research articles in his entire career now I have seen people publishing 10s of articles during PhD. The quality of published article is getting low day by day. There is are multiple reasons but most important reason in my opinion is Publish or perish culture imposed on academia.
– MBK
2 days ago
2-3 decades ago a professor use to publish few research articles in his entire career now I have seen people publishing 10s of articles during PhD. The quality of published article is getting low day by day. There is are multiple reasons but most important reason in my opinion is Publish or perish culture imposed on academia.
– MBK
2 days ago
8
8
@Trilarion not a very good one imo - there's all sorts of reasons one paper could have fewer citations than another, and I think poor quality is reasonably far down the list
– llama
2 days ago
@Trilarion not a very good one imo - there's all sorts of reasons one paper could have fewer citations than another, and I think poor quality is reasonably far down the list
– llama
2 days ago
2
2
Maybe that's field-dependent, but I see that a lot in political science. A major way to bolster the "relevance" of a publication is to situate it within a debate. Bonus points if you criticize "conventional wisdom". So I'm not.sure about the premise of this question.
– henning
2 days ago
Maybe that's field-dependent, but I see that a lot in political science. A major way to bolster the "relevance" of a publication is to situate it within a debate. Bonus points if you criticize "conventional wisdom". So I'm not.sure about the premise of this question.
– henning
2 days ago
2
2
Actually there is a lot of published criticism on papers of different fields. See for instance PubPeer.
– Scientist
yesterday
Actually there is a lot of published criticism on papers of different fields. See for instance PubPeer.
– Scientist
yesterday
1
1
Having written such an article myself: they do exist, but getting the balance between constructive criticism and pure negativity right is tricky. Articles that cite our paper also often uncritically cite the paper we are refuting.
– Konrad Rudolph
yesterday
Having written such an article myself: they do exist, but getting the balance between constructive criticism and pure negativity right is tricky. Articles that cite our paper also often uncritically cite the paper we are refuting.
– Konrad Rudolph
yesterday
|
show 5 more comments
10 Answers
10
active
oldest
votes
up vote
42
down vote
Such criticisms are common in journals like Nature and PNAS. They're often not called "papers" -- PNAS calls them "Letters", Science calls them "Technical Comments", and Nature calls them "Brief Communications Arising" -- but they do get a significant degree of pre-publication scrutiny and often are peer reviewed. Some examples from the past few weeks:
Is “choline and geranate” an ionic liquid or deep eutectic solvent system? (PNAS)
Tagging the musical beat: Neural entrainment or event-related potentials? (PNAS)
Comment on “Predicting reaction performance in C–N cross-coupling using machine learning” (Science)
Comment on “The earliest modern humans outside Africa” (Science)
Assumptions for emergent constraints (Nature)
Emergent constraints on climate sensitivity (Nature)
Climate constraint reflects forced signal (Nature)
Other journals also have systems for responses, though most are not as organized about it.
So the premise is not quite right -- criticisms of high-profile papers are not unusual -- but it's true that the vast majority of papers don't get formal critiques like this -- even though the vast majority of papers probably do have something that could be criticized.
That's part of the reason. Very few papers are completely above reproach (except for my own, of course). If I was to send a critical letter about every issue I see in every paper I read, I'd spend my time doing nothing else, and the journals would be filled with my letters to the editor with no room for new articles.
And we're all adults here. It's expected that scientists read papers critically; that's part of our job. That means we all find things to criticize. Finding a problem in a published paper isn't a shocking, scandalous thing. It's like a toll collector breaking a $20 bill, a minor technical part of your job. It would be condescending of me to assume that I'm the only person who noticed the issues in question, and that it's up to me to save my colleagues from their stupidity and ignorance.
Finally, when I do respond to publications that need criticizing, I rarely phrase my responses that way. For example, several years ago I read an interesting paper in my field that, I thought, made a significant error in its assumptions. To show that, I repeated some of their experiments, adding in the missing controls, and then extended the work to show where the corrected observations led. I didn't publish as a criticism of the previous work (which was from researchers who I know and admire and who have done a lot of great stuff). I published it as a paper that can stand on its own, noting the previous work in passing but not making a big deal of the mistake. Hopefully, if I ever make a mistake (unlikely!) my colleagues will correct me in the same way. Science is hard, and it shouldn't be like a hockey goalie, where every time you make a mistake a buzzer goes off and thirty thousand people yell at you.
8
This answer is quite optimistic in some regards and seems to assume that spotting problems while reading publications is easy as well as redoing experiments to verify scientific claims one way or the other isn't a waste of time. However, in the spirit of the answer I will refrain from writing my own answer criticizing this answer. :)
– Trilarion
2 days ago
1
I don't know if spotting problems is easy per se, but it's such a routine part of the job that novices are, or should be, intensively trained in it. The fact that academia SE and similar sites constantly see questions like this one, from new PhD students, suggests that even people fairly new to the field are able to spot problems quickly.
– iayork
2 days ago
I think this is true for your immediate field of expertise, but if you move a bit further away it might be more complicated. What I think could help would be something similar to the customer reviews on Amazon product pages, in this case reader reviews of scientific publications. It would save me a lot of time if I could see what problems other readers have spotted without me needing to check every publication on my own every time. Problem spotting is much easier if we would all do it together instead of everyone for himself/herself.
– Trilarion
2 days ago
There is also a category "Analysis" in Nature: nature.com/nature/articles?type=analysis
– Norbert
yesterday
add a comment |
up vote
35
down vote
Many do, just in a polite way. Most of numerical modelling seems to be a variation of nicely stating "previous folks did things wrong and/or inefficient, we improve upon that; or at least offer another approach despite not showing their failings." There, criticism/failing of other approaches is often shown throughout the article.
Additionally, you are starting your PhD.
- What you feel might be a serious omission regarding method applicability could be a well-known limitation of the method everyone in the community is aware of. In this case, your "you are propagating BS" is a pointless and wrong rant - they know their methods better than you. Say everyone might already know X will offer about 1% higher error, but 100x easier simulation/experiment. If you suspect this might be the case, rather write a polite mail asking for clarification and whether the obtained results are still valid or how the approach could be improved etc etc.
- On the other hand, everyone could be assuming something that is actually incorrect for the problem, and you know it. Here, bluntly stating "you are propagating BS" isn't technically wrong, but you won't be getting any new friends with that. Rather write a nice article showing how and why everyone is approaching the problem wrong way and how you solve their issues - you would be instantly a highly regarded member of the community.
But yeah, a lot of fairly shaky research is taken as true and this leads to severe reproducibility problem all over fields. There are journals for publishing reproductions of other's work, but most prefer doing something original. Can't blame them: Invent X and have name immortalized vs "yeah, X's results check out".
New contributor
15
If you don't see criticism in papers, its because you are not recognizing it when you see it. Its there. Its just between the lines.
– Ian Sudbery
2 days ago
1
Generally good answer. Although repeating the premise that "everyone already knows X" alongside the premise that the OP doesn't is clearly contradictory.
– Lightness Races in Orbit
2 days ago
5
@LightnessRacesinOrbit "Everyone" means "most experts", not literally everyone.
– David Richerby
2 days ago
1
@DavidRicherby The word "everyone" means "everyone" - say what you mean!
– Lightness Races in Orbit
yesterday
6
@LightnessRacesinOrbit if you insist that the word "everyone" must mean literally every single person who is currently alive, the word becomes almost useless. Language is not mathematics. We don't communicate by concatenating dictionary definitions.
– David Richerby
yesterday
add a comment |
up vote
15
down vote
Politics. Criticising peers directly leads to lack of political support.
This is because science is increasingly based on connections and short-term appraisal, and this is greatly detrimental to the advancement of hardcore science, and intellectualism in general.
We are social animals. This leads to a natural tendency of taking others' actions personally and "return favours". Also people are very wary of others' opinions and public image, and as a result are naturally sensitive to negative criticism. Usually, when you hurt the feelings of someone in public, you've made yourself an enemy.
I firmly believe in criticism and frankness as the shortest path to improvement. That has put me in delicate situations many times. In my early papers I was eager to lift obvious flaws in whatever specific topic being discussed, as to attempt to contribute. I have the habit of signing my peer reviews, and I am rather picky as a reviewer. This eventually resulted in being avoided in the citation and collaboration lists of relevant peers. Nowadays I am internally struggling on how to discuss faulty literature and how to approach clearly flawed logic.
I have been told that in order to say what you want you must reach a certain level of authority. However reaching that point largely depends on connections and good relationships. There are no Isaac Newtons today.
3
This eventually resulted in being avoided in the citation... lists of relevant peers. — I'm sorry you have unethical peers.
– JeffE
yesterday
This answer makes me sad.
– goblin
13 hours ago
add a comment |
up vote
9
down vote
Have you ever encountered a "Letter-to-editor" article?
Check this, for instance, an article on the Turkish Journal of Urology about writing a letter to the editor:
How to write an editorial letter?
It is a kind of a "short-communication", and generally used to criticize previously published articles. If this is uncommon or absent in any of research subject, it is more of an issue of the researchers in that field.
2
The key question there is I guess if anyone will ever see such a critique. I mean, I encounter most of the papers I read either by a search engine or by being linked to it by someone. Does anyone still read entire journals, introductions and all?
– KubaFYI
2 days ago
4
@KubaFYI These responses/letters act just like other journal articles - they get indexed and have their own reference. They can therefore crop up in a Google Scholar search or similar. In addition, journals websites often prominently link to the response from the original article's page. For example, see this article: pnas.org/content/115/32/8221
– user2390246
2 days ago
2
@KubaFYI: Any time I see a paper that seems wrong to me, the first thing I do is to do a citation search for other papers that cite that paper. If someone has published a letter or comment saying it's wrong, it will pop up, because they will cite the paper they're criticizing.
– Ben Crowell
2 days ago
add a comment |
up vote
9
down vote
I am going to convert my comment into answer. Most of the researcher are aware of this but there is a pressure in majority of academia around the globe and that is Publish or Perish.
On average Writing a good article takes 1-2 years but to save tenure (in my country) an AP has to publish 3 SCI articles per year. With such kind of pressure a researcher can not write a quality research paper. Moreover, strong critique require time to analyse the findings of other researchers, while the race of publish don’t allow to spend much time on that part of article, which I believe is one of the most important part of article.
Yuval Noah Harari author of famous book Sapiens: A Brief History of Humankind, once he said in his interview that he start thinking out of box when he promoted to Associate Professor position, because in his university there is no requirement of publication once you get Associate Professor position. That's why he wrote this book within 2 years of time.
1
@scaaahu Critiques require time to analyse the findings of other researchers, while due to race of publish mostly researcher avoid it.
– MBK
2 days ago
4
We really need to all sit down and collectively agree that going on like that doesn't make any sense, if we are to be true to the explicit goal of academia of advancing human understanding of the world. Now what is just a metric had become the implicit goal.
– KubaFYI
2 days ago
2
I'm pretty sure that all (or many) academics do agree with this. Unfortunately, while the explicit goal of academics might be advancing human knowledge, the academy is not run by academics. Mostly funding bodies, governments and university administrators to make the decisions..
– Ian Sudbery
2 days ago
add a comment |
up vote
6
down vote
Simply speaking: The better long term strategy for staying in academia is to be the nice guy that plays along.
Academia is full of systemic corruption because jobs in a specific field of research are very limited/competitive due to the pyramid nature of human resources (number of students > number of PhDs > number of professors). Academia is structured around the selection of outstanding and remarkable individuals. However, history has told us that most metrics tend to be overshadowed by character traits in the long run. Especially if you think about the fact that most professors are not doing much research themselves but are rather responslible for the management of workgroups and projects. Hence, the nice, non-critical guy has a structural advantage in the system. Letters of recommendation, which tend to be more positive for a friendly person compared to "troublemakers", are also a direct indication of this.
The system is the people and the people select their system. People don't like criticism. So the system/people deselected criticism.
2
most professors are not doing much research themselves — ...in some fields. This is certainly not true in all fields.
– JeffE
yesterday
add a comment |
up vote
3
down vote
Other answers discuss good reasons (besides the fair point that answers do exist): e.g. good criticism takes time, also the authors may know the specifics better than you.
Another reason why criticism is usually very tuned down: the reviewers for your article are selected based on familiarity with the field, which means they possibly have a (favorite) method / approach of their own. So new articles need to strike a balance between saying "the existing methods are insufficient" (which is why we are publishing our novel one) while at the same time not being too harsh (which will offend the reviewers).
New contributor
add a comment |
up vote
1
down vote
- Criticism is published, as pointed out by iayork. It may also appear in less formal places like blogs. So one reason you don't see it is that you aren't looking in the right places.
- The best place for criticism is in the peer review process. (That is, peer review should catch bad work before it is published, rendering subsequent criticism unnecessary. However, bad work does sometimes get published, so criticism of published work is appropriate then.) Reviews are not usually published, but some venues do publish reviews. E.g., ICLR and NeurIPS reviews are available with the papers (for ICLR, even the rejected papers have reviews published).
- Constructive criticism is better than simply trashing other people's work. Attacks can backfire -- they make the attacker look unprofessional, and things can degenerate into a back-and-forth flame war. Constructive criticism appears in the form of follow-up work that fixes the shortcomings of prior work. This is more polite and less obvious, so you may not notice it.
- Even when bad work gets published, it may simply be best to ignore it. Engaging in criticism is a messy business and some of the mud may stick to the critic. Criticism usually only appears when the work in question is getting a lot of attention.
2
"The right place for criticism is in the peer review process." -- is this to say published work ought not be criticised? I don't see why.
– Scientist
2 days ago
1
@Scientist Of course published work should be criticized! However, peer review should be the first line of defense. Most of the time when published work is being criticized, it's because it should not have been published in the first place. I have edited to emphasize this.
– Thomas
2 days ago
2
"it's because it should not have been published in the first place" You are assuming the criticism is because of poor quality or (obviously) wrong conclusions. In this case yes, it shouldn't be published. But there are a lot of cases where different views exist, different models, different explanations or similar. Or both the authors and reviewers missed something. In that case review is the worst place to put your critisim (although some try) since it prohibits the exchange of ideas and also doesn't get published.
– DSVA
yesterday
2
So what would you call it then? Here's a paper clearly critisicing the idea of "secondary orbital interactions" pubs.acs.org/doi/abs/10.1021/ar0000152 and here's a response ncbi.nlm.nih.gov/pubmed/17109435 clearly criticising the first paper. "Criticism is the practice of judging the merits and faults of something."
– DSVA
yesterday
1
I’m sure all published literature is open for criticism, and must be criticized. Science is built on correction of flawed assumptions. We’re always off the mark and missing important factors. It’s a never ending discussion.
– Scientist
yesterday
|
show 3 more comments
up vote
1
down vote
After the second year of Ph.D. study, I started to learn extensively on how to write highly regarded publications. The first thing that was hammered in me is how to describe the assumptions and limitations of the given research, and give a framework or scope of the given research. My first publication described many of these limitations and scope in detail, but my second and third publications did not because I already established these issues in my first one. So, it is paramount that you check ALL the citations, especially the ones by the same author. These usually give you the 'background' of the research. In addition, be familiar with 'well known' assumptions, and practices as well. They are accepted for a reason.
Second, I have seen an article (peer-reviewed in high-ranking journal) that outright attacked another earlier published work by another author. Even though the attacking article may be correct, it did not exactly contribute anything to the field. On other hand, if you developed a novel methodology that addresses the issue that was presented in that offending publication, you can say, "here's my work on this, and I did this." and then cite that paper, and then say something along like, 'improves the results because their approach did not address this particular issue'.
Third,if you burn bridges and piss off the wrong people by attacking them outright, you will not get very far. It is a fact of life. Learn how to navigate the world with proper conduct and delicate touch (people ARE sensitive), and stay true to yourself.
add a comment |
up vote
1
down vote
Some of the available answers allude to the following principle, but none say it directly.
Identifying idiosyncratic flaws of a single article, if published alone, makes your article a satellite of that primary article; the importance of your article cannot possibly exceed the importance of the primary article, but usually the satellite weighs less.
The best way to reach publication-worthiness (in a journal of the same caliber like the primary publication), you should ideally get some results of your own, studying the same problem. That buys you a ticket under which you can not only showcase your own work, but you can also compare it to the state of the field (i.e., the flawed article, and any other relevant prior work that you are aware of) in as much detail as you find intellectually stimulating to a "generic" reader of the journal where you publish this.
The point of comparing your work to previous work is to make your contribution shine in its relevance and methodological superiority, rather than to punish your predecessor or to save your colleagues from falling into believing obviously flawed results. And that is an entirely non-political reason to approach the comparison with utmost indifference and politeness, in a way where many readers (who never encountered the actual earlier article) will not even realize the references to it to have motivated your article. The flawed article isn't enormously important, right? I'd even argue that you should be primarily comparing your new work to the best available results of others, unless certain flaws are propagating over most prior work and your own work stands out (in your eyes) by being more sophisticated and free of them.
The reason why some high profile journals pay more attention to "letters to the editor" than solid "average" journals is that if the primary article is widely seen as a breakthrough, its satellites might still be publication worthy if the critique is deep or if it adds a relevant multi-disciplinary angle, and if their points are really good.
add a comment |
10 Answers
10
active
oldest
votes
10 Answers
10
active
oldest
votes
active
oldest
votes
active
oldest
votes
up vote
42
down vote
Such criticisms are common in journals like Nature and PNAS. They're often not called "papers" -- PNAS calls them "Letters", Science calls them "Technical Comments", and Nature calls them "Brief Communications Arising" -- but they do get a significant degree of pre-publication scrutiny and often are peer reviewed. Some examples from the past few weeks:
Is “choline and geranate” an ionic liquid or deep eutectic solvent system? (PNAS)
Tagging the musical beat: Neural entrainment or event-related potentials? (PNAS)
Comment on “Predicting reaction performance in C–N cross-coupling using machine learning” (Science)
Comment on “The earliest modern humans outside Africa” (Science)
Assumptions for emergent constraints (Nature)
Emergent constraints on climate sensitivity (Nature)
Climate constraint reflects forced signal (Nature)
Other journals also have systems for responses, though most are not as organized about it.
So the premise is not quite right -- criticisms of high-profile papers are not unusual -- but it's true that the vast majority of papers don't get formal critiques like this -- even though the vast majority of papers probably do have something that could be criticized.
That's part of the reason. Very few papers are completely above reproach (except for my own, of course). If I was to send a critical letter about every issue I see in every paper I read, I'd spend my time doing nothing else, and the journals would be filled with my letters to the editor with no room for new articles.
And we're all adults here. It's expected that scientists read papers critically; that's part of our job. That means we all find things to criticize. Finding a problem in a published paper isn't a shocking, scandalous thing. It's like a toll collector breaking a $20 bill, a minor technical part of your job. It would be condescending of me to assume that I'm the only person who noticed the issues in question, and that it's up to me to save my colleagues from their stupidity and ignorance.
Finally, when I do respond to publications that need criticizing, I rarely phrase my responses that way. For example, several years ago I read an interesting paper in my field that, I thought, made a significant error in its assumptions. To show that, I repeated some of their experiments, adding in the missing controls, and then extended the work to show where the corrected observations led. I didn't publish as a criticism of the previous work (which was from researchers who I know and admire and who have done a lot of great stuff). I published it as a paper that can stand on its own, noting the previous work in passing but not making a big deal of the mistake. Hopefully, if I ever make a mistake (unlikely!) my colleagues will correct me in the same way. Science is hard, and it shouldn't be like a hockey goalie, where every time you make a mistake a buzzer goes off and thirty thousand people yell at you.
8
This answer is quite optimistic in some regards and seems to assume that spotting problems while reading publications is easy as well as redoing experiments to verify scientific claims one way or the other isn't a waste of time. However, in the spirit of the answer I will refrain from writing my own answer criticizing this answer. :)
– Trilarion
2 days ago
1
I don't know if spotting problems is easy per se, but it's such a routine part of the job that novices are, or should be, intensively trained in it. The fact that academia SE and similar sites constantly see questions like this one, from new PhD students, suggests that even people fairly new to the field are able to spot problems quickly.
– iayork
2 days ago
I think this is true for your immediate field of expertise, but if you move a bit further away it might be more complicated. What I think could help would be something similar to the customer reviews on Amazon product pages, in this case reader reviews of scientific publications. It would save me a lot of time if I could see what problems other readers have spotted without me needing to check every publication on my own every time. Problem spotting is much easier if we would all do it together instead of everyone for himself/herself.
– Trilarion
2 days ago
There is also a category "Analysis" in Nature: nature.com/nature/articles?type=analysis
– Norbert
yesterday
add a comment |
up vote
42
down vote
Such criticisms are common in journals like Nature and PNAS. They're often not called "papers" -- PNAS calls them "Letters", Science calls them "Technical Comments", and Nature calls them "Brief Communications Arising" -- but they do get a significant degree of pre-publication scrutiny and often are peer reviewed. Some examples from the past few weeks:
Is “choline and geranate” an ionic liquid or deep eutectic solvent system? (PNAS)
Tagging the musical beat: Neural entrainment or event-related potentials? (PNAS)
Comment on “Predicting reaction performance in C–N cross-coupling using machine learning” (Science)
Comment on “The earliest modern humans outside Africa” (Science)
Assumptions for emergent constraints (Nature)
Emergent constraints on climate sensitivity (Nature)
Climate constraint reflects forced signal (Nature)
Other journals also have systems for responses, though most are not as organized about it.
So the premise is not quite right -- criticisms of high-profile papers are not unusual -- but it's true that the vast majority of papers don't get formal critiques like this -- even though the vast majority of papers probably do have something that could be criticized.
That's part of the reason. Very few papers are completely above reproach (except for my own, of course). If I was to send a critical letter about every issue I see in every paper I read, I'd spend my time doing nothing else, and the journals would be filled with my letters to the editor with no room for new articles.
And we're all adults here. It's expected that scientists read papers critically; that's part of our job. That means we all find things to criticize. Finding a problem in a published paper isn't a shocking, scandalous thing. It's like a toll collector breaking a $20 bill, a minor technical part of your job. It would be condescending of me to assume that I'm the only person who noticed the issues in question, and that it's up to me to save my colleagues from their stupidity and ignorance.
Finally, when I do respond to publications that need criticizing, I rarely phrase my responses that way. For example, several years ago I read an interesting paper in my field that, I thought, made a significant error in its assumptions. To show that, I repeated some of their experiments, adding in the missing controls, and then extended the work to show where the corrected observations led. I didn't publish as a criticism of the previous work (which was from researchers who I know and admire and who have done a lot of great stuff). I published it as a paper that can stand on its own, noting the previous work in passing but not making a big deal of the mistake. Hopefully, if I ever make a mistake (unlikely!) my colleagues will correct me in the same way. Science is hard, and it shouldn't be like a hockey goalie, where every time you make a mistake a buzzer goes off and thirty thousand people yell at you.
8
This answer is quite optimistic in some regards and seems to assume that spotting problems while reading publications is easy as well as redoing experiments to verify scientific claims one way or the other isn't a waste of time. However, in the spirit of the answer I will refrain from writing my own answer criticizing this answer. :)
– Trilarion
2 days ago
1
I don't know if spotting problems is easy per se, but it's such a routine part of the job that novices are, or should be, intensively trained in it. The fact that academia SE and similar sites constantly see questions like this one, from new PhD students, suggests that even people fairly new to the field are able to spot problems quickly.
– iayork
2 days ago
I think this is true for your immediate field of expertise, but if you move a bit further away it might be more complicated. What I think could help would be something similar to the customer reviews on Amazon product pages, in this case reader reviews of scientific publications. It would save me a lot of time if I could see what problems other readers have spotted without me needing to check every publication on my own every time. Problem spotting is much easier if we would all do it together instead of everyone for himself/herself.
– Trilarion
2 days ago
There is also a category "Analysis" in Nature: nature.com/nature/articles?type=analysis
– Norbert
yesterday
add a comment |
up vote
42
down vote
up vote
42
down vote
Such criticisms are common in journals like Nature and PNAS. They're often not called "papers" -- PNAS calls them "Letters", Science calls them "Technical Comments", and Nature calls them "Brief Communications Arising" -- but they do get a significant degree of pre-publication scrutiny and often are peer reviewed. Some examples from the past few weeks:
Is “choline and geranate” an ionic liquid or deep eutectic solvent system? (PNAS)
Tagging the musical beat: Neural entrainment or event-related potentials? (PNAS)
Comment on “Predicting reaction performance in C–N cross-coupling using machine learning” (Science)
Comment on “The earliest modern humans outside Africa” (Science)
Assumptions for emergent constraints (Nature)
Emergent constraints on climate sensitivity (Nature)
Climate constraint reflects forced signal (Nature)
Other journals also have systems for responses, though most are not as organized about it.
So the premise is not quite right -- criticisms of high-profile papers are not unusual -- but it's true that the vast majority of papers don't get formal critiques like this -- even though the vast majority of papers probably do have something that could be criticized.
That's part of the reason. Very few papers are completely above reproach (except for my own, of course). If I was to send a critical letter about every issue I see in every paper I read, I'd spend my time doing nothing else, and the journals would be filled with my letters to the editor with no room for new articles.
And we're all adults here. It's expected that scientists read papers critically; that's part of our job. That means we all find things to criticize. Finding a problem in a published paper isn't a shocking, scandalous thing. It's like a toll collector breaking a $20 bill, a minor technical part of your job. It would be condescending of me to assume that I'm the only person who noticed the issues in question, and that it's up to me to save my colleagues from their stupidity and ignorance.
Finally, when I do respond to publications that need criticizing, I rarely phrase my responses that way. For example, several years ago I read an interesting paper in my field that, I thought, made a significant error in its assumptions. To show that, I repeated some of their experiments, adding in the missing controls, and then extended the work to show where the corrected observations led. I didn't publish as a criticism of the previous work (which was from researchers who I know and admire and who have done a lot of great stuff). I published it as a paper that can stand on its own, noting the previous work in passing but not making a big deal of the mistake. Hopefully, if I ever make a mistake (unlikely!) my colleagues will correct me in the same way. Science is hard, and it shouldn't be like a hockey goalie, where every time you make a mistake a buzzer goes off and thirty thousand people yell at you.
Such criticisms are common in journals like Nature and PNAS. They're often not called "papers" -- PNAS calls them "Letters", Science calls them "Technical Comments", and Nature calls them "Brief Communications Arising" -- but they do get a significant degree of pre-publication scrutiny and often are peer reviewed. Some examples from the past few weeks:
Is “choline and geranate” an ionic liquid or deep eutectic solvent system? (PNAS)
Tagging the musical beat: Neural entrainment or event-related potentials? (PNAS)
Comment on “Predicting reaction performance in C–N cross-coupling using machine learning” (Science)
Comment on “The earliest modern humans outside Africa” (Science)
Assumptions for emergent constraints (Nature)
Emergent constraints on climate sensitivity (Nature)
Climate constraint reflects forced signal (Nature)
Other journals also have systems for responses, though most are not as organized about it.
So the premise is not quite right -- criticisms of high-profile papers are not unusual -- but it's true that the vast majority of papers don't get formal critiques like this -- even though the vast majority of papers probably do have something that could be criticized.
That's part of the reason. Very few papers are completely above reproach (except for my own, of course). If I was to send a critical letter about every issue I see in every paper I read, I'd spend my time doing nothing else, and the journals would be filled with my letters to the editor with no room for new articles.
And we're all adults here. It's expected that scientists read papers critically; that's part of our job. That means we all find things to criticize. Finding a problem in a published paper isn't a shocking, scandalous thing. It's like a toll collector breaking a $20 bill, a minor technical part of your job. It would be condescending of me to assume that I'm the only person who noticed the issues in question, and that it's up to me to save my colleagues from their stupidity and ignorance.
Finally, when I do respond to publications that need criticizing, I rarely phrase my responses that way. For example, several years ago I read an interesting paper in my field that, I thought, made a significant error in its assumptions. To show that, I repeated some of their experiments, adding in the missing controls, and then extended the work to show where the corrected observations led. I didn't publish as a criticism of the previous work (which was from researchers who I know and admire and who have done a lot of great stuff). I published it as a paper that can stand on its own, noting the previous work in passing but not making a big deal of the mistake. Hopefully, if I ever make a mistake (unlikely!) my colleagues will correct me in the same way. Science is hard, and it shouldn't be like a hockey goalie, where every time you make a mistake a buzzer goes off and thirty thousand people yell at you.
edited 2 days ago
answered 2 days ago
iayork
10.6k32941
10.6k32941
8
This answer is quite optimistic in some regards and seems to assume that spotting problems while reading publications is easy as well as redoing experiments to verify scientific claims one way or the other isn't a waste of time. However, in the spirit of the answer I will refrain from writing my own answer criticizing this answer. :)
– Trilarion
2 days ago
1
I don't know if spotting problems is easy per se, but it's such a routine part of the job that novices are, or should be, intensively trained in it. The fact that academia SE and similar sites constantly see questions like this one, from new PhD students, suggests that even people fairly new to the field are able to spot problems quickly.
– iayork
2 days ago
I think this is true for your immediate field of expertise, but if you move a bit further away it might be more complicated. What I think could help would be something similar to the customer reviews on Amazon product pages, in this case reader reviews of scientific publications. It would save me a lot of time if I could see what problems other readers have spotted without me needing to check every publication on my own every time. Problem spotting is much easier if we would all do it together instead of everyone for himself/herself.
– Trilarion
2 days ago
There is also a category "Analysis" in Nature: nature.com/nature/articles?type=analysis
– Norbert
yesterday
add a comment |
8
This answer is quite optimistic in some regards and seems to assume that spotting problems while reading publications is easy as well as redoing experiments to verify scientific claims one way or the other isn't a waste of time. However, in the spirit of the answer I will refrain from writing my own answer criticizing this answer. :)
– Trilarion
2 days ago
1
I don't know if spotting problems is easy per se, but it's such a routine part of the job that novices are, or should be, intensively trained in it. The fact that academia SE and similar sites constantly see questions like this one, from new PhD students, suggests that even people fairly new to the field are able to spot problems quickly.
– iayork
2 days ago
I think this is true for your immediate field of expertise, but if you move a bit further away it might be more complicated. What I think could help would be something similar to the customer reviews on Amazon product pages, in this case reader reviews of scientific publications. It would save me a lot of time if I could see what problems other readers have spotted without me needing to check every publication on my own every time. Problem spotting is much easier if we would all do it together instead of everyone for himself/herself.
– Trilarion
2 days ago
There is also a category "Analysis" in Nature: nature.com/nature/articles?type=analysis
– Norbert
yesterday
8
8
This answer is quite optimistic in some regards and seems to assume that spotting problems while reading publications is easy as well as redoing experiments to verify scientific claims one way or the other isn't a waste of time. However, in the spirit of the answer I will refrain from writing my own answer criticizing this answer. :)
– Trilarion
2 days ago
This answer is quite optimistic in some regards and seems to assume that spotting problems while reading publications is easy as well as redoing experiments to verify scientific claims one way or the other isn't a waste of time. However, in the spirit of the answer I will refrain from writing my own answer criticizing this answer. :)
– Trilarion
2 days ago
1
1
I don't know if spotting problems is easy per se, but it's such a routine part of the job that novices are, or should be, intensively trained in it. The fact that academia SE and similar sites constantly see questions like this one, from new PhD students, suggests that even people fairly new to the field are able to spot problems quickly.
– iayork
2 days ago
I don't know if spotting problems is easy per se, but it's such a routine part of the job that novices are, or should be, intensively trained in it. The fact that academia SE and similar sites constantly see questions like this one, from new PhD students, suggests that even people fairly new to the field are able to spot problems quickly.
– iayork
2 days ago
I think this is true for your immediate field of expertise, but if you move a bit further away it might be more complicated. What I think could help would be something similar to the customer reviews on Amazon product pages, in this case reader reviews of scientific publications. It would save me a lot of time if I could see what problems other readers have spotted without me needing to check every publication on my own every time. Problem spotting is much easier if we would all do it together instead of everyone for himself/herself.
– Trilarion
2 days ago
I think this is true for your immediate field of expertise, but if you move a bit further away it might be more complicated. What I think could help would be something similar to the customer reviews on Amazon product pages, in this case reader reviews of scientific publications. It would save me a lot of time if I could see what problems other readers have spotted without me needing to check every publication on my own every time. Problem spotting is much easier if we would all do it together instead of everyone for himself/herself.
– Trilarion
2 days ago
There is also a category "Analysis" in Nature: nature.com/nature/articles?type=analysis
– Norbert
yesterday
There is also a category "Analysis" in Nature: nature.com/nature/articles?type=analysis
– Norbert
yesterday
add a comment |
up vote
35
down vote
Many do, just in a polite way. Most of numerical modelling seems to be a variation of nicely stating "previous folks did things wrong and/or inefficient, we improve upon that; or at least offer another approach despite not showing their failings." There, criticism/failing of other approaches is often shown throughout the article.
Additionally, you are starting your PhD.
- What you feel might be a serious omission regarding method applicability could be a well-known limitation of the method everyone in the community is aware of. In this case, your "you are propagating BS" is a pointless and wrong rant - they know their methods better than you. Say everyone might already know X will offer about 1% higher error, but 100x easier simulation/experiment. If you suspect this might be the case, rather write a polite mail asking for clarification and whether the obtained results are still valid or how the approach could be improved etc etc.
- On the other hand, everyone could be assuming something that is actually incorrect for the problem, and you know it. Here, bluntly stating "you are propagating BS" isn't technically wrong, but you won't be getting any new friends with that. Rather write a nice article showing how and why everyone is approaching the problem wrong way and how you solve their issues - you would be instantly a highly regarded member of the community.
But yeah, a lot of fairly shaky research is taken as true and this leads to severe reproducibility problem all over fields. There are journals for publishing reproductions of other's work, but most prefer doing something original. Can't blame them: Invent X and have name immortalized vs "yeah, X's results check out".
New contributor
15
If you don't see criticism in papers, its because you are not recognizing it when you see it. Its there. Its just between the lines.
– Ian Sudbery
2 days ago
1
Generally good answer. Although repeating the premise that "everyone already knows X" alongside the premise that the OP doesn't is clearly contradictory.
– Lightness Races in Orbit
2 days ago
5
@LightnessRacesinOrbit "Everyone" means "most experts", not literally everyone.
– David Richerby
2 days ago
1
@DavidRicherby The word "everyone" means "everyone" - say what you mean!
– Lightness Races in Orbit
yesterday
6
@LightnessRacesinOrbit if you insist that the word "everyone" must mean literally every single person who is currently alive, the word becomes almost useless. Language is not mathematics. We don't communicate by concatenating dictionary definitions.
– David Richerby
yesterday
add a comment |
up vote
35
down vote
Many do, just in a polite way. Most of numerical modelling seems to be a variation of nicely stating "previous folks did things wrong and/or inefficient, we improve upon that; or at least offer another approach despite not showing their failings." There, criticism/failing of other approaches is often shown throughout the article.
Additionally, you are starting your PhD.
- What you feel might be a serious omission regarding method applicability could be a well-known limitation of the method everyone in the community is aware of. In this case, your "you are propagating BS" is a pointless and wrong rant - they know their methods better than you. Say everyone might already know X will offer about 1% higher error, but 100x easier simulation/experiment. If you suspect this might be the case, rather write a polite mail asking for clarification and whether the obtained results are still valid or how the approach could be improved etc etc.
- On the other hand, everyone could be assuming something that is actually incorrect for the problem, and you know it. Here, bluntly stating "you are propagating BS" isn't technically wrong, but you won't be getting any new friends with that. Rather write a nice article showing how and why everyone is approaching the problem wrong way and how you solve their issues - you would be instantly a highly regarded member of the community.
But yeah, a lot of fairly shaky research is taken as true and this leads to severe reproducibility problem all over fields. There are journals for publishing reproductions of other's work, but most prefer doing something original. Can't blame them: Invent X and have name immortalized vs "yeah, X's results check out".
New contributor
15
If you don't see criticism in papers, its because you are not recognizing it when you see it. Its there. Its just between the lines.
– Ian Sudbery
2 days ago
1
Generally good answer. Although repeating the premise that "everyone already knows X" alongside the premise that the OP doesn't is clearly contradictory.
– Lightness Races in Orbit
2 days ago
5
@LightnessRacesinOrbit "Everyone" means "most experts", not literally everyone.
– David Richerby
2 days ago
1
@DavidRicherby The word "everyone" means "everyone" - say what you mean!
– Lightness Races in Orbit
yesterday
6
@LightnessRacesinOrbit if you insist that the word "everyone" must mean literally every single person who is currently alive, the word becomes almost useless. Language is not mathematics. We don't communicate by concatenating dictionary definitions.
– David Richerby
yesterday
add a comment |
up vote
35
down vote
up vote
35
down vote
Many do, just in a polite way. Most of numerical modelling seems to be a variation of nicely stating "previous folks did things wrong and/or inefficient, we improve upon that; or at least offer another approach despite not showing their failings." There, criticism/failing of other approaches is often shown throughout the article.
Additionally, you are starting your PhD.
- What you feel might be a serious omission regarding method applicability could be a well-known limitation of the method everyone in the community is aware of. In this case, your "you are propagating BS" is a pointless and wrong rant - they know their methods better than you. Say everyone might already know X will offer about 1% higher error, but 100x easier simulation/experiment. If you suspect this might be the case, rather write a polite mail asking for clarification and whether the obtained results are still valid or how the approach could be improved etc etc.
- On the other hand, everyone could be assuming something that is actually incorrect for the problem, and you know it. Here, bluntly stating "you are propagating BS" isn't technically wrong, but you won't be getting any new friends with that. Rather write a nice article showing how and why everyone is approaching the problem wrong way and how you solve their issues - you would be instantly a highly regarded member of the community.
But yeah, a lot of fairly shaky research is taken as true and this leads to severe reproducibility problem all over fields. There are journals for publishing reproductions of other's work, but most prefer doing something original. Can't blame them: Invent X and have name immortalized vs "yeah, X's results check out".
New contributor
Many do, just in a polite way. Most of numerical modelling seems to be a variation of nicely stating "previous folks did things wrong and/or inefficient, we improve upon that; or at least offer another approach despite not showing their failings." There, criticism/failing of other approaches is often shown throughout the article.
Additionally, you are starting your PhD.
- What you feel might be a serious omission regarding method applicability could be a well-known limitation of the method everyone in the community is aware of. In this case, your "you are propagating BS" is a pointless and wrong rant - they know their methods better than you. Say everyone might already know X will offer about 1% higher error, but 100x easier simulation/experiment. If you suspect this might be the case, rather write a polite mail asking for clarification and whether the obtained results are still valid or how the approach could be improved etc etc.
- On the other hand, everyone could be assuming something that is actually incorrect for the problem, and you know it. Here, bluntly stating "you are propagating BS" isn't technically wrong, but you won't be getting any new friends with that. Rather write a nice article showing how and why everyone is approaching the problem wrong way and how you solve their issues - you would be instantly a highly regarded member of the community.
But yeah, a lot of fairly shaky research is taken as true and this leads to severe reproducibility problem all over fields. There are journals for publishing reproductions of other's work, but most prefer doing something original. Can't blame them: Invent X and have name immortalized vs "yeah, X's results check out".
New contributor
New contributor
answered 2 days ago
Zizy
34713
34713
New contributor
New contributor
15
If you don't see criticism in papers, its because you are not recognizing it when you see it. Its there. Its just between the lines.
– Ian Sudbery
2 days ago
1
Generally good answer. Although repeating the premise that "everyone already knows X" alongside the premise that the OP doesn't is clearly contradictory.
– Lightness Races in Orbit
2 days ago
5
@LightnessRacesinOrbit "Everyone" means "most experts", not literally everyone.
– David Richerby
2 days ago
1
@DavidRicherby The word "everyone" means "everyone" - say what you mean!
– Lightness Races in Orbit
yesterday
6
@LightnessRacesinOrbit if you insist that the word "everyone" must mean literally every single person who is currently alive, the word becomes almost useless. Language is not mathematics. We don't communicate by concatenating dictionary definitions.
– David Richerby
yesterday
add a comment |
15
If you don't see criticism in papers, its because you are not recognizing it when you see it. Its there. Its just between the lines.
– Ian Sudbery
2 days ago
1
Generally good answer. Although repeating the premise that "everyone already knows X" alongside the premise that the OP doesn't is clearly contradictory.
– Lightness Races in Orbit
2 days ago
5
@LightnessRacesinOrbit "Everyone" means "most experts", not literally everyone.
– David Richerby
2 days ago
1
@DavidRicherby The word "everyone" means "everyone" - say what you mean!
– Lightness Races in Orbit
yesterday
6
@LightnessRacesinOrbit if you insist that the word "everyone" must mean literally every single person who is currently alive, the word becomes almost useless. Language is not mathematics. We don't communicate by concatenating dictionary definitions.
– David Richerby
yesterday
15
15
If you don't see criticism in papers, its because you are not recognizing it when you see it. Its there. Its just between the lines.
– Ian Sudbery
2 days ago
If you don't see criticism in papers, its because you are not recognizing it when you see it. Its there. Its just between the lines.
– Ian Sudbery
2 days ago
1
1
Generally good answer. Although repeating the premise that "everyone already knows X" alongside the premise that the OP doesn't is clearly contradictory.
– Lightness Races in Orbit
2 days ago
Generally good answer. Although repeating the premise that "everyone already knows X" alongside the premise that the OP doesn't is clearly contradictory.
– Lightness Races in Orbit
2 days ago
5
5
@LightnessRacesinOrbit "Everyone" means "most experts", not literally everyone.
– David Richerby
2 days ago
@LightnessRacesinOrbit "Everyone" means "most experts", not literally everyone.
– David Richerby
2 days ago
1
1
@DavidRicherby The word "everyone" means "everyone" - say what you mean!
– Lightness Races in Orbit
yesterday
@DavidRicherby The word "everyone" means "everyone" - say what you mean!
– Lightness Races in Orbit
yesterday
6
6
@LightnessRacesinOrbit if you insist that the word "everyone" must mean literally every single person who is currently alive, the word becomes almost useless. Language is not mathematics. We don't communicate by concatenating dictionary definitions.
– David Richerby
yesterday
@LightnessRacesinOrbit if you insist that the word "everyone" must mean literally every single person who is currently alive, the word becomes almost useless. Language is not mathematics. We don't communicate by concatenating dictionary definitions.
– David Richerby
yesterday
add a comment |
up vote
15
down vote
Politics. Criticising peers directly leads to lack of political support.
This is because science is increasingly based on connections and short-term appraisal, and this is greatly detrimental to the advancement of hardcore science, and intellectualism in general.
We are social animals. This leads to a natural tendency of taking others' actions personally and "return favours". Also people are very wary of others' opinions and public image, and as a result are naturally sensitive to negative criticism. Usually, when you hurt the feelings of someone in public, you've made yourself an enemy.
I firmly believe in criticism and frankness as the shortest path to improvement. That has put me in delicate situations many times. In my early papers I was eager to lift obvious flaws in whatever specific topic being discussed, as to attempt to contribute. I have the habit of signing my peer reviews, and I am rather picky as a reviewer. This eventually resulted in being avoided in the citation and collaboration lists of relevant peers. Nowadays I am internally struggling on how to discuss faulty literature and how to approach clearly flawed logic.
I have been told that in order to say what you want you must reach a certain level of authority. However reaching that point largely depends on connections and good relationships. There are no Isaac Newtons today.
3
This eventually resulted in being avoided in the citation... lists of relevant peers. — I'm sorry you have unethical peers.
– JeffE
yesterday
This answer makes me sad.
– goblin
13 hours ago
add a comment |
up vote
15
down vote
Politics. Criticising peers directly leads to lack of political support.
This is because science is increasingly based on connections and short-term appraisal, and this is greatly detrimental to the advancement of hardcore science, and intellectualism in general.
We are social animals. This leads to a natural tendency of taking others' actions personally and "return favours". Also people are very wary of others' opinions and public image, and as a result are naturally sensitive to negative criticism. Usually, when you hurt the feelings of someone in public, you've made yourself an enemy.
I firmly believe in criticism and frankness as the shortest path to improvement. That has put me in delicate situations many times. In my early papers I was eager to lift obvious flaws in whatever specific topic being discussed, as to attempt to contribute. I have the habit of signing my peer reviews, and I am rather picky as a reviewer. This eventually resulted in being avoided in the citation and collaboration lists of relevant peers. Nowadays I am internally struggling on how to discuss faulty literature and how to approach clearly flawed logic.
I have been told that in order to say what you want you must reach a certain level of authority. However reaching that point largely depends on connections and good relationships. There are no Isaac Newtons today.
3
This eventually resulted in being avoided in the citation... lists of relevant peers. — I'm sorry you have unethical peers.
– JeffE
yesterday
This answer makes me sad.
– goblin
13 hours ago
add a comment |
up vote
15
down vote
up vote
15
down vote
Politics. Criticising peers directly leads to lack of political support.
This is because science is increasingly based on connections and short-term appraisal, and this is greatly detrimental to the advancement of hardcore science, and intellectualism in general.
We are social animals. This leads to a natural tendency of taking others' actions personally and "return favours". Also people are very wary of others' opinions and public image, and as a result are naturally sensitive to negative criticism. Usually, when you hurt the feelings of someone in public, you've made yourself an enemy.
I firmly believe in criticism and frankness as the shortest path to improvement. That has put me in delicate situations many times. In my early papers I was eager to lift obvious flaws in whatever specific topic being discussed, as to attempt to contribute. I have the habit of signing my peer reviews, and I am rather picky as a reviewer. This eventually resulted in being avoided in the citation and collaboration lists of relevant peers. Nowadays I am internally struggling on how to discuss faulty literature and how to approach clearly flawed logic.
I have been told that in order to say what you want you must reach a certain level of authority. However reaching that point largely depends on connections and good relationships. There are no Isaac Newtons today.
Politics. Criticising peers directly leads to lack of political support.
This is because science is increasingly based on connections and short-term appraisal, and this is greatly detrimental to the advancement of hardcore science, and intellectualism in general.
We are social animals. This leads to a natural tendency of taking others' actions personally and "return favours". Also people are very wary of others' opinions and public image, and as a result are naturally sensitive to negative criticism. Usually, when you hurt the feelings of someone in public, you've made yourself an enemy.
I firmly believe in criticism and frankness as the shortest path to improvement. That has put me in delicate situations many times. In my early papers I was eager to lift obvious flaws in whatever specific topic being discussed, as to attempt to contribute. I have the habit of signing my peer reviews, and I am rather picky as a reviewer. This eventually resulted in being avoided in the citation and collaboration lists of relevant peers. Nowadays I am internally struggling on how to discuss faulty literature and how to approach clearly flawed logic.
I have been told that in order to say what you want you must reach a certain level of authority. However reaching that point largely depends on connections and good relationships. There are no Isaac Newtons today.
answered 2 days ago
Scientist
6,45412355
6,45412355
3
This eventually resulted in being avoided in the citation... lists of relevant peers. — I'm sorry you have unethical peers.
– JeffE
yesterday
This answer makes me sad.
– goblin
13 hours ago
add a comment |
3
This eventually resulted in being avoided in the citation... lists of relevant peers. — I'm sorry you have unethical peers.
– JeffE
yesterday
This answer makes me sad.
– goblin
13 hours ago
3
3
This eventually resulted in being avoided in the citation... lists of relevant peers. — I'm sorry you have unethical peers.
– JeffE
yesterday
This eventually resulted in being avoided in the citation... lists of relevant peers. — I'm sorry you have unethical peers.
– JeffE
yesterday
This answer makes me sad.
– goblin
13 hours ago
This answer makes me sad.
– goblin
13 hours ago
add a comment |
up vote
9
down vote
Have you ever encountered a "Letter-to-editor" article?
Check this, for instance, an article on the Turkish Journal of Urology about writing a letter to the editor:
How to write an editorial letter?
It is a kind of a "short-communication", and generally used to criticize previously published articles. If this is uncommon or absent in any of research subject, it is more of an issue of the researchers in that field.
2
The key question there is I guess if anyone will ever see such a critique. I mean, I encounter most of the papers I read either by a search engine or by being linked to it by someone. Does anyone still read entire journals, introductions and all?
– KubaFYI
2 days ago
4
@KubaFYI These responses/letters act just like other journal articles - they get indexed and have their own reference. They can therefore crop up in a Google Scholar search or similar. In addition, journals websites often prominently link to the response from the original article's page. For example, see this article: pnas.org/content/115/32/8221
– user2390246
2 days ago
2
@KubaFYI: Any time I see a paper that seems wrong to me, the first thing I do is to do a citation search for other papers that cite that paper. If someone has published a letter or comment saying it's wrong, it will pop up, because they will cite the paper they're criticizing.
– Ben Crowell
2 days ago
add a comment |
up vote
9
down vote
Have you ever encountered a "Letter-to-editor" article?
Check this, for instance, an article on the Turkish Journal of Urology about writing a letter to the editor:
How to write an editorial letter?
It is a kind of a "short-communication", and generally used to criticize previously published articles. If this is uncommon or absent in any of research subject, it is more of an issue of the researchers in that field.
2
The key question there is I guess if anyone will ever see such a critique. I mean, I encounter most of the papers I read either by a search engine or by being linked to it by someone. Does anyone still read entire journals, introductions and all?
– KubaFYI
2 days ago
4
@KubaFYI These responses/letters act just like other journal articles - they get indexed and have their own reference. They can therefore crop up in a Google Scholar search or similar. In addition, journals websites often prominently link to the response from the original article's page. For example, see this article: pnas.org/content/115/32/8221
– user2390246
2 days ago
2
@KubaFYI: Any time I see a paper that seems wrong to me, the first thing I do is to do a citation search for other papers that cite that paper. If someone has published a letter or comment saying it's wrong, it will pop up, because they will cite the paper they're criticizing.
– Ben Crowell
2 days ago
add a comment |
up vote
9
down vote
up vote
9
down vote
Have you ever encountered a "Letter-to-editor" article?
Check this, for instance, an article on the Turkish Journal of Urology about writing a letter to the editor:
How to write an editorial letter?
It is a kind of a "short-communication", and generally used to criticize previously published articles. If this is uncommon or absent in any of research subject, it is more of an issue of the researchers in that field.
Have you ever encountered a "Letter-to-editor" article?
Check this, for instance, an article on the Turkish Journal of Urology about writing a letter to the editor:
How to write an editorial letter?
It is a kind of a "short-communication", and generally used to criticize previously published articles. If this is uncommon or absent in any of research subject, it is more of an issue of the researchers in that field.
answered 2 days ago
Güray Hatipoğlu
1,147215
1,147215
2
The key question there is I guess if anyone will ever see such a critique. I mean, I encounter most of the papers I read either by a search engine or by being linked to it by someone. Does anyone still read entire journals, introductions and all?
– KubaFYI
2 days ago
4
@KubaFYI These responses/letters act just like other journal articles - they get indexed and have their own reference. They can therefore crop up in a Google Scholar search or similar. In addition, journals websites often prominently link to the response from the original article's page. For example, see this article: pnas.org/content/115/32/8221
– user2390246
2 days ago
2
@KubaFYI: Any time I see a paper that seems wrong to me, the first thing I do is to do a citation search for other papers that cite that paper. If someone has published a letter or comment saying it's wrong, it will pop up, because they will cite the paper they're criticizing.
– Ben Crowell
2 days ago
add a comment |
2
The key question there is I guess if anyone will ever see such a critique. I mean, I encounter most of the papers I read either by a search engine or by being linked to it by someone. Does anyone still read entire journals, introductions and all?
– KubaFYI
2 days ago
4
@KubaFYI These responses/letters act just like other journal articles - they get indexed and have their own reference. They can therefore crop up in a Google Scholar search or similar. In addition, journals websites often prominently link to the response from the original article's page. For example, see this article: pnas.org/content/115/32/8221
– user2390246
2 days ago
2
@KubaFYI: Any time I see a paper that seems wrong to me, the first thing I do is to do a citation search for other papers that cite that paper. If someone has published a letter or comment saying it's wrong, it will pop up, because they will cite the paper they're criticizing.
– Ben Crowell
2 days ago
2
2
The key question there is I guess if anyone will ever see such a critique. I mean, I encounter most of the papers I read either by a search engine or by being linked to it by someone. Does anyone still read entire journals, introductions and all?
– KubaFYI
2 days ago
The key question there is I guess if anyone will ever see such a critique. I mean, I encounter most of the papers I read either by a search engine or by being linked to it by someone. Does anyone still read entire journals, introductions and all?
– KubaFYI
2 days ago
4
4
@KubaFYI These responses/letters act just like other journal articles - they get indexed and have their own reference. They can therefore crop up in a Google Scholar search or similar. In addition, journals websites often prominently link to the response from the original article's page. For example, see this article: pnas.org/content/115/32/8221
– user2390246
2 days ago
@KubaFYI These responses/letters act just like other journal articles - they get indexed and have their own reference. They can therefore crop up in a Google Scholar search or similar. In addition, journals websites often prominently link to the response from the original article's page. For example, see this article: pnas.org/content/115/32/8221
– user2390246
2 days ago
2
2
@KubaFYI: Any time I see a paper that seems wrong to me, the first thing I do is to do a citation search for other papers that cite that paper. If someone has published a letter or comment saying it's wrong, it will pop up, because they will cite the paper they're criticizing.
– Ben Crowell
2 days ago
@KubaFYI: Any time I see a paper that seems wrong to me, the first thing I do is to do a citation search for other papers that cite that paper. If someone has published a letter or comment saying it's wrong, it will pop up, because they will cite the paper they're criticizing.
– Ben Crowell
2 days ago
add a comment |
up vote
9
down vote
I am going to convert my comment into answer. Most of the researcher are aware of this but there is a pressure in majority of academia around the globe and that is Publish or Perish.
On average Writing a good article takes 1-2 years but to save tenure (in my country) an AP has to publish 3 SCI articles per year. With such kind of pressure a researcher can not write a quality research paper. Moreover, strong critique require time to analyse the findings of other researchers, while the race of publish don’t allow to spend much time on that part of article, which I believe is one of the most important part of article.
Yuval Noah Harari author of famous book Sapiens: A Brief History of Humankind, once he said in his interview that he start thinking out of box when he promoted to Associate Professor position, because in his university there is no requirement of publication once you get Associate Professor position. That's why he wrote this book within 2 years of time.
1
@scaaahu Critiques require time to analyse the findings of other researchers, while due to race of publish mostly researcher avoid it.
– MBK
2 days ago
4
We really need to all sit down and collectively agree that going on like that doesn't make any sense, if we are to be true to the explicit goal of academia of advancing human understanding of the world. Now what is just a metric had become the implicit goal.
– KubaFYI
2 days ago
2
I'm pretty sure that all (or many) academics do agree with this. Unfortunately, while the explicit goal of academics might be advancing human knowledge, the academy is not run by academics. Mostly funding bodies, governments and university administrators to make the decisions..
– Ian Sudbery
2 days ago
add a comment |
up vote
9
down vote
I am going to convert my comment into answer. Most of the researcher are aware of this but there is a pressure in majority of academia around the globe and that is Publish or Perish.
On average Writing a good article takes 1-2 years but to save tenure (in my country) an AP has to publish 3 SCI articles per year. With such kind of pressure a researcher can not write a quality research paper. Moreover, strong critique require time to analyse the findings of other researchers, while the race of publish don’t allow to spend much time on that part of article, which I believe is one of the most important part of article.
Yuval Noah Harari author of famous book Sapiens: A Brief History of Humankind, once he said in his interview that he start thinking out of box when he promoted to Associate Professor position, because in his university there is no requirement of publication once you get Associate Professor position. That's why he wrote this book within 2 years of time.
1
@scaaahu Critiques require time to analyse the findings of other researchers, while due to race of publish mostly researcher avoid it.
– MBK
2 days ago
4
We really need to all sit down and collectively agree that going on like that doesn't make any sense, if we are to be true to the explicit goal of academia of advancing human understanding of the world. Now what is just a metric had become the implicit goal.
– KubaFYI
2 days ago
2
I'm pretty sure that all (or many) academics do agree with this. Unfortunately, while the explicit goal of academics might be advancing human knowledge, the academy is not run by academics. Mostly funding bodies, governments and university administrators to make the decisions..
– Ian Sudbery
2 days ago
add a comment |
up vote
9
down vote
up vote
9
down vote
I am going to convert my comment into answer. Most of the researcher are aware of this but there is a pressure in majority of academia around the globe and that is Publish or Perish.
On average Writing a good article takes 1-2 years but to save tenure (in my country) an AP has to publish 3 SCI articles per year. With such kind of pressure a researcher can not write a quality research paper. Moreover, strong critique require time to analyse the findings of other researchers, while the race of publish don’t allow to spend much time on that part of article, which I believe is one of the most important part of article.
Yuval Noah Harari author of famous book Sapiens: A Brief History of Humankind, once he said in his interview that he start thinking out of box when he promoted to Associate Professor position, because in his university there is no requirement of publication once you get Associate Professor position. That's why he wrote this book within 2 years of time.
I am going to convert my comment into answer. Most of the researcher are aware of this but there is a pressure in majority of academia around the globe and that is Publish or Perish.
On average Writing a good article takes 1-2 years but to save tenure (in my country) an AP has to publish 3 SCI articles per year. With such kind of pressure a researcher can not write a quality research paper. Moreover, strong critique require time to analyse the findings of other researchers, while the race of publish don’t allow to spend much time on that part of article, which I believe is one of the most important part of article.
Yuval Noah Harari author of famous book Sapiens: A Brief History of Humankind, once he said in his interview that he start thinking out of box when he promoted to Associate Professor position, because in his university there is no requirement of publication once you get Associate Professor position. That's why he wrote this book within 2 years of time.
edited 2 days ago
answered 2 days ago
MBK
1,9341123
1,9341123
1
@scaaahu Critiques require time to analyse the findings of other researchers, while due to race of publish mostly researcher avoid it.
– MBK
2 days ago
4
We really need to all sit down and collectively agree that going on like that doesn't make any sense, if we are to be true to the explicit goal of academia of advancing human understanding of the world. Now what is just a metric had become the implicit goal.
– KubaFYI
2 days ago
2
I'm pretty sure that all (or many) academics do agree with this. Unfortunately, while the explicit goal of academics might be advancing human knowledge, the academy is not run by academics. Mostly funding bodies, governments and university administrators to make the decisions..
– Ian Sudbery
2 days ago
add a comment |
1
@scaaahu Critiques require time to analyse the findings of other researchers, while due to race of publish mostly researcher avoid it.
– MBK
2 days ago
4
We really need to all sit down and collectively agree that going on like that doesn't make any sense, if we are to be true to the explicit goal of academia of advancing human understanding of the world. Now what is just a metric had become the implicit goal.
– KubaFYI
2 days ago
2
I'm pretty sure that all (or many) academics do agree with this. Unfortunately, while the explicit goal of academics might be advancing human knowledge, the academy is not run by academics. Mostly funding bodies, governments and university administrators to make the decisions..
– Ian Sudbery
2 days ago
1
1
@scaaahu Critiques require time to analyse the findings of other researchers, while due to race of publish mostly researcher avoid it.
– MBK
2 days ago
@scaaahu Critiques require time to analyse the findings of other researchers, while due to race of publish mostly researcher avoid it.
– MBK
2 days ago
4
4
We really need to all sit down and collectively agree that going on like that doesn't make any sense, if we are to be true to the explicit goal of academia of advancing human understanding of the world. Now what is just a metric had become the implicit goal.
– KubaFYI
2 days ago
We really need to all sit down and collectively agree that going on like that doesn't make any sense, if we are to be true to the explicit goal of academia of advancing human understanding of the world. Now what is just a metric had become the implicit goal.
– KubaFYI
2 days ago
2
2
I'm pretty sure that all (or many) academics do agree with this. Unfortunately, while the explicit goal of academics might be advancing human knowledge, the academy is not run by academics. Mostly funding bodies, governments and university administrators to make the decisions..
– Ian Sudbery
2 days ago
I'm pretty sure that all (or many) academics do agree with this. Unfortunately, while the explicit goal of academics might be advancing human knowledge, the academy is not run by academics. Mostly funding bodies, governments and university administrators to make the decisions..
– Ian Sudbery
2 days ago
add a comment |
up vote
6
down vote
Simply speaking: The better long term strategy for staying in academia is to be the nice guy that plays along.
Academia is full of systemic corruption because jobs in a specific field of research are very limited/competitive due to the pyramid nature of human resources (number of students > number of PhDs > number of professors). Academia is structured around the selection of outstanding and remarkable individuals. However, history has told us that most metrics tend to be overshadowed by character traits in the long run. Especially if you think about the fact that most professors are not doing much research themselves but are rather responslible for the management of workgroups and projects. Hence, the nice, non-critical guy has a structural advantage in the system. Letters of recommendation, which tend to be more positive for a friendly person compared to "troublemakers", are also a direct indication of this.
The system is the people and the people select their system. People don't like criticism. So the system/people deselected criticism.
2
most professors are not doing much research themselves — ...in some fields. This is certainly not true in all fields.
– JeffE
yesterday
add a comment |
up vote
6
down vote
Simply speaking: The better long term strategy for staying in academia is to be the nice guy that plays along.
Academia is full of systemic corruption because jobs in a specific field of research are very limited/competitive due to the pyramid nature of human resources (number of students > number of PhDs > number of professors). Academia is structured around the selection of outstanding and remarkable individuals. However, history has told us that most metrics tend to be overshadowed by character traits in the long run. Especially if you think about the fact that most professors are not doing much research themselves but are rather responslible for the management of workgroups and projects. Hence, the nice, non-critical guy has a structural advantage in the system. Letters of recommendation, which tend to be more positive for a friendly person compared to "troublemakers", are also a direct indication of this.
The system is the people and the people select their system. People don't like criticism. So the system/people deselected criticism.
2
most professors are not doing much research themselves — ...in some fields. This is certainly not true in all fields.
– JeffE
yesterday
add a comment |
up vote
6
down vote
up vote
6
down vote
Simply speaking: The better long term strategy for staying in academia is to be the nice guy that plays along.
Academia is full of systemic corruption because jobs in a specific field of research are very limited/competitive due to the pyramid nature of human resources (number of students > number of PhDs > number of professors). Academia is structured around the selection of outstanding and remarkable individuals. However, history has told us that most metrics tend to be overshadowed by character traits in the long run. Especially if you think about the fact that most professors are not doing much research themselves but are rather responslible for the management of workgroups and projects. Hence, the nice, non-critical guy has a structural advantage in the system. Letters of recommendation, which tend to be more positive for a friendly person compared to "troublemakers", are also a direct indication of this.
The system is the people and the people select their system. People don't like criticism. So the system/people deselected criticism.
Simply speaking: The better long term strategy for staying in academia is to be the nice guy that plays along.
Academia is full of systemic corruption because jobs in a specific field of research are very limited/competitive due to the pyramid nature of human resources (number of students > number of PhDs > number of professors). Academia is structured around the selection of outstanding and remarkable individuals. However, history has told us that most metrics tend to be overshadowed by character traits in the long run. Especially if you think about the fact that most professors are not doing much research themselves but are rather responslible for the management of workgroups and projects. Hence, the nice, non-critical guy has a structural advantage in the system. Letters of recommendation, which tend to be more positive for a friendly person compared to "troublemakers", are also a direct indication of this.
The system is the people and the people select their system. People don't like criticism. So the system/people deselected criticism.
edited yesterday
answered yesterday
image
22526
22526
2
most professors are not doing much research themselves — ...in some fields. This is certainly not true in all fields.
– JeffE
yesterday
add a comment |
2
most professors are not doing much research themselves — ...in some fields. This is certainly not true in all fields.
– JeffE
yesterday
2
2
most professors are not doing much research themselves — ...in some fields. This is certainly not true in all fields.
– JeffE
yesterday
most professors are not doing much research themselves — ...in some fields. This is certainly not true in all fields.
– JeffE
yesterday
add a comment |
up vote
3
down vote
Other answers discuss good reasons (besides the fair point that answers do exist): e.g. good criticism takes time, also the authors may know the specifics better than you.
Another reason why criticism is usually very tuned down: the reviewers for your article are selected based on familiarity with the field, which means they possibly have a (favorite) method / approach of their own. So new articles need to strike a balance between saying "the existing methods are insufficient" (which is why we are publishing our novel one) while at the same time not being too harsh (which will offend the reviewers).
New contributor
add a comment |
up vote
3
down vote
Other answers discuss good reasons (besides the fair point that answers do exist): e.g. good criticism takes time, also the authors may know the specifics better than you.
Another reason why criticism is usually very tuned down: the reviewers for your article are selected based on familiarity with the field, which means they possibly have a (favorite) method / approach of their own. So new articles need to strike a balance between saying "the existing methods are insufficient" (which is why we are publishing our novel one) while at the same time not being too harsh (which will offend the reviewers).
New contributor
add a comment |
up vote
3
down vote
up vote
3
down vote
Other answers discuss good reasons (besides the fair point that answers do exist): e.g. good criticism takes time, also the authors may know the specifics better than you.
Another reason why criticism is usually very tuned down: the reviewers for your article are selected based on familiarity with the field, which means they possibly have a (favorite) method / approach of their own. So new articles need to strike a balance between saying "the existing methods are insufficient" (which is why we are publishing our novel one) while at the same time not being too harsh (which will offend the reviewers).
New contributor
Other answers discuss good reasons (besides the fair point that answers do exist): e.g. good criticism takes time, also the authors may know the specifics better than you.
Another reason why criticism is usually very tuned down: the reviewers for your article are selected based on familiarity with the field, which means they possibly have a (favorite) method / approach of their own. So new articles need to strike a balance between saying "the existing methods are insufficient" (which is why we are publishing our novel one) while at the same time not being too harsh (which will offend the reviewers).
New contributor
New contributor
answered 2 days ago
cheersmate
1314
1314
New contributor
New contributor
add a comment |
add a comment |
up vote
1
down vote
- Criticism is published, as pointed out by iayork. It may also appear in less formal places like blogs. So one reason you don't see it is that you aren't looking in the right places.
- The best place for criticism is in the peer review process. (That is, peer review should catch bad work before it is published, rendering subsequent criticism unnecessary. However, bad work does sometimes get published, so criticism of published work is appropriate then.) Reviews are not usually published, but some venues do publish reviews. E.g., ICLR and NeurIPS reviews are available with the papers (for ICLR, even the rejected papers have reviews published).
- Constructive criticism is better than simply trashing other people's work. Attacks can backfire -- they make the attacker look unprofessional, and things can degenerate into a back-and-forth flame war. Constructive criticism appears in the form of follow-up work that fixes the shortcomings of prior work. This is more polite and less obvious, so you may not notice it.
- Even when bad work gets published, it may simply be best to ignore it. Engaging in criticism is a messy business and some of the mud may stick to the critic. Criticism usually only appears when the work in question is getting a lot of attention.
2
"The right place for criticism is in the peer review process." -- is this to say published work ought not be criticised? I don't see why.
– Scientist
2 days ago
1
@Scientist Of course published work should be criticized! However, peer review should be the first line of defense. Most of the time when published work is being criticized, it's because it should not have been published in the first place. I have edited to emphasize this.
– Thomas
2 days ago
2
"it's because it should not have been published in the first place" You are assuming the criticism is because of poor quality or (obviously) wrong conclusions. In this case yes, it shouldn't be published. But there are a lot of cases where different views exist, different models, different explanations or similar. Or both the authors and reviewers missed something. In that case review is the worst place to put your critisim (although some try) since it prohibits the exchange of ideas and also doesn't get published.
– DSVA
yesterday
2
So what would you call it then? Here's a paper clearly critisicing the idea of "secondary orbital interactions" pubs.acs.org/doi/abs/10.1021/ar0000152 and here's a response ncbi.nlm.nih.gov/pubmed/17109435 clearly criticising the first paper. "Criticism is the practice of judging the merits and faults of something."
– DSVA
yesterday
1
I’m sure all published literature is open for criticism, and must be criticized. Science is built on correction of flawed assumptions. We’re always off the mark and missing important factors. It’s a never ending discussion.
– Scientist
yesterday
|
show 3 more comments
up vote
1
down vote
- Criticism is published, as pointed out by iayork. It may also appear in less formal places like blogs. So one reason you don't see it is that you aren't looking in the right places.
- The best place for criticism is in the peer review process. (That is, peer review should catch bad work before it is published, rendering subsequent criticism unnecessary. However, bad work does sometimes get published, so criticism of published work is appropriate then.) Reviews are not usually published, but some venues do publish reviews. E.g., ICLR and NeurIPS reviews are available with the papers (for ICLR, even the rejected papers have reviews published).
- Constructive criticism is better than simply trashing other people's work. Attacks can backfire -- they make the attacker look unprofessional, and things can degenerate into a back-and-forth flame war. Constructive criticism appears in the form of follow-up work that fixes the shortcomings of prior work. This is more polite and less obvious, so you may not notice it.
- Even when bad work gets published, it may simply be best to ignore it. Engaging in criticism is a messy business and some of the mud may stick to the critic. Criticism usually only appears when the work in question is getting a lot of attention.
2
"The right place for criticism is in the peer review process." -- is this to say published work ought not be criticised? I don't see why.
– Scientist
2 days ago
1
@Scientist Of course published work should be criticized! However, peer review should be the first line of defense. Most of the time when published work is being criticized, it's because it should not have been published in the first place. I have edited to emphasize this.
– Thomas
2 days ago
2
"it's because it should not have been published in the first place" You are assuming the criticism is because of poor quality or (obviously) wrong conclusions. In this case yes, it shouldn't be published. But there are a lot of cases where different views exist, different models, different explanations or similar. Or both the authors and reviewers missed something. In that case review is the worst place to put your critisim (although some try) since it prohibits the exchange of ideas and also doesn't get published.
– DSVA
yesterday
2
So what would you call it then? Here's a paper clearly critisicing the idea of "secondary orbital interactions" pubs.acs.org/doi/abs/10.1021/ar0000152 and here's a response ncbi.nlm.nih.gov/pubmed/17109435 clearly criticising the first paper. "Criticism is the practice of judging the merits and faults of something."
– DSVA
yesterday
1
I’m sure all published literature is open for criticism, and must be criticized. Science is built on correction of flawed assumptions. We’re always off the mark and missing important factors. It’s a never ending discussion.
– Scientist
yesterday
|
show 3 more comments
up vote
1
down vote
up vote
1
down vote
- Criticism is published, as pointed out by iayork. It may also appear in less formal places like blogs. So one reason you don't see it is that you aren't looking in the right places.
- The best place for criticism is in the peer review process. (That is, peer review should catch bad work before it is published, rendering subsequent criticism unnecessary. However, bad work does sometimes get published, so criticism of published work is appropriate then.) Reviews are not usually published, but some venues do publish reviews. E.g., ICLR and NeurIPS reviews are available with the papers (for ICLR, even the rejected papers have reviews published).
- Constructive criticism is better than simply trashing other people's work. Attacks can backfire -- they make the attacker look unprofessional, and things can degenerate into a back-and-forth flame war. Constructive criticism appears in the form of follow-up work that fixes the shortcomings of prior work. This is more polite and less obvious, so you may not notice it.
- Even when bad work gets published, it may simply be best to ignore it. Engaging in criticism is a messy business and some of the mud may stick to the critic. Criticism usually only appears when the work in question is getting a lot of attention.
- Criticism is published, as pointed out by iayork. It may also appear in less formal places like blogs. So one reason you don't see it is that you aren't looking in the right places.
- The best place for criticism is in the peer review process. (That is, peer review should catch bad work before it is published, rendering subsequent criticism unnecessary. However, bad work does sometimes get published, so criticism of published work is appropriate then.) Reviews are not usually published, but some venues do publish reviews. E.g., ICLR and NeurIPS reviews are available with the papers (for ICLR, even the rejected papers have reviews published).
- Constructive criticism is better than simply trashing other people's work. Attacks can backfire -- they make the attacker look unprofessional, and things can degenerate into a back-and-forth flame war. Constructive criticism appears in the form of follow-up work that fixes the shortcomings of prior work. This is more polite and less obvious, so you may not notice it.
- Even when bad work gets published, it may simply be best to ignore it. Engaging in criticism is a messy business and some of the mud may stick to the critic. Criticism usually only appears when the work in question is getting a lot of attention.
edited 2 days ago
answered 2 days ago
Thomas
9,13042037
9,13042037
2
"The right place for criticism is in the peer review process." -- is this to say published work ought not be criticised? I don't see why.
– Scientist
2 days ago
1
@Scientist Of course published work should be criticized! However, peer review should be the first line of defense. Most of the time when published work is being criticized, it's because it should not have been published in the first place. I have edited to emphasize this.
– Thomas
2 days ago
2
"it's because it should not have been published in the first place" You are assuming the criticism is because of poor quality or (obviously) wrong conclusions. In this case yes, it shouldn't be published. But there are a lot of cases where different views exist, different models, different explanations or similar. Or both the authors and reviewers missed something. In that case review is the worst place to put your critisim (although some try) since it prohibits the exchange of ideas and also doesn't get published.
– DSVA
yesterday
2
So what would you call it then? Here's a paper clearly critisicing the idea of "secondary orbital interactions" pubs.acs.org/doi/abs/10.1021/ar0000152 and here's a response ncbi.nlm.nih.gov/pubmed/17109435 clearly criticising the first paper. "Criticism is the practice of judging the merits and faults of something."
– DSVA
yesterday
1
I’m sure all published literature is open for criticism, and must be criticized. Science is built on correction of flawed assumptions. We’re always off the mark and missing important factors. It’s a never ending discussion.
– Scientist
yesterday
|
show 3 more comments
2
"The right place for criticism is in the peer review process." -- is this to say published work ought not be criticised? I don't see why.
– Scientist
2 days ago
1
@Scientist Of course published work should be criticized! However, peer review should be the first line of defense. Most of the time when published work is being criticized, it's because it should not have been published in the first place. I have edited to emphasize this.
– Thomas
2 days ago
2
"it's because it should not have been published in the first place" You are assuming the criticism is because of poor quality or (obviously) wrong conclusions. In this case yes, it shouldn't be published. But there are a lot of cases where different views exist, different models, different explanations or similar. Or both the authors and reviewers missed something. In that case review is the worst place to put your critisim (although some try) since it prohibits the exchange of ideas and also doesn't get published.
– DSVA
yesterday
2
So what would you call it then? Here's a paper clearly critisicing the idea of "secondary orbital interactions" pubs.acs.org/doi/abs/10.1021/ar0000152 and here's a response ncbi.nlm.nih.gov/pubmed/17109435 clearly criticising the first paper. "Criticism is the practice of judging the merits and faults of something."
– DSVA
yesterday
1
I’m sure all published literature is open for criticism, and must be criticized. Science is built on correction of flawed assumptions. We’re always off the mark and missing important factors. It’s a never ending discussion.
– Scientist
yesterday
2
2
"The right place for criticism is in the peer review process." -- is this to say published work ought not be criticised? I don't see why.
– Scientist
2 days ago
"The right place for criticism is in the peer review process." -- is this to say published work ought not be criticised? I don't see why.
– Scientist
2 days ago
1
1
@Scientist Of course published work should be criticized! However, peer review should be the first line of defense. Most of the time when published work is being criticized, it's because it should not have been published in the first place. I have edited to emphasize this.
– Thomas
2 days ago
@Scientist Of course published work should be criticized! However, peer review should be the first line of defense. Most of the time when published work is being criticized, it's because it should not have been published in the first place. I have edited to emphasize this.
– Thomas
2 days ago
2
2
"it's because it should not have been published in the first place" You are assuming the criticism is because of poor quality or (obviously) wrong conclusions. In this case yes, it shouldn't be published. But there are a lot of cases where different views exist, different models, different explanations or similar. Or both the authors and reviewers missed something. In that case review is the worst place to put your critisim (although some try) since it prohibits the exchange of ideas and also doesn't get published.
– DSVA
yesterday
"it's because it should not have been published in the first place" You are assuming the criticism is because of poor quality or (obviously) wrong conclusions. In this case yes, it shouldn't be published. But there are a lot of cases where different views exist, different models, different explanations or similar. Or both the authors and reviewers missed something. In that case review is the worst place to put your critisim (although some try) since it prohibits the exchange of ideas and also doesn't get published.
– DSVA
yesterday
2
2
So what would you call it then? Here's a paper clearly critisicing the idea of "secondary orbital interactions" pubs.acs.org/doi/abs/10.1021/ar0000152 and here's a response ncbi.nlm.nih.gov/pubmed/17109435 clearly criticising the first paper. "Criticism is the practice of judging the merits and faults of something."
– DSVA
yesterday
So what would you call it then? Here's a paper clearly critisicing the idea of "secondary orbital interactions" pubs.acs.org/doi/abs/10.1021/ar0000152 and here's a response ncbi.nlm.nih.gov/pubmed/17109435 clearly criticising the first paper. "Criticism is the practice of judging the merits and faults of something."
– DSVA
yesterday
1
1
I’m sure all published literature is open for criticism, and must be criticized. Science is built on correction of flawed assumptions. We’re always off the mark and missing important factors. It’s a never ending discussion.
– Scientist
yesterday
I’m sure all published literature is open for criticism, and must be criticized. Science is built on correction of flawed assumptions. We’re always off the mark and missing important factors. It’s a never ending discussion.
– Scientist
yesterday
|
show 3 more comments
up vote
1
down vote
After the second year of Ph.D. study, I started to learn extensively on how to write highly regarded publications. The first thing that was hammered in me is how to describe the assumptions and limitations of the given research, and give a framework or scope of the given research. My first publication described many of these limitations and scope in detail, but my second and third publications did not because I already established these issues in my first one. So, it is paramount that you check ALL the citations, especially the ones by the same author. These usually give you the 'background' of the research. In addition, be familiar with 'well known' assumptions, and practices as well. They are accepted for a reason.
Second, I have seen an article (peer-reviewed in high-ranking journal) that outright attacked another earlier published work by another author. Even though the attacking article may be correct, it did not exactly contribute anything to the field. On other hand, if you developed a novel methodology that addresses the issue that was presented in that offending publication, you can say, "here's my work on this, and I did this." and then cite that paper, and then say something along like, 'improves the results because their approach did not address this particular issue'.
Third,if you burn bridges and piss off the wrong people by attacking them outright, you will not get very far. It is a fact of life. Learn how to navigate the world with proper conduct and delicate touch (people ARE sensitive), and stay true to yourself.
add a comment |
up vote
1
down vote
After the second year of Ph.D. study, I started to learn extensively on how to write highly regarded publications. The first thing that was hammered in me is how to describe the assumptions and limitations of the given research, and give a framework or scope of the given research. My first publication described many of these limitations and scope in detail, but my second and third publications did not because I already established these issues in my first one. So, it is paramount that you check ALL the citations, especially the ones by the same author. These usually give you the 'background' of the research. In addition, be familiar with 'well known' assumptions, and practices as well. They are accepted for a reason.
Second, I have seen an article (peer-reviewed in high-ranking journal) that outright attacked another earlier published work by another author. Even though the attacking article may be correct, it did not exactly contribute anything to the field. On other hand, if you developed a novel methodology that addresses the issue that was presented in that offending publication, you can say, "here's my work on this, and I did this." and then cite that paper, and then say something along like, 'improves the results because their approach did not address this particular issue'.
Third,if you burn bridges and piss off the wrong people by attacking them outright, you will not get very far. It is a fact of life. Learn how to navigate the world with proper conduct and delicate touch (people ARE sensitive), and stay true to yourself.
add a comment |
up vote
1
down vote
up vote
1
down vote
After the second year of Ph.D. study, I started to learn extensively on how to write highly regarded publications. The first thing that was hammered in me is how to describe the assumptions and limitations of the given research, and give a framework or scope of the given research. My first publication described many of these limitations and scope in detail, but my second and third publications did not because I already established these issues in my first one. So, it is paramount that you check ALL the citations, especially the ones by the same author. These usually give you the 'background' of the research. In addition, be familiar with 'well known' assumptions, and practices as well. They are accepted for a reason.
Second, I have seen an article (peer-reviewed in high-ranking journal) that outright attacked another earlier published work by another author. Even though the attacking article may be correct, it did not exactly contribute anything to the field. On other hand, if you developed a novel methodology that addresses the issue that was presented in that offending publication, you can say, "here's my work on this, and I did this." and then cite that paper, and then say something along like, 'improves the results because their approach did not address this particular issue'.
Third,if you burn bridges and piss off the wrong people by attacking them outright, you will not get very far. It is a fact of life. Learn how to navigate the world with proper conduct and delicate touch (people ARE sensitive), and stay true to yourself.
After the second year of Ph.D. study, I started to learn extensively on how to write highly regarded publications. The first thing that was hammered in me is how to describe the assumptions and limitations of the given research, and give a framework or scope of the given research. My first publication described many of these limitations and scope in detail, but my second and third publications did not because I already established these issues in my first one. So, it is paramount that you check ALL the citations, especially the ones by the same author. These usually give you the 'background' of the research. In addition, be familiar with 'well known' assumptions, and practices as well. They are accepted for a reason.
Second, I have seen an article (peer-reviewed in high-ranking journal) that outright attacked another earlier published work by another author. Even though the attacking article may be correct, it did not exactly contribute anything to the field. On other hand, if you developed a novel methodology that addresses the issue that was presented in that offending publication, you can say, "here's my work on this, and I did this." and then cite that paper, and then say something along like, 'improves the results because their approach did not address this particular issue'.
Third,if you burn bridges and piss off the wrong people by attacking them outright, you will not get very far. It is a fact of life. Learn how to navigate the world with proper conduct and delicate touch (people ARE sensitive), and stay true to yourself.
answered yesterday
Dr. Paul Kenneth Shreeman
385
385
add a comment |
add a comment |
up vote
1
down vote
Some of the available answers allude to the following principle, but none say it directly.
Identifying idiosyncratic flaws of a single article, if published alone, makes your article a satellite of that primary article; the importance of your article cannot possibly exceed the importance of the primary article, but usually the satellite weighs less.
The best way to reach publication-worthiness (in a journal of the same caliber like the primary publication), you should ideally get some results of your own, studying the same problem. That buys you a ticket under which you can not only showcase your own work, but you can also compare it to the state of the field (i.e., the flawed article, and any other relevant prior work that you are aware of) in as much detail as you find intellectually stimulating to a "generic" reader of the journal where you publish this.
The point of comparing your work to previous work is to make your contribution shine in its relevance and methodological superiority, rather than to punish your predecessor or to save your colleagues from falling into believing obviously flawed results. And that is an entirely non-political reason to approach the comparison with utmost indifference and politeness, in a way where many readers (who never encountered the actual earlier article) will not even realize the references to it to have motivated your article. The flawed article isn't enormously important, right? I'd even argue that you should be primarily comparing your new work to the best available results of others, unless certain flaws are propagating over most prior work and your own work stands out (in your eyes) by being more sophisticated and free of them.
The reason why some high profile journals pay more attention to "letters to the editor" than solid "average" journals is that if the primary article is widely seen as a breakthrough, its satellites might still be publication worthy if the critique is deep or if it adds a relevant multi-disciplinary angle, and if their points are really good.
add a comment |
up vote
1
down vote
Some of the available answers allude to the following principle, but none say it directly.
Identifying idiosyncratic flaws of a single article, if published alone, makes your article a satellite of that primary article; the importance of your article cannot possibly exceed the importance of the primary article, but usually the satellite weighs less.
The best way to reach publication-worthiness (in a journal of the same caliber like the primary publication), you should ideally get some results of your own, studying the same problem. That buys you a ticket under which you can not only showcase your own work, but you can also compare it to the state of the field (i.e., the flawed article, and any other relevant prior work that you are aware of) in as much detail as you find intellectually stimulating to a "generic" reader of the journal where you publish this.
The point of comparing your work to previous work is to make your contribution shine in its relevance and methodological superiority, rather than to punish your predecessor or to save your colleagues from falling into believing obviously flawed results. And that is an entirely non-political reason to approach the comparison with utmost indifference and politeness, in a way where many readers (who never encountered the actual earlier article) will not even realize the references to it to have motivated your article. The flawed article isn't enormously important, right? I'd even argue that you should be primarily comparing your new work to the best available results of others, unless certain flaws are propagating over most prior work and your own work stands out (in your eyes) by being more sophisticated and free of them.
The reason why some high profile journals pay more attention to "letters to the editor" than solid "average" journals is that if the primary article is widely seen as a breakthrough, its satellites might still be publication worthy if the critique is deep or if it adds a relevant multi-disciplinary angle, and if their points are really good.
add a comment |
up vote
1
down vote
up vote
1
down vote
Some of the available answers allude to the following principle, but none say it directly.
Identifying idiosyncratic flaws of a single article, if published alone, makes your article a satellite of that primary article; the importance of your article cannot possibly exceed the importance of the primary article, but usually the satellite weighs less.
The best way to reach publication-worthiness (in a journal of the same caliber like the primary publication), you should ideally get some results of your own, studying the same problem. That buys you a ticket under which you can not only showcase your own work, but you can also compare it to the state of the field (i.e., the flawed article, and any other relevant prior work that you are aware of) in as much detail as you find intellectually stimulating to a "generic" reader of the journal where you publish this.
The point of comparing your work to previous work is to make your contribution shine in its relevance and methodological superiority, rather than to punish your predecessor or to save your colleagues from falling into believing obviously flawed results. And that is an entirely non-political reason to approach the comparison with utmost indifference and politeness, in a way where many readers (who never encountered the actual earlier article) will not even realize the references to it to have motivated your article. The flawed article isn't enormously important, right? I'd even argue that you should be primarily comparing your new work to the best available results of others, unless certain flaws are propagating over most prior work and your own work stands out (in your eyes) by being more sophisticated and free of them.
The reason why some high profile journals pay more attention to "letters to the editor" than solid "average" journals is that if the primary article is widely seen as a breakthrough, its satellites might still be publication worthy if the critique is deep or if it adds a relevant multi-disciplinary angle, and if their points are really good.
Some of the available answers allude to the following principle, but none say it directly.
Identifying idiosyncratic flaws of a single article, if published alone, makes your article a satellite of that primary article; the importance of your article cannot possibly exceed the importance of the primary article, but usually the satellite weighs less.
The best way to reach publication-worthiness (in a journal of the same caliber like the primary publication), you should ideally get some results of your own, studying the same problem. That buys you a ticket under which you can not only showcase your own work, but you can also compare it to the state of the field (i.e., the flawed article, and any other relevant prior work that you are aware of) in as much detail as you find intellectually stimulating to a "generic" reader of the journal where you publish this.
The point of comparing your work to previous work is to make your contribution shine in its relevance and methodological superiority, rather than to punish your predecessor or to save your colleagues from falling into believing obviously flawed results. And that is an entirely non-political reason to approach the comparison with utmost indifference and politeness, in a way where many readers (who never encountered the actual earlier article) will not even realize the references to it to have motivated your article. The flawed article isn't enormously important, right? I'd even argue that you should be primarily comparing your new work to the best available results of others, unless certain flaws are propagating over most prior work and your own work stands out (in your eyes) by being more sophisticated and free of them.
The reason why some high profile journals pay more attention to "letters to the editor" than solid "average" journals is that if the primary article is widely seen as a breakthrough, its satellites might still be publication worthy if the critique is deep or if it adds a relevant multi-disciplinary angle, and if their points are really good.
answered 13 hours ago
Jirka Hanika
69147
69147
add a comment |
add a comment |
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2facademia.stackexchange.com%2fquestions%2f120532%2fwhy-dont-i-see-publications-criticising-other-publications%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
42
2-3 decades ago a professor use to publish few research articles in his entire career now I have seen people publishing 10s of articles during PhD. The quality of published article is getting low day by day. There is are multiple reasons but most important reason in my opinion is Publish or perish culture imposed on academia.
– MBK
2 days ago
8
@Trilarion not a very good one imo - there's all sorts of reasons one paper could have fewer citations than another, and I think poor quality is reasonably far down the list
– llama
2 days ago
2
Maybe that's field-dependent, but I see that a lot in political science. A major way to bolster the "relevance" of a publication is to situate it within a debate. Bonus points if you criticize "conventional wisdom". So I'm not.sure about the premise of this question.
– henning
2 days ago
2
Actually there is a lot of published criticism on papers of different fields. See for instance PubPeer.
– Scientist
yesterday
1
Having written such an article myself: they do exist, but getting the balance between constructive criticism and pure negativity right is tricky. Articles that cite our paper also often uncritically cite the paper we are refuting.
– Konrad Rudolph
yesterday