How should we critique research?

It can’t just be that a criticism is boring and provokes eye-rolling—someone who in every genetics discussion from ~2000–2010 harped on statistical power & polygenicity and stated that all these exciting new candidate-gene & gene-environment interaction results were so much hogwash and the entire literature garbage would have been deeply irritating to read, wear out their welcome fast, and have been absolutely right. All of our models are false, but some may be useful, and a good statistical analysis is merely ‘good enough’. It usually doesn’t have to do with predictive power (whether quantified as R2 or AUC or ROC etc); sheer prediction is the goal of a subset of research (although if one could show that a particular choice led to a lower predictive score, that would be a good critique), and in many contexts, the best model is not particularly predictive at all, and a model being too predictive is a red flag.

Source: www.gwern.net