I know that power analysis is the statistically valid way to ensure you use the
ID: 30666 • Letter: I
Question
I know that power analysis is the statistically valid way to ensure you use the correct numer of samples or repeats in an experiment. But I have never seen any biologist actually conduct a power analysis. Mostly, researchers seems to use a rule of thumb (three technical, three biological replicates is a common one).
Should I be doing a power analysis each time I design an experiment, or can I just use one of the common biology rules of thumb? If not, what are the consequences for the validity of my results? And is there a situation where I will be required to use power analysis, which makes it advantageous to get used to doing it now?
Explanation / Answer
You've already gotten a decent answer to this, but I'll provide my own thoughts on the subject.
Yes
It's necessary. It is absolutely something you should do before beginning an experiment, and preferably something you should do in collaboration with the person who is going to be helping you analyze your data. To address a couple points:
sjcockell is partially correct. To do power analysis, you at least need to have some notion of the effect measure you're likely to see. And these are indeed just estimations of what you'll see. But in nearly all circumstances, you'll likely have some ideas already. Are there similar experiments you can draw from? Your own pilot studies? A "feel" born of experience in your particular system?
It's also trivially easy to calculate power under a number of difference scenarios, to ensure your experiment is sufficiently powered if things go considerably worse than expected. For example, in a study I once did the power calculations for, we weren't sure what the ratio of exposed to unexposed subjects would be.
Which left me with the confidence that even if I was in my "worst case" scenario, I'd have reasonably good power at realistic effect sizes.
That's the true strength of power calculations. They'll tell you things about your study. What you need. What doesn't matter. Whether or not before you spend time and money pursuing an idea if you have a reasonable chance at success. Sit down with someone, take an hour or two (at most for a simple experiment) and do it right. Or ask CrossValidated for advice.