An alpha level of 0.05 is arbitrary and was set as a standard by scientists. One of the key concepts in hypothesis testing is that of significance level or, the alpha level, which specifies the probability level for the evidence to be an unreasonable estimate. Unreasonable means that the estimate should not have taken its particular value unless some non-chance factor(s) had operated to alter the nature of the sample such that it was no longer representative of the population of interest. (Price, 2000)
As a researcher, you have complete control over the value of this significance level. The alpha level should be considered based on the research context and of the researcher’s personal convictions about how strong they want the evidence to be, before concluding that a particular estimate is reasonable or unreasonable. (Price, 2000) An alpha level of 0.05 is the recommended norm for a two tailed test. The alpha level should be considered based on personal convictions of how strong you want your evidence to be. The alpha level is the probability or p-value that the researcher is willing to accept as significant. It can also be interpreted as the chance of making a Type 1 or Type 2 error. When you set a more stringent (smaller) alpha level, like .01 or .001, (which decreases the probability of making a Type I error) you increase the likelihood of making a Type II error. Hence, it is suggested that an alpha level of .05 is a good compromise between the likelihoods of making Type I and Type II errors.
An experiment where you may want a lower alpha level (e.g., 0.01) would be for example a drug study for coagulation times. You would want to be certain the drug is effective, therefore a lower alpha level would be prudent.
References: Chapter Fourteen Analysis of Variance. (2012). Retrieved from www.ssc.wisc.edu/~aaradill/310_spring2012_chapter14.pdf Crossman, A PediaView.com. (2013). Interaction (Statistics). Retrieved March 25, 2013, from PediaView.com: http://pediaview.com/openpedia/Interaction_(statistics) 5DQ1