## Revised evidence for statistical standards

**W**e just submitted a letter to PNAS with Andrew Gelman last week, in reaction to Val Johnson’s recent paper “Revised standards for statistical evidence”, essentially summing up our earlier comments within 500 words. Actually, we wrote one draft each! In particular, Andrew came up with the (neat) rhetorical idea of alternative Ronald Fishers living in parallel universes who had each set a different significance reference level and for whom alternative Val Johnsons would rise and propose a modification of the corresponding Fisher’s level. For which I made the above graph, left out of the letter and its 500 words. It relates “the old z” and “the new z”, meaning the boundaries of the rejection zones when, for each golden dot, the “old z” is the previous “new z” and “the new z” is Johnson’s transform. We even figured out that Val’s transform was bringing the significance down by a factor of 10 in a large range of values. As an aside, we also wondered why most of the supplementary material was spent on deriving UMPBTs for specific (formal) problems when the goal of the paper sounded much more global…

**A**s I am aware we are not the only ones to have submitted a letter about Johnson’s proposal, I am quite curious at the reception we will get from the editor! (Although I have to point out that all of my earlier submissions of letters to to PNAS got accepted.)

March 15, 2014 at 10:45 am

[…] X presents our discussion on his blog here. […]

December 19, 2013 at 8:43 pm

As the implications of Johnson’s suggestion are so consequential his paper deserves to be considered from many points of view. A couple of other responses to it have been published on an Australian academic news site (each is critical of the paper from a different perspective).

https://theconversation.com/the-problem-with-p-values-how-significant-are-they-really-20029

http://theconversation.com/give-p-a-chance-significance-testing-is-misunderstood-20207