
Kevin C. answered 08/10/20
UC Math Grad Specializing in Math and Physics, Energetic and Fun
Hi Amir!
A 5% increase in a data set is not necessarily statistically significant or insignificant. 5% is often used for the significance level though which may be confusing. Whether or not a 5% increase is statistically significant has to do with how much the data naturally varies (the standard deviation) and what significance level you are testing. Statistical significance has to do with deciding what probability one decides to set as a tolerance for rejecting the hypothesis that newly observed data does not lead to a new conclusion via rejection of the null hypothesis (see link https://en.wikipedia.org/wiki/Statistical_significance) I will demonstrate with an example:
Let's say that we are looking at the stock price of a company. We can start by assuming that the price behaves as a random variable with a normal distribution for statistical convenience. We might want to test whether the price of the stock has increased by a statistically significant amount from its 6-month average which we will say is $20. What we are looking at is whether newly observed prices indicate an actual change in the average (or 'true' price of the stock), or whether it can be accounted for by random fluctuations in stock price. Let's say that over this time, the standard deviation of the stock price is $2. For this stock, a 5% increase in price would be $1, raising the price to $21 but, if we decided we want to test to a 5% significance level, we need a price increase that has a 5% probability or less of occurring given the assumption that the 6-month average is to remain at $20- we would need the increase to be to $23.92 calculated via the use of a z-score and table. Thus in this case, a 5% increase is not statistically significant. Alternatively, if the stock price experiences much less variability and thus has a standard deviation of $0.5, then our observed 5% increase in the stock price to $21 would be two standard deviations away and thus be statistically significant to a 5% significance level.
Let me know if you have any follow-up questions.
~Kevin