Oddly enough, the test statistic measures something called unusualness. One might not think that unusualness could be measured at all, much less measured exactly, but in fact, not only can it be, but you have probably been measuring it for a while in class without realizing it.
Just as distance can be measured in feet or inches, and those can be converted freely back and forth, unusualness can be measured in both probability units and standard deviation units. When we do hypothesis testing, the typical thing is to calculate a test statistic from our data, which gives us a measure our data's unusualness in units of standard deviation. We can then do one of two things:
First, we can use a z-table, t-table, or chi-squared table to compare our test statistic to a critical value found on the table. If the test statistic is greater than the critical value, we reject the null hypothesis and accept the alternative hypothesis, and conclude that our data is different from whatever the null hypothesis suggests.
Second, we can get a probability either from a computer or from a z-table (it's possible to get a probability from a t-table or chi-squared table too) and convert our standard deviation units to probability units. This probability is called the p-value of our test. We then compare this p-value to our alpha level (typically 0.05) and if the p-value is less than alpha we reject the null hypothesis and accept the alternative hypothesis, and conclude that our data is different from whatever the null hypothesis suggests.
