One commonly used statistics package is called R. The upside is that it is free and that it is incredibly versatile and can handle even very complex statistics. The downside is that it can be a little dependent on writing out syntax and code... though this is definitely less of a problem with the simple statistics. You can find out more here: http://www.r-project.org/
Again R can "feel" a bit strange at first but there are a number of good sites that can help you get the right code down for the simpler statistical tests. Here is one support site that I have referred to a good deal myself: http://www.statmethods.net/
Finally, if you have access to Microsoft Excel you might be surprised to find out that it can compute various statistics like chi-square, t-tests, F-tests, variance, standard deviations, means, and even more. You can find all of the options under formulas -> More options -> Statistical
I hope those options help.