Short version: the mean is 5.811 ppb after rejecting one outlier, the relative error is +13.1%, and the 95% confidence interval does not contain the accepted value. Sam's procedure is both biased high and too imprecise for the stated spec. Full working below.
(a) Run the outlier check FIRST, then the statistics. Rejecting a point changes every number that follows, so the Q test has to come before you report anything.
Sorted: 5.13, 5.71, 5.72, 5.75, 5.77, 5.79, 5.80, 6.14. Total range w = 6.14 - 5.13 = 1.01 ppb.
Q = (gap to nearest neighbor) / (total range) low 5.13: Q = (5.71 - 5.13) / 1.01 = 0.58 / 1.01 = 0.574 high 6.14: Q = (6.14 - 5.80) / 1.01 = 0.34 / 1.01 = 0.337 Q(crit), n = 8, 95% confidence = 0.526
5.13 gives Q = 0.574, which exceeds 0.526, so it is rejected at the 95% level. 6.14 is retained. State your confidence level, because 5.13 survives at 99%, where Q(crit) = 0.633.
all 8 values 5.13 rejected (n = 7) mean 5.726 ppb 5.811 ppb std deviation s 0.277 ppb 0.149 ppb RSD 4.85% / 48.5 ppt 2.56% / 25.6 ppt range 1.01 ppb 0.43 ppb relative range 17.6% / 176 ppt 7.40% / 74.0 ppt
Report the n = 7 column as your answer and show the n = 8 column beside it, so your reader can see exactly what the rejection bought.
A trap hiding in part (a). If you re-run the Q test on the surviving seven, 6.14 now gives Q = 0.34 / 0.43 = 0.791 against Q(crit) = 0.568 and would also be rejected. Do not do that. Dixon's Q is meant to be applied once. Re-applying it to a set you have already trimmed will keep eating data, because every rejection shrinks the range sitting in the denominator.
(b) Relative error. Accepted value 5.14 ppb.
absolute error = 5.811 - 5.14 = +0.671 ppb relative error = 0.671 / 5.14 = +0.131 = +13.1% = +131 ppt (using all eight: 0.586 / 5.14 = +11.4% = +114 ppt)
The sign is the point. It is positive on every reasonable treatment of the data, so the method reads high. A consistent one-way offset is the fingerprint of systematic error, and it sets up part (d).
(c) 95% confidence limits. CL = mean +/- t * s / sqrt(n), with t at n - 1 = 6 degrees of freedom.
t(95%, df = 6) = 2.447 s / sqrt(n) = 0.1487 / sqrt(7) = 0.0562 ppb t * s / sqrt(n) = 2.447 * 0.0562 = 0.138 ppb CL = 5.811 +/- 0.138 ppb -> 5.67 to 5.95 ppb (all eight, t(95%, df = 7) = 2.365: 5.726 +/- 0.232 = 5.49 to 5.96 ppb)
This is the most important line in the whole problem: 5.14 falls outside that interval, and outside the n = 8 interval too. So the gap between what Sam measures and what Aemon accepts is not random scatter. It is a real, statistically significant bias, which is precisely what a calibration fault produces.
(d) What a calibration error actually touches. A miscalibrated instrument is a systematic (determinate) error: it moves every reading the same way, so it changes where the data sits, not how tightly the data clusters.
AFFECTED mean, relative error, the CENTER of the confidence interval NOT AFFECTED the outlier conclusion - Q depends only on scatter
The precision measures need a finer answer, and this is why the problem made you compute both absolute and relative ones:
additive offset (bad blank / wrong intercept):
s and range unchanged - the offset cancels in every (xi - mean)
RSD, rel. range shift slightly, ONLY because the mean in the
denominator moved
multiplicative error (wrong slope / sensitivity):
s and range scale by the same factor as the readings
RSD, rel. range unchanged - numerator and denominator scale togetherSo "which parameters are affected" has two correct answers depending on the fault, and naming which one you assumed is what separates a good response from a vague one.
(e) Is the data precise? No. Check the unit before anything else: here ppt means parts per thousand, not parts per trillion. An RSD is a ratio, so it cannot be a concentration. A spec of 2.0 ppt is 0.20%.
required RSD = 2.0 ppt = 0.20%
found RSD = 25.6 ppt = 2.56% (n = 7, outlier removed)
RSD = 48.5 ppt = 4.85% (all eight values)The method misses the precision spec by roughly 13-fold even after discarding the worst point, so the data is not precise by this criterion. Note that precision and accuracy fail independently here: fixing the calibration would remove the +13.1% bias from part (b) and would do nothing at all to the RSD. Sam's procedure needs work on both counts.
One last thing, and it is the most interesting feature of this data set. The point the Q test told you to discard, 5.13, sits 0.01 ppb from the accepted value of 5.14. The Q test is a test of internal consistency only - it has no knowledge of the true value, so it will cheerfully reject the one measurement that was right. Never treat outlier rejection as a search for truth; treat it as a rule for handling scatter, always report that you rejected a point and why, and if the rejected point lands on the accepted value, take it as a strong hint that the instrument was intermittently reading correctly while the other seven trials shared a common fault.
