For a two-tailed test, the critical values are the values associated with left and right tail probabilities of alpha/2 = 0.005.
Since the t and standard normal (z) distributions are symmetric, the critical values are +/- the value associated with the right tail probability of 0.005.
For example, for a test which utilizes the standard normal distribution (z-scores), the critical values would be +/- the z-score associated with probability cumulative probability to the left of 0.995 (right tail 0.005 probability) = +/- 2.58.