The formula
t = (x̄₁ − x̄₂) / √(s₁²/n₁ + s₂²/n₂)
What it means
The t statistic is the difference between the means divided by the error of that difference: it measures how many standard errors separate the two groups. A large t means the difference is hard to put down to sampling chance. The p-value puts a number on that difficulty: it is the probability of seeing a difference at least this large if the two groups were really the same.
How to work it out by hand
- Note the mean, the deviation and the size of each group
- Work out the error of the difference: the root of the sum of each variance over its size
- Divide the difference of means by that error: that is the t statistic
- Look up the p-value with the matching degrees of freedom
What is worth knowing
A p-value below 0.05 does not mean the difference is important, only that it is hard to attribute to chance; with huge samples, irrelevant differences come out significant. And the other way round: a high p does not prove the groups are equal, only that there is not enough evidence. Welch's correction, applied here by default, avoids assuming the two variances are equal, and that assumption when false gives optimistic p-values.