If you get 13.6-14.6 volts, the alternator is charging and is not the problem.
If you get less than that, ie 12 volts or less, your alternator is no bueno.
Also check the voltage on your "dead" battery. You'd typically need at least 12.6 volts to start a car.
Here's a handy chart to show what the power level on a 12v battery is when measuring it with a multimeter.
100% 12.70+
75% 12.40
50% 12.20
25% 12.00
0% 11.80
If it's below 11v, that battery is likely completely toast and you're better off buying a new one.