I'm trying to lose weight, wouldn't drinking milk in a sense minimise the amount of weight I lose since it adds on fat? Say for example I exercise for an hour and then drink a glass later on, wouldn't that glass of milk add on fat and hence spoil all that exercise I done?

It depends. If you are drinking milk in excess of what is reasonable within a healthy diet, then it could hinder. But, seriously, if you eat too much of anything that will happen. Milk is still healthier than most processed foods for you! Drink away. :)

Stay away from dairy when you're dieting. Milk is high in calories and fat. Since I love milk, when I'm dieting I drink basically "fake milk" that tastes identical to milk. It's called Hood Calorie Counter dairy beverage and it's in a greenish carton. 35 calories per serving, whereas real milk is around 140 calories per serving. Drink lots of water when trying to lose weight, and eat lots of veggies and lean meats. With exercise, you'll drop weight like nobody's business.

if you drink milk it will help you keep the muscles you worked out so yeah 1 glass wont hurt but only if you worked out your muscles