Compliment or no?

I know this is just me but when people say something like "oh you've lost a lot of weight" or something like that, I never know whether or not to take it as a compliment. I know it's meant to be, but to me sometimes it feels like people are saying, well you were fat but now, you're not quite so fat. I really don't know why I always think that, but I do. Maybe its because I have a thing about my weight and hate people taking about it and have done ever since I can remember. It's just one of those things I have to deal with.