Sometimes, the seemingly strong are the ones who feel the weakest inside.
The seemingly harmless comments that are just passed off, have the potential to build up a wall of hurt. Knowing this, why not use a compliment as the driving force that knocks this wall down? The words you say, or even a simple act of kindness may just be the thing that encourages a person. It may make them realize that life isn't as bad as it seems. 
Think about it.... Make a difference.