Pariah Planet by Murray Leinster

It seemed that the smell of hunger was in the air. The armed men were cadaverous. Lights came on, and stark, harsh shadows lay black upon the ground. Calhoun's captors were uniformed, but the uniforms hung loosely upon them. Where the lights struck upon their faces, their cheeks were hollow. They were emaciated. And there were the splotches of pigment of which Calhoun had heard. The leader of the truculent group was blue, except for two fingers which in the glaring illumination seemed whiter than white.

"Out! " said that man savagely. "We're taking over your stock of food. You'll get your share of it, like everybody else, but -- out!"