Star Hunter by Andre Norton

It was some time before Hume found what he wanted, an islet in midstream lacking any growth and rising to a rough pinnacle. The sides were seamed with crevices and caves which promised protection for one's back in any desperate struggle. And they had discovered it none too soon, for the late afternoon shadows were lengthening.

There had been no attack, just the trailing to herd the men to the northeast. And Rynch had lost the first tight pinch of panic, though he knew the folly of underestimating the unknown.