Unicorns were believed to be real to the Greeks, and even used as a metaphor in modern Biblical translations for a horned beast of great strength. Such a creature was believed to have come from India. It was called the Monoceros, a Greek word, which had a horse body with a horn on the center of the forehead, of a certain length.
Jabu trained to be a Saint in Oran, Algeria during the period the other orphans were. The cloth was likely kept in this area by his master to be given to one who was proved to be a Saint. During The Lost Canvas, Yato is the Saint who receives the Unicorn Bronze Cloth. It is not revealed where he got his cloth, having had it since the beginning of the series.