There are a number of endangered languages with only a few speakers left. Livonian, for example, the language of indigenous Latvians, has perhaps 6 native speakers and 20 others.
Presumably the least spoken language in the world would be a language which is known by no-one whatsoever. I suggest that that would be the language of the Neanderthals, since not only does no-one currently know it, it is likely that no modern languages are even descended from it.
