I was really annoyed about this string natural sorting order so I took quite some time to investigate this issue. I hope this helps.
localeCompare() character support is badass, just use it.
As pointed out by Shog9, the answer to your question is:
There are quite a bunch of custom implementations out there, trying to do string comparison more precisely called "natural string sort order"
When "playing" with these implementations, I always noticed some strange "natural sorting order" choice, or rather mistakes (or omissions in the best cases).
Typically, special characters (space, dash, ampersand, brackets, and so on) are not processed correctly.
You will then find them appearing mixed up in different places, typically that could be:
some will be between the uppercase 'Z' and the lowercase 'a'
When one would have expected special characters to all be "grouped" together in one place, except for the space special character maybe (which would always be the first character). That is, either all before numbers, or all between numbers and letters (lowercase & uppercase being "together" one after another), or all after letters.
My conclusion is that they all fail to provide a consistent order when I start adding barely unusual characters (ie. characters with diacritics or charcters such as dash, exclamation mark and so on).
Research on the custom implementations:
Natural Compare Lite
Research on the browser-native implementations:
http://jsbin.com/beboroyifomu/2/edit?js,console - basic latin characters in string comparison : consistency check in string vs when a character is alone
Thanks to Shog9's nice answer, which put me in the "right" direction I believe