Dwarves is an english word, the popularized and modernized version of dwarfs thanks to J.R.R. Tolkien. "Dwarfs" generally refer to the genetically shorter people affected with dwarfism; whereas "dwarves" typically refers to the Dwarf race of fantasy and mythology.
It's true that in early Norse mythological tales there is little mention mention of dwarves being short as far as I know; except in the fornaldarsagas where dwarves are indeed desribed as smaller and uglier than the average person.
In the Prose Edda, it is told that dwarves were like maggots that lived in the flesh of Ymir before the gods gave them reasion, so perhaps the size stigma came from the idea of maggots, because they are so small. The Poetic Edda gives a different story (see the poem Voluspa).
Alternatively, it is proposed that the whole 'dwarves are small ugly old bearded men' kinda thing stemmed from Christianisation, because sometimes they took things in mythology a little too literally!
Dwarfs vs. dwarves
Dwarfs is the standard plural of the noun dwarf. Dwarves is a newer variant popularized (though not invented) by English author J.R.R. Tolkien in his fantasy fiction works, including The Hobbit and The Lord of the Rings. The Tolkien spelling is appropriate when referring to little people in fantasy worlds. Dwarfs is better everywhere else.
I support name.less, though I am not the best at spelling (this fact will most likely be made obvious) I have seen "dwarves" regarding the mythological beings spelled that way my entire life. Not to mention the English language is constantly changing. For example based on location you could spell the word for the color gray, grey and it world mean the same thing. Regardless of the spelling it gets the point across. But there is no reason to argue about it any further.
Also I find the fact about stature interesting.
