Slight comes from Old Norse sléttr meaning 'smooth, flat'. Something smooth was seen as plain, and something plain was seen as trivial — giving English a word for smallness that began as a word for flatness.
Small in degree; inconsiderable; not thorough or detailed; thin or slim in build.
From Middle English slight, from Old Norse sléttr meaning 'smooth, flat, level', from Proto-Germanic *slihtaz meaning 'flat, smooth, simple'. The original meaning was physical: a slight surface was a smooth, level one. The shift to 'small, insignificant' followed the logic that something smooth is plain, and something plain is trivial. The verb 'to slight' (to treat with disrespect) emerged in the 16th century from the idea of smoothing something away — treating it as flat, featureless, beneath notice
Slight and sleight (as in 'sleight of hand') look like they should be the same word, but they have different origins. Slight comes from Old Norse sléttr ('smooth'), while sleight comes from Old Norse slœgð ('cunning, skill'). A slight is something trivially small; a sleight is something cleverly done. Their similarity is a coincidence of spelling, not of meaning.