I don't know that there's a reason ever been given, but it would be the row that sounds the most meaningless and like someone's tongue is being repressed.

While 'r'/'l' is a very common consonant for onomatopoeia, grammatical elements, and verb conjugation, few words start with it and the ones that do largely have foreign roots. They sound at once tongue-tying, slippery, luxurious, and flowery. They also live in an "lesser known" mind-space like our "wxyz" letters are at the end of the alphabet and a bit rare in usage (though "ri" is commonly understood as "reason, logic, principle").

But by then we're venturing into the psychology of "this is how sounds feel", which can be quite subjective (e.g. perhaps you don't feel that "mor" feels "morbid" like Tolkien's Mordor, instead identifying it with words like "more" and "moral"). Still, one can find sites in Japanese (particularly for parents trying to pick euphonic baby names) that talk about this topic. Search terms like 語感, 音のイメージ bring up some of this content. Even 音の印象 brings up the famous kiki/bouba experiment.