I hope this answer helps:
You're asking about "this phenomenon," but referring to a few different phenomena in your examples, so there are some nuances that are different for different parts of your question.
The short answer is yes, "where" and "there" are not just better understood as adverbs, they are adverbs, even though they're not the kind of words we usually think of as adverbs. They modify verbs, so they are adverbs.
The questions you have about the "to" and "from" phrases are best answered by pointing out that "to where" and "to there" used to be expressed by the words "whither" and "thither," and likewise, "whence" and "thence" for the respective "from" phrases. So "whither do you go?" "To London." "Whence do you come?" "From York." You still see it occasionally if someone is casting out demons or some such and says "return from whence you came!" (even though "from whence" is redundant). So analytically, it's probably clearest to say that we have generally replaced "whither" and "whence" with "where."
I would argue that in the "beside" examples, because they don't involve motion, but rather location, "where" is standing in for the location, and locations are often defined with prepositions. "Where's Jean?" "In Paris." "Where's Juliet?" "On the balcony." "Where's the troll?" "Under the bridge."
The basic idea is that prepositional phrases function adverbially in sentences.
Mary left WITH HER RABBIT.
They threw milkshakes AT THE POLITICIAN.
I looked AROUND.
In all of those cases, the prepositional phrase is modifying the verb, and "where," "when," and "how" all stand in for prepositional phrases.