Here's a bit of historical reference (I haven't been born yet when the described events took place, so maybe someone more knowledgeable will correct me. All this is from reading old articles and some books).
Having dispensed with the disclaimer, it seems like back in the days of Fortran vs Lisp "symbolic" was a kind of a buzzword as "object oriented" is today. I.e. programs were typically seen as just huge mathematical formulae where numbers will eventually be plugged and the placeholders for numbers were immaterial. All symbolic information contained in a program would vanish as soon as that program was either run, compiled or interpreted. The novelty of Lisp was that it allowed symbols to persist in a program even after it is run, compiled or interpreted. This inspired such terminology as "symbolic algebra" (as in manipulation on algebraic formulae as done on paper / blackboard rather than by direct calculation). In order to support this (and other symbolic things) symbols were to be equipped with a name and some properties. From a non-symbolic point of view, one might say that "symbols are just named pointers", and while this isn't true, if anything they are more of pointers to structs, but for practical purposes, symbols are designators of the left hand side of a variable-value pair. This also makes it possible to see
symbol-value function as pointer dereferencing in non-symbolic languages.
Modern Lisps vary in that how much values can be associated with one symbol (suppose you had a non-symbolic language with multiple memory stacks / heaps, you could imagine a situation where the same pointer has meaning when interpreted in the context of different stacks / heaps). So, Lisp2 languages (Emacs Lisp being one such language) have separate storage for functions and variables, this is why there's also a
symbol-function, which "dereferences a pointer pointing to a function storage". Scheme doesn't have this special storage and Clojure AFAIK, doesn't have neither that nor