The lisp printer needs to be set up to print Circular Objects. To do so you need to eval:
(setq print-circle t)
Then the output of eval
(setq x '#1=(a #1#)) will print as
#1=(a #1#) (as expected).
Read more at the Manual
If you want to output and re-read Circular Objects, it would be unwise to unroll them while printing, because this would result in an infinite loop. So there has to be some syntactic expression, for how to print and read Circular Objects.
Within Common Lisp
# often starts a reader macro, which changes the input before the compiler/interpreter sees it. (Some sort of a very sophisticated preprocessor from a C programmers perspective). With that it is possible to transform those expressions into memory objects, i.e. connect last list element with first.
ELisp has no reader macros, but I guess this special case is implemented somewhere in Emacs' C code. Why Elisp does not detect Circular Obejcts by default, I do not know. Probably some legacy and performance reasons.