There are two ways of approaching the goal of finding out which principles guide the processing of eval-last-sexp:
looking at the elisp and the underlying C source code fully understanding what it does and how it works in order to derive from that knowledge the underlying principle (if there is actually such there).
observing what happens in various different cases of invoking eval-last-sexp and deduct from the observation the principle able to explain all of the observed cases making their outcome predictable
This answer uses the second approach and explains the observed cases using following test text:
;; "End of file during parsing"
▮
" DEMONSTRATION instead of explanation: "
;;▮ 'Lisp_Reader' is this part of Emacs which generates from the elisp
;;▮ code text objects/symbols/values used in further steps to generate
;;▮ executable code and run it. The 'Lisp_Reader' requires the elisp
;;▮ code text to follow certain rules in order to parse/scan the text
;;▮ into a meaningful result.
;;,``,'',`','`,;; ;; ;; ;; ;; ;; ;; ;; ;; ;; ;; # # @ @ ;;;;;;;;;;▮"▮
?▮ ; <- PROBLEM -> elisp--preceding-sexp: End of file during parsing
; \__ elisp expects that ? is followed by some character
The black rectangles mark the tested positions of the cursor/point in the process of finding out how eval-last-sexp works.
The deducted principle is:
The backward search consists of TWO different phases:
- starting from the cursor position, in the first phase of going back, go character by character skipping characters which are considered not to be a part of an expression like: space, semicolon, comma, back-tick, ...:
;;,``,'',`','`, #, @
don't care going back about the correctness of detection of strings or comments by assuming that the start position is at end of an actual expression and not within a string or a comment.
if in the first phase (which ends with detection of a newline) no character which can belong to an expression has been found, proceed with a second phase and go the next backwards steps no more character by character but line by line parsing each line from its beginning to make sure it is not a comment line.
The above principle (splitting the backward search into two phases) does explain how it comes that positioning the cursor in an empty line below the text or at the end of line containing only "expression-neutral" characters results in detection of the string starting the block of comment lines.
Whether the in elisp and C coded algorithm actually works exactly as described above remains still open, but the described principle makes it at least possible to predict the result depending on cursor position and explains the surprising (when assuming going back only character by character) behavior.
A little bit fun is the Error message elisp raises in case when searching back hits the beginning of the text (or when it starts directly behind a question mark):
"End of file during parsing"
P.S. to make it easier to play around with eval-last-sexp I have bound it to F8:
(global-set-key (kbd "<f8>") 'eval-last-sexp)
elisp--preceding-sexpwhich calls(forward-sexp -1). Trying that manually in the various places you mention shows that it is the one that's doing the heavy lifting. So you only (ahem) have to understandforward-sexp.forward-sexphinges on understandingscan_lists, a 400-line C function insyntax.c. If you look at it, it's probably not too bad, but it is full of low-level finicky details, so it's not going to be a cakewalk. Something to do on a rainy day when you have nothing better to do perhaps... But I doubt that there is much of a principle hidden in there: it's more a hard slog through all the cases.