How do you convert a vector of float numbers to bytes?

Question

I need to convert a vector of float numbers to bytes. I can't seem to find a way to do that in emacs-lisp. Is it possible?

For example how do I convert this vector [3.14] to bytes?

[Update] See this comment by @dalanicolai, especially for the function IEEE-float-2-hex.

Then, see https://stackoverflow.com/questions/12003231/how-do-i-convert-a-string-of-hex-into-ascii-using-elisp for how to convert a hex string to chars.

With those two functions I can run this:

(cl-loop for char across (reverse (decode-hex-string (IEEE-float-2-hex 3.14)))
       concat (byte-to-string char))

to get a byte string (the original question), or this to write it to a file.

(with-temp-file "bin3"
  (cl-loop for char across (reverse (decode-hex-string (IEEE-float-2-hex 3.14)))
       do (insert (byte-to-string char))))

The reverse here seems related to big vs little endian. Otherwise, that seems to write a byte string to a file that represents 3.14. This python can read it correctly. I still don't know how to read it in to emacs-lisp, but that isn't something I need at the moment (I just would like to know).

#+BEGIN_SRC jupyter-python
import struct
with open('bin3', 'rb') as f:
    print(struct.unpack('f', f.read()))
#+END_SRC

#+RESULTS:
:RESULTS:
(3.1399998664855957,)
:END:

The small difference from 3.14 is from not being able to exactly represent a float number in 32 bits.

Answer 1

Finally, I have also found this message by Ehud Karni on the mailing list. Indeed that code also gives you the correct numbers in hexadecimal. I will paste it here (it does not work when directly copy pasting from the mailing list message)

(defvar IEEE-sign-mlt 128 "multiplier for sign")                       ;  1 bits
(defvar IEEE-exp-bias 127 "exponent bias for float")                    ;  8 bits
(defvar IEEE-mantissa-divisor ?\x800000 "mantissa divisor for float")  ; 23 bits
(defvar IEEE-float-mantissa-divisor (/ 1.0 (float IEEE-mantissa-divisor))
  "floated 1 / mantissa divisor")


(defun IEEE-float-2-hex (FNUM)
  "Convert a floating point number FNUM to 8 Hex digits string (big endian)"
  (interactive "nEnter float to convert: ")
  (or (floatp FNUM)
      (setq FNUM (float FNUM)))
  (let ((IEEE-sign 0)
        (IEEE-exp  0)
        (IEEE-mantissa 0)
        hex)
    (and (< FNUM 0)                         ; negative
         (setq IEEE-sign IEEE-sign-mlt)      ; yes, sign=1 (* 2**7)
         (setq FNUM (- FNUM)))               ; negate (abs) it
    (cond
     ((= FNUM 0))                        ; Zero - we all set
     ((= FNUM 1e+INF)                    ; infinite ?
      (setq IEEE-exp IEEE-exp-bias))  ; exp = max, mantissa = 0

     (t                                  ;; real float
      (setq IEEE-exp (floor (log FNUM 2.0)))      ; exponent [ log 2 FNUM ] - integer
     (if (<= IEEE-exp (- IEEE-exp-bias))         ; check for De-normalized number
         (setq IEEE-exp 0                        ; really - 
           IEEE-exp-bias
               IEEE-mantissa (truncate (* IEEE-mantissa-divisor
                      0.5 FNUM (expt 2.0 IEEE-exp-bias))))
         ;; normal float
         (setq IEEE-mantissa (truncate (* IEEE-mantissa-divisor
                      (- (/ FNUM (expt 2.0 IEEE-exp)) 1))))
         (setq IEEE-exp (+ IEEE-exp IEEE-exp-bias))) ;; offset-ed exp
     ))
  (setq hex (format "%02X%06X" (+ IEEE-sign (/ IEEE-exp 2))
                    (+ IEEE-mantissa
                       (* (% IEEE-exp 2) 
              IEEE-mantissa-divisor))))

  ;;         (message "Float=%f sign=%d, exp=%d, mant=%d  HEX=%s"
  ;;                  FNUM IEEE-sign IEEE-exp IEEE-mantissa hex)

  (print hex)))

(defun IEEE-hex-2-float (HEX)
  "Convert an 8 Hex digits string (big endian) to floating point number"
  (interactive "sEnter hex value (8 hex digits): ")
  (if (or (not (stringp HEX))
          (/= (length HEX) 8)
          (string-match "[^0-9a-fA-F]" HEX))
      (error "Arg must be a string of EXACTLY 8 hex (1-8, a-f, A-F) 
digits"))

  (let ((S-EXP (string-to-number (substring HEX 0 2) 16))
        (MANTISSA (string-to-number (substring HEX 2) 16))
        (RSLT 1)
        IEEE-sign
        IEEE-exp
        IEEE-mantissa
        )
    (setq IEEE-mantissa (logior MANTISSA IEEE-mantissa-divisor)) ; always set upper bit
    (if (= IEEE-mantissa IEEE-mantissa-divisor)             ; = --> zero mantissa, check special cases
        (cond
         ((string-equal HEX "00000000")
          (setq RSLT (float 0)))      ; Zero
         ((string-equal HEX "3F800000")
          (setq RSLT  1e+INF))        ; + infinity
         ((string-equal HEX "BF800000")
          (setq RSLT -1e+INF))        ; - infinity
         ))
    (print (if (/= RSLT 1)
               RSLT                                            ; special cases (0, infinity)

         (setq IEEE-sign (ash S-EXP -7))                     ; shift right 7 bits
         (setq IEEE-exp (+ (ash (% S-EXP IEEE-sign-mlt) 1)   ; shift (part) exp 1
                               (ash MANTISSA -23)))              ; 1 bit from mantissa
         (if (= IEEE-exp 0)                                  ; De-normalized ?
         (setq IEEE-mantissa (ash
                      (logxor IEEE-mantissa IEEE-mantissa-divisor) 1))) ; clear upper bit
         ;;             (message "Hex=%s sign=%d, exp=%d, mant=%d"
         ;;                      HEX IEEE-sign IEEE-exp IEEE-mantissa)

         (*                                                  ; result float
          (- 1 IEEE-sign IEEE-sign)                       ; sign
          (expt 2.0 (- IEEE-exp IEEE-exp-bias))           ; 2 ** exponent (un-biased)
          (float IEEE-mantissa)                           ; mantissa part
          IEEE-float-mantissa-divisor                     ; 1 / mantissa divisor
          )))))

You will get the bytes in hexadecimal representation as a string. Unfortunately I don't know how to 'convert' those strings to hexadecimal numbers (starting with #x), but if you find a way to convert the hexadecimal strings to numbers, then you could use this solution also.

Answer 2

You can't. Emacs Lisp is a highly abstract, strongly typed dynamic language where everything is a reference. It determines the data type through the highest bits of pointers in C language. If you want to treat a floating point number as an integer, you have to modify some bits of the pointer. This can't be done in Emacs Lisp; you must modify it through a debugger like GDB.

But you can write a function to find out the precise representation of a floating point number. As per the elisp manual, Emacs Lisp follows IEEE-754.

Answer 3

EDIT

I quickly checked the result with the result given here, but now I see that I did not scroll till the end. It seems that the code in the answer returns the 'non-normalized' binary representation (that page b.t.w. very explicitly provides the recipe for how to obtain the 32-bit IEEE 754 representation and this page for the 64-bit version).

END EDIT

From this answer I found that calc can do this.

Then looking a little into the calc internals we find that we can achieve it as follows:

(mapcar (lambda (f)
          (math-format-radix-float
            (math-read-number (number-to-string f)) 12))
        [3.14])

use cl-mapcar or vconcat if you prefer to get a vector.

Indeed, I guess, this currently is the only (provided) way to achieve it with Emacs.

I have seen your posts about using dynamic modules with Emacs. Of course in C, obtaining the binary representation of some number is quite 'straightforward' (also recipes can easily be found). So, if you'd like a more 'transparent' way of doing things, then I would say simply add a function for it to some dynamic module.

Answer 4

If you already have a python solution then you can use python to achieve what you want:

Create a python file that reads from stdin and prints to stdout as follows:

#!/usr/bin/env python

import sys
from struct import pack

for line in sys.stdin:
    sys.stdout.write(''.join([f'{b:d} ' for b in pack('f', float(line))]) + '\n')

save it with the name float_to_bytes, make it executable and make it available on your PATH (or use the exact path below).

Then in Emacs try the following code to send the vector to the python script and read back in its output to byte-strings:

(with-current-buffer (pop-to-buffer "ftb")
  (with-temp-buffer
    (seq-do (lambda (f) (insert (number-to-string f) "\n")) [3.14])
    (shell-command-on-region (point-min) (point-max) "float_to_bytes"
                 "ftb"))
  (let (byte-strings)
    (while (not (eobp))
      (push (mapconcat #'byte-to-string (read (concat "(" (thing-at-point 'line) ")"))) byte-strings)
      (forward-line))
    (vconcat (nreverse byte-strings))))

You can find the vector in the seq-do form. This code ends up in the 'output' buffer, and it returns the vector of byte-strings. You can tweak things to your likings.

I will keep my other answer, as, although it does not answer the question, I still find it informative.

I guess a cleaner way to read in the lines is used by process-lines, you can use it as an inspiration for your final solution (make it use call-process-region instead of call-process).