# How do you convert a vector of float numbers to bytes?

I need to convert a vector of float numbers to bytes. I can't seem to find a way to do that in emacs-lisp. Is it possible?

For example how do I convert this vector `[3.14]` to bytes?

[Update] See this comment by @dalanicolai, especially for the function `IEEE-float-2-hex`.

Then, see https://stackoverflow.com/questions/12003231/how-do-i-convert-a-string-of-hex-into-ascii-using-elisp for how to convert a hex string to chars.

With those two functions I can run this:

``````(cl-loop for char across (reverse (decode-hex-string (IEEE-float-2-hex 3.14)))
concat (byte-to-string char))
``````

to get a byte string (the original question), or this to write it to a file.

``````(with-temp-file "bin3"
(cl-loop for char across (reverse (decode-hex-string (IEEE-float-2-hex 3.14)))
do (insert (byte-to-string char))))
``````

The `reverse` here seems related to big vs little endian. Otherwise, that seems to write a byte string to a file that represents 3.14. This python can read it correctly. I still don't know how to read it in to emacs-lisp, but that isn't something I need at the moment (I just would like to know).

``````#+BEGIN_SRC jupyter-python
import struct
with open('bin3', 'rb') as f:
print(struct.unpack('f', f.read()))
#+END_SRC

#+RESULTS:
:RESULTS:
(3.1399998664855957,)
:END:
``````

The small difference from 3.14 is from not being able to exactly represent a float number in 32 bits.

• This looks like an X-Y question: what are you really trying to do? Commented Jan 2 at 1:45
• I am trying to get raw bytes of a vector of floats as a string so I can write them to a database. Commented Jan 2 at 1:53
• Why can't you write the floating point number as a string directly? Why do you need the bytes? Commented Jan 2 at 2:05
• I am trying to use a vector search extension in sqlite (github.com/asg017/sqlite-vss) that is expecting raw byte strings. It says it can also use json arrays, but now I want to know how to do this with raw-byte strings too. Commented Jan 2 at 3:21
• It seems that you are also mistaken about the vector implementation (and concepts like variables, values, references, etc.). Otherwise, you would ask how to convert the floating point number `3.14` (instead of `[3.14]`) to raw-byte string. Commented Jan 2 at 5:30

## 4 Answers

Finally, I have also found this message by Ehud Karni on the mailing list. Indeed that code also gives you the correct numbers in hexadecimal. I will paste it here (it does not work when directly copy pasting from the mailing list message)

``````(defvar IEEE-sign-mlt 128 "multiplier for sign")                       ;  1 bits
(defvar IEEE-exp-bias 127 "exponent bias for float")                    ;  8 bits
(defvar IEEE-mantissa-divisor ?\x800000 "mantissa divisor for float")  ; 23 bits
(defvar IEEE-float-mantissa-divisor (/ 1.0 (float IEEE-mantissa-divisor))
"floated 1 / mantissa divisor")

(defun IEEE-float-2-hex (FNUM)
"Convert a floating point number FNUM to 8 Hex digits string (big endian)"
(interactive "nEnter float to convert: ")
(or (floatp FNUM)
(setq FNUM (float FNUM)))
(let ((IEEE-sign 0)
(IEEE-exp  0)
(IEEE-mantissa 0)
hex)
(and (< FNUM 0)                         ; negative
(setq IEEE-sign IEEE-sign-mlt)      ; yes, sign=1 (* 2**7)
(setq FNUM (- FNUM)))               ; negate (abs) it
(cond
((= FNUM 0))                        ; Zero - we all set
((= FNUM 1e+INF)                    ; infinite ?
(setq IEEE-exp IEEE-exp-bias))  ; exp = max, mantissa = 0

(t                                  ;; real float
(setq IEEE-exp (floor (log FNUM 2.0)))      ; exponent [ log 2 FNUM ] - integer
(if (<= IEEE-exp (- IEEE-exp-bias))         ; check for De-normalized number
(setq IEEE-exp 0                        ; really -
IEEE-exp-bias
IEEE-mantissa (truncate (* IEEE-mantissa-divisor
0.5 FNUM (expt 2.0 IEEE-exp-bias))))
;; normal float
(setq IEEE-mantissa (truncate (* IEEE-mantissa-divisor
(- (/ FNUM (expt 2.0 IEEE-exp)) 1))))
(setq IEEE-exp (+ IEEE-exp IEEE-exp-bias))) ;; offset-ed exp
))
(setq hex (format "%02X%06X" (+ IEEE-sign (/ IEEE-exp 2))
(+ IEEE-mantissa
(* (% IEEE-exp 2)
IEEE-mantissa-divisor))))

;;         (message "Float=%f sign=%d, exp=%d, mant=%d  HEX=%s"
;;                  FNUM IEEE-sign IEEE-exp IEEE-mantissa hex)

(print hex)))

(defun IEEE-hex-2-float (HEX)
"Convert an 8 Hex digits string (big endian) to floating point number"
(interactive "sEnter hex value (8 hex digits): ")
(if (or (not (stringp HEX))
(/= (length HEX) 8)
(string-match "[^0-9a-fA-F]" HEX))
(error "Arg must be a string of EXACTLY 8 hex (1-8, a-f, A-F)
digits"))

(let ((S-EXP (string-to-number (substring HEX 0 2) 16))
(MANTISSA (string-to-number (substring HEX 2) 16))
(RSLT 1)
IEEE-sign
IEEE-exp
IEEE-mantissa
)
(setq IEEE-mantissa (logior MANTISSA IEEE-mantissa-divisor)) ; always set upper bit
(if (= IEEE-mantissa IEEE-mantissa-divisor)             ; = --> zero mantissa, check special cases
(cond
((string-equal HEX "00000000")
(setq RSLT (float 0)))      ; Zero
((string-equal HEX "3F800000")
(setq RSLT  1e+INF))        ; + infinity
((string-equal HEX "BF800000")
(setq RSLT -1e+INF))        ; - infinity
))
(print (if (/= RSLT 1)
RSLT                                            ; special cases (0, infinity)

(setq IEEE-sign (ash S-EXP -7))                     ; shift right 7 bits
(setq IEEE-exp (+ (ash (% S-EXP IEEE-sign-mlt) 1)   ; shift (part) exp 1
(ash MANTISSA -23)))              ; 1 bit from mantissa
(if (= IEEE-exp 0)                                  ; De-normalized ?
(setq IEEE-mantissa (ash
(logxor IEEE-mantissa IEEE-mantissa-divisor) 1))) ; clear upper bit
;;             (message "Hex=%s sign=%d, exp=%d, mant=%d"
;;                      HEX IEEE-sign IEEE-exp IEEE-mantissa)

(*                                                  ; result float
(- 1 IEEE-sign IEEE-sign)                       ; sign
(expt 2.0 (- IEEE-exp IEEE-exp-bias))           ; 2 ** exponent (un-biased)
(float IEEE-mantissa)                           ; mantissa part
IEEE-float-mantissa-divisor                     ; 1 / mantissa divisor
)))))
``````

You will get the bytes in hexadecimal representation as a string. Unfortunately I don't know how to 'convert' those strings to hexadecimal numbers (starting with `#x`), but if you find a way to convert the hexadecimal strings to numbers, then you could use this solution also.

• I updated the question with some additional details that finish this up. Thanks for the tips! Commented Jan 2 at 13:44
• That's a nice 'round-up' of the answer/question. Of course the `IEEE-float-2-hex` function also switches from number to string representation, it would be nice to just modify that function directly to make it return numbers instead of the hex string. But indeed it is probably not worth the extra time if you have the working solution already. Commented Jan 2 at 14:35

You can't. Emacs Lisp is a highly abstract, strongly typed dynamic language where everything is a reference. It determines the data type through the highest bits of pointers in C language. If you want to treat a floating point number as an integer, you have to modify some bits of the pointer. This can't be done in Emacs Lisp; you must modify it through a debugger like GDB.

But you can write a function to find out the precise representation of a floating point number. As per the elisp manual, Emacs Lisp follows IEEE-754.

• I'll disagree with the "strongly typed" part, but I believe the rest is spot-on. Commented Jan 2 at 1:44
• I would also have said elisp data is strongly typed. My understanding (which I am not claiming is necessarily correct) is that each datum has a single definite and unchangeable type. Each variable "name" is a symbol, with a reference to some such datum in its "variable slot", which slot is itself untyped (typeless?) and which can refer to data of different types at different times -- dynamic typing. Is this not correct? Commented Jan 2 at 18:41
• I believe Emacs Lisp is strongly typed and dynamical typing. The former means that data will not change its own type, nor will implicit type conversions occur; the latter means that the type of the value is only known when dereferencing (that is, when looking at the so-called "variable slot"). We need’t discuss whether the "variable slot" is typeless or not, as it is transparent to the programmer. Commented Jan 2 at 20:08

EDIT

I quickly checked the result with the result given here, but now I see that I did not scroll till the end. It seems that the code in the answer returns the 'non-normalized' binary representation (that page b.t.w. very explicitly provides the recipe for how to obtain the 32-bit IEEE 754 representation and this page for the 64-bit version).

END EDIT

From this answer I found that `calc` can do this.

Then looking a little into the calc internals we find that we can achieve it as follows:

``````(mapcar (lambda (f)
(math-format-radix-float
(math-read-number (number-to-string f)) 12))
[3.14])
``````

use `cl-mapcar` or `vconcat` if you prefer to get a vector.

Indeed, I guess, this currently is the only (provided) way to achieve it with Emacs.

I have seen your posts about using dynamic modules with Emacs. Of course in C, obtaining the binary representation of some number is quite 'straightforward' (also recipes can easily be found). So, if you'd like a more 'transparent' way of doing things, then I would say simply add a function for it to some dynamic module.

• this code does not work for me. I get "Symbol’s function definition is void: math-format-radix-float". That function does not seem to exist in Emacs (ver 30 for me). Commented Jan 1 at 21:30
• I forgot to mention that the function is found in the `calc-bin` 'library' (on Emacs 29) :) Commented Jan 1 at 21:41
• hm. that just outputs ("3.14") for me. That doesn't seem like the answer. I am expecting something that looks more like "\x1f\x85\xebQ\xb8\x1e\t@" Commented Jan 1 at 21:51
• Ah, I did not realize that I had to configure the radix first using `M-x calc-binary-radix` (or `M-x calc-hex-radix` if you prefer). Calc and its documentation is somewhat of a black box to me. Commented Jan 1 at 22:05
• I see. If I run M-x calc, then M-x calc-binary-radix I get your result. It still doesn't seem like the raw bytes I am looking for. I want something that is basically equivalent to the output of this in Python `from struct import pack; pack('f', 3.14)` which looks like `b'\xc3\xf5H@'`. Commented Jan 1 at 22:18

If you already have a python solution then you can use python to achieve what you want:

Create a python file that reads from stdin and prints to stdout as follows:

``````#!/usr/bin/env python

import sys
from struct import pack

for line in sys.stdin:
sys.stdout.write(''.join([f'{b:d} ' for b in pack('f', float(line))]) + '\n')
``````

save it with the name `float_to_bytes`, make it executable and make it available on your PATH (or use the exact path below).

Then in Emacs try the following code to send the vector to the python script and read back in its output to byte-strings:

``````(with-current-buffer (pop-to-buffer "ftb")
(with-temp-buffer
(seq-do (lambda (f) (insert (number-to-string f) "\n")) [3.14])
(shell-command-on-region (point-min) (point-max) "float_to_bytes"
"ftb"))
(let (byte-strings)
(while (not (eobp))
(push (mapconcat #'byte-to-string (read (concat "(" (thing-at-point 'line) ")"))) byte-strings)
(forward-line))
(vconcat (nreverse byte-strings))))
``````

You can find the vector in the `seq-do` form. This code ends up in the 'output' buffer, and it returns the vector of byte-strings. You can tweak things to your likings.

I will keep my other answer, as, although it does not answer the question, I still find it informative.

I guess a cleaner way to read in the lines is used by `process-lines`, you can use it as an inspiration for your final solution (make it use `call-process-region` instead of `call-process`).