It is perfectly possible to have packages.el put byte-compiled code into versioned subdirectories of the elpa package. This gives a structure like this:
The load path is constructed in such a fashion, that the versioned directory appears before the source directory, e.g.:
To accomplish all of this, I have just written the package package-compat, which uses mostly adviced functions to provide that feature for Emacs 23 to Emacs 27.
The package mechanism is just used for distribution in this case, since the code must be loaded before any packages are activated and/or installed.
"https://almost-un.org/selpa/packages" to the variable
(add-to-list 'package-archives ("almost-un.org" . "https://almost-un.org/selpa/packages/") t)
and download the package with
(package-install 'package-compat) or download the tar(1) archive from the above page.
Then follow the instructions in any of the Emacs lisp files.