helm-list-elisp-package already does what you want: it only downloads descriptions of all configured ELPA packages for the first time.
Even without downloading, It may still looks slow because of a great number of packages. In this case, you can use
helm-resume to resurrect previously invoked
helm-list-elisp-package, it is much faster.
Please also see the implementation of
helm-list-elisp-package and the docstring of
Don't fetch the updated list of packages:
(defun helm-list-elisp-packages-no-fetch ()
(setq helm-el-package--initialized-p t)
Do fetch the updated list of packages:
C-u M-x helm-list-elisp-packages
Only fetch for the first time during one Emacs session and don't refetch for the rest time: