I think it could be simpler, based on @xuhdev's answer:
(setq electric-pair-inhibit-predicate (lambda (c) (char-equal c ?\")))
The doc says
The function is called with a single char (the opening char just
inserted). If it returns non-nil, then ‘electric-pair-mode’ will not
insert a matching closer.
So the lambda only needs to return a boolean.