emacsclient -c "filename1"[...] for org files [...] What is slowing down the first emacsclient frame but not subsequent frames?
The fact that you are visiting an org-mode file for the first time when you open that frame, which means that Emacs must load the org libraries (which are numerous and large).
If possible it would be great if the daemon could also take on the loading that seems to be executed during that first call to emacsclient.
Simply put this in your init file:
(require 'org)
Depending on config (e.g. mode hooks, as you mentioned), there might be additional things being loaded once you actually trigger the major mode for the first time, so you might want to require anything additional.
The features variable will show you all the things which are currently loaded, and after-load-functions can be used to log which files are being loaded, but you might just try something like this as a shortcut:
(defun my-load-org-mode-more () (with-temp-buffer (org-mode)))
(add-hook 'emacs-startup-hook #'my-load-org-mode-more)