Yes. What you want is to zoom (the font size for) a frame, regardless of which buffers are displayed there.
zoom-frm.el, and these descriptions of zooming a frame and zooming a buffer.
See, in particular, command
zoom-in/out, which does both. It is bound by default to the keys normally bound to just buffer zooming:
C-x C-0, and
C-x C-=. You can also bind commands
zoom-out to mouse-wheel rotations. Using
C-u with any of these commands toggles between frame zooming and buffer zooming.
Besides interactively zooming frames or buffers, you can of course set the default font size you want for all frames.
The same Emacs-Wiki page has more info about setting fonts and font sizes.
Update after you decided that you want to set the default font size
The answer is in the last line I wrote above: See section Globally Change the Default Font on the Emacs Wiki Set Fonts page. It tells you how to set the default font. (It also tells you how to zoom font sizes interactively.)
Here is that information in a nutshell - put this in your init file:
(add-to-list 'default-frame-alist '(font . FONT ))
(set-face-attribute 'default t :font FONT )
But do read the entire section of that page, as it tells you more, including how to set the default font using
.Xresources and the Emacs daemon. An example of a line suitable for
~/.Xresources for setting the default font size is: