The mode sets up its own keymap which overrides the global map. The
:bind mechanism creates a binding in the global map, but in an Org mode file, the key is looked up in the mode map before the global map, so you get the definition in the mode map.
The usual way to override this is by using the mode hook. Add to your init file the following code:
(eval-after-load 'org '(add-hook 'org-mode-hook (lambda () (define-key org-mode-map (kbd "<M-S-return>") #'org-insert-subheading))))
The mode hook (
org-mode-hook in this case) is run at the end of the mode function (
org-mode in this case). "Running" the hook executes each function in the hook. In this case, when the function call is evaluated, it calls
define-key to redefine the given key (
(kbd "<M-S-return>")) in the given keymap (
org-mode-map) to the given command (
#'org-insert-subheading). And that is done after the mode function has set up the mode map, so it overrides whatever the mode function did.
Hooks are an important customization method in Emacs: you should read the "Hooks" section of the manual (well, you should read the whole manual eventually). And BTW, you can get to the "Hooks" section of the manual in Emacs:
C-h r i Hooks RET.