I had this problem and got it to work. Needed to spelunk through the code in epg.el and epg-config.el to determine what to fiddle with. This is what worked for me:
When encrypting, GPG uses the recipient's public key (so only she can decrypt it, with her private key). Your own keys are not used (neither private nor public). You private key is only used either to sign what you send, or to decrypt what ou receive.
It's where package.el keeps the keys used to authenticate the packages that it downloads. You don't need to gitignore it, but of course, it might leads to merge conflicts in some cases, so you may choose to gitignore it (it should be auto-created if/when necessary).
You can create your own command to set all variables before starting ERC:
(defun my/erc-freenode ()
(let ((erc-plist (car (auth-source-search :host "irc.freenode.net")))
(setq erc-server "irc.freenode.net")
(setq erc-nick (plist-get erc-plist :user))
(setq erc-password (funcall (plist-get ...
Not really an Emacs related problem but, one of the solution could be:
create a SUM of your .org file,
then create an encrypted signature of the SUM file, named SUM.sign,
and finally, share the 3 files whenever necessary
That way, your .org file will stay the same.