The minor mode's hook is called both when activating and deactivating the minor mode, so how 'bout

    (add-hook 'my-minor-mode-name-hook
              (lambda ()
                (if my-minor-mode-name
                    (add-hook 'after-save-hook #'a-func-from-my-minor-mode nil 'local)
                  (remove-hook 'after-save-hook #'a-func-from-my-minor-mode 'local))))

Note that your calls to `add-hook` are both global, so even though you only add to `after-save-hook` when the minor mode is called, this addition applies to *all* buffers.  Maybe that was indeed what you wanted, but I assumed it's not in the above example code.