I don't think there's an easy way out for this one. You might do something like:
(add-hook 'type'typo-mode-hook
(lambda ()
(when (and typetypo-mode (derived-mode-p 'markdown-mode))
(let ((map (make-sparse-keymap)))
(set-keymap-parent map typetypo-mode-map)
(define-key map (kbd "`") 'self-insert-command)
(push `(typetypo-mode . ,map)
minor-mode-overriding-map-alist)))))