As Drew says, you cannot do this without knowing specifically which keymaps and bindings are being affected.
There are common cases in which all the bindings are liable to be in a single keymap (which is not normally modified after the library has been loaded), in which case you could clobber that keymap using something like:
(eval-after-load "olivetti"
'(setq olivetti-mode-map (make-sparse-keymap)))
You can, of course, unbind specific keys from the keymap in question, allowing lower-priority keymaps to be processed for those keys instead.
(eval-after-load "olivetti"
'(progn (define-key olivetti-mode-map (kbd "C-c [") nil)
(define-key olivetti-mode-map (kbd "C-c ]") nil)))
In the case of minor modes, the keymap look-up and precedence is determined by the order of minor-mode-map-alist
.
One consequence is that you can inhibit a minor mode's bindings by removing it from that list:
(eval-after-load "olivetti"
'(assq-delete-all 'olivetti-mode minor-mode-map-alist))
A softer option (if the conflict is between minor modes) is to either load libraries which define minor modes in the order you want them to be loaded in1, or otherwise to manipulate the order of that list to achieve the same effect (for example I do the latter in an after-load-functions
callback, in order to keep one particular minor mode at the head of the list at all times).
1 By default minor modes defined more recently will have precedence when bindings conflict; so if you were to (require 'olivetti)
very early in your init, you may find the problem goes away.