Magit doesn't work on my system. I'd like to remove it and everything that came with it when I installed the package. How can I do this? What packages, beside Magit, were installed on my emacs when I installed Magit?
Through emacs' package manager, via MELPA.
In which case a manual approach is:
That will show you the dependencies for all your packages.
magit RET to check that one, and you can then look for sub-dependencies in turn.
If an earlier version of Magit had a dependency that was removed in subsequent versions, it's possible that it would now be an 'orphaned' package, as I suspect package.el doesn't attempt to detect such things. If you were really keen, and assuming you version-control your config, you could always check the dependencies over your repo history to see whether any such thing had occurred.
(Of course if you version-control your config, you almost certainly have a version-control record of what else you added when you installed or updated Magit.)