First, you are apparently confusing the mouse pointer with the text cursor (aka cursor). It seems that you mostly talking about the latter, but referring to it as the "cursor".
Second, insertion of a character is always between characters (or before the first or after the last character). This is necessarily so, by definition. Even if it were to behave and be conceived as replacing an existing character by the one you want to insert, that insertion would still be between two chars (or before the first or after the last).
Third, as for the representation of the (text) cursor: you have a choice. You can use a narrow vertical bar or a box, hollow or solid (on some platforms there might be additional options). If you choose a box cursor then the box is placed on top of the character to the right of the insertion point.
You apparently already know how to change the (text) cursor style, and have opted for a bar. But your rant/question is more about the mouse pointer shape, I think, as it is apparently the pointer position (the part of the pointer shape that matters, determining where a click really is) that is throwing you off.
Fourth, you ask about setting point (the text cursor position) by clicking the mouse (left mouse button, aka
mouse-1). You say that clicking with the mouse pointer positioned exactly between two chars ends up setting point to the left of the character on the left of the pointer, and not at the pointer (between the two characters).
clicking mouse puts the point one character back from the mouse position
Why draw the mouse cursor in emacs window with vertical thin bar if you cannot effectively position it between characters?
I don't see that behavior. I position the I-beam pointer (which is what I think you mean here) between two characters, click it, and get the text insertion position (and the text cursor) between those characters.
If you can reproduce the behavior you describe then it is a bug, I think. Use
M-x report-emacs-bug to report it.
However, I suspect that the problem is that you are positioning the wrong part of your mouse pointer. What you call "mouse position" is probably not the position of the part of the pointer that matters.
Just as for text cursor representation, the mouse pointer can be represented in different ways. If it is an arrow of some kind, the pointer position (which sets point when you click
mouse-1) is often the tip of the arrow. Find out exactly which part of your pointer is used, and then get used to that.
You say that Emacs behaves differently from the other programs you are used to, with respect to the pointer. Maybe your experience is limited? Having used all kinds of programs on lots of different platforms ever since computers had pointers and cursors, I can say that the behavior and representations are variable. And they vary in Emacs too, depending on the platform.
You can also easily change the pointer representation (choose another pointer). See the Elisp manual, node Pointer Shape.
It is also the case that, by default, the pointer shape changes depending on where it is, depending, in fact, on what you can do with it at that position. At a position where you can insert text it is often a vertical I-beam shape (an insertion pointer). At other positions it is often an arrow.