Macros do not evaluate to a result, they evaluate to an expansion, which is a form that substitutes the macro at the place where it's written, the expansion site. The expansion itself is not evaluated by the compiler.
As such, you normally don't try to generate function definitions completely. Rather you'd write a macro to performs whatever expansion you'd like to have, and simply use that macro within a function body, i.e.
(defmacro my/path-separator ()
(if (eq system-type 'windows-nt) "\\" "/"))
(defun join-dirs (a b)
(concat a (my/path-separator) b))
But in this specific example, you should not use macros at all. Emacs' byte code is not actually OS- or architecture-specific, and thus you must not generate system-specific code during expansion. In other words, the path separator is not a compile time constant, it's a load time constant and should be computed at load time rather than at compile time.
Generally you should only use macros for syntax, but never for semantics.
And actually you'd just use
(expand-file-name "b" "a") here :)