The self is not acted upon by the verb (he makes the fun, he doesn't make me), but becomes beneficiary of the action, and thus is the indirect object.
Compare...
Er kauft mir einen Mantel. (He buys me a coat.)
Es macht mir Spaß. (It does me fun/pleasure.)
...same construct.