日本人の相手 or 相手の日本人. These are two different things. It sound confusing if the definition of 相手 is not clear. So Let's talk about 相手.
相手 could be:
- the person who does something (a project) with you. So 相手 could be a "partner." For example ダンスの相手 is your partner to dance with.
- the object of your action. For example, if you are having a business negotiation, the person you are talking to could be 商談の相手 or "the business person you are negotiating with."
- the opponent to compete (a game). For example 試合の相手 is an opponent of a game.
I think the key here is what you meant to say. Do you want to say "a Japanese person who I am talking to" or "the partner (or opponent) who is doing some project (or a game) with a Japanese person"? The first one is 相手の日本人, I think. The second one is 日本人の相手. If you say 相手の日本人, you are talking about 日本人. 相手 is modifying 日本人. If you say 日本人の相手, you are talking about 相手. In this case, 日本人modifys the 相手.