English answers "no" to contradict a negative question's assumption and "yes" to confirm it. What I mean by assumption is this. When someone asks you a question like, "Are you not going with us?", that question form means that they already assume your answer will be that you are going. This kind of question assumes that your answer will be "yes", that your answer will be positive. Remember, a NEGATIVE question assumes a POSITIVE answer. So if you ARE going, your answer should be "yes", just as they expect it will be. If you are NOT going, you have to contradict their assumption that you ARE going by using the negative "no". So:
Are you not going with us?
If you ARE: "Yes" or "Yes, I am going."
If you are NOT: "No" or "No, I'm not."
Meaning that you were correct! Good job!