Hi! The answer is "Let's go to Anne's house." The reason is because when you use the word go it's intended to move away from the current standing point, and come is to move toward the person spoken to. In addition, go, is usually used with the preposition "to".
Let me know if this makes sense. Have a great day!