As someone who can get easily distracted as I read a book, I prefer to use my Kindle to annotate. I highlight phrases I like, ask questions, and make brief notes if I find that I relate to something the author said or experienced. Annotating for me is also helpful when I'm discussing a book with someone else, and I want to share a poignant quote. This makes it easier to find the information I want to share.
A way you could annotate a physical book without it looking bad in the margins is to use small sticky notes. Write a brief comment, question, or "aha" moment on the note, then stick it near the paragraph to which it relates.