You may have answered your own question. You know it's not a noun phrase because it doesn't start with a noun or have a noun in it. But it is a pretty tricky question. "Let down" is a phrasal verb. And "me" is a pronoun which is acting as the object of that verb. It's tricky because they went put that object in the middle of the phrasal verb. We call that a separable phrasal verb, meaning that the object is in the middle of the phrase. Phrasal verbs where the object is not inside the phrase are called inseparable.
Hope that helps
