"Driving me crazy" is an English idiomatic expression. As such it can be expressed in ASL using a comparable expression that would mean the same thing but ASL doesn't use this same construction. That said ASL users often borrow expressions directly from English. So you will encounter signers who will borrow the expression literally and sign YOU DRIVE ME CRAZY. This would be quite acceptable since ASL users will frequently borrow clever or appropriate English expressions like that and use them as borrowed utterances in ASL.
To express the same meaning as "You drive me crazy" in ASL one might sign. FINISH (meaning stop that right now) MAKE ME CRAZY YOU. Or STOP NOW! FINISH WITH YOU, LEAVE. MAKE ME ANGRY YOU.
There would be various ways to express this depending on whether it is being said in jest or seriously in an argumentative way or in an accusatory manner.
Hope this helps you understand a bit more about how ASL works and its relationship to English.
Regards