I am familiar with several non-secular yogas so your point can't be that Yoga is inherently secular !!
See Christian Yoga by Dechanet, In discussing deliverance you take at least 3 dogmas to be true
1) That there is a state unknown to you but you believe it to exist, called deliverance
2) that you believe it is not affected by who is your teacher
3 you essentially eliminate part of your human makeup , which is Mind AND Will. There either is truth or there isn't and to assuage your non-religiousness by saying it doesn't matter what you believe or if you believe -- well that is a very strong belief
Fanaticism is of at least 2 sorts. One is the person acting on their urges and passions and throwing off the reins of conscience and reason. This is far more likely to occur in a secular setting. Even the supposed ignoring of the suffering of others because you don't believe it matters -- that is fanaticism by my lights.
But Two there is the fanaticism of wrong thinking, and the only cure there is right thinking.
reading your post my main thought is: How could anyone be concerned only with their own deliverance? and for you to be right does not in any way imply others are wrong. A 'secular' approach that was rational and charitable would accept all "men of goodwill"