You Are Catching My Attention: Are Vision Transformers Bad Learners Under Backdoor Attacks?