Sharp Analysis for KL-Regularized Contextual Bandits and RLHF | Read Paper on Bytez