$f$-Divergence Regularized RLHF: Two Tales of Sampling and Unified Analyses | Read Paper on Bytez