Beyond Black-Box Advice: Learning-Augmented Algorithms for MDPs with Q-Value Predictions