RAMP: Reinforcement Adaptive Mixed Precision Quantization for Efficient On Device LLM Inference
RAMP uses reinforcement learning for adaptive mixed-precision quantization, achieving 6% size and 1-3% quality improvements for on-device LLM inference.
Arpit Singh Gautam, Saurabh Jha