A low bit rates image compression framework based on adaptive block downsampling and super resolution (SR) was presented. At the encoder side, the downsampling mode and quantization mode of each macroblock are determined adaptively using the ratio distortion optimization method, then the downsampled macroblocks are compressed by the standard JPEG. At the decoder side, the sparse representation-based SR algorithm is applied to recover full resolution macroblocks from decoded blocks. The experimental results show that the proposed framework outperforms the standard JPEG and the state-of-the-art downsampling-based compression methods in terms of both subjective and objective comparisons. Specifically, the peak signal-to-noise ratio gain of the proposed framework over JPEG reaches up to 2 to 4 dB at low bit rates, and the critical bit rate to JPEG is raised to about 2.3 bits per pixel. Moreover, the proposed framework can be extended to other block-based compression schemes.