Texture and color are important features in analyzing images of natural scenes. Fractal descriptors based on quaternion Fourier transform and local polynomial regression are proposed for color texture image analysis. First, considering the relation between the power spectrum and frequency in the quaternion Fourier transform domain, we proposed fractal dimensions of a color image using a fast quaternion Fourier transform. Second, local polynomial regression is applied to estimate log(spectrum)–log(frequency) curve, which is not usually linear in a natural texture image. Finally, a local polynomial regression curve is defined as fractal descriptors for the color image classification problem with multiclasses. The experimental results show that our proposed approach is more effective than other color texture analysis methods both in the correct classification rate and the duration.