We improved an image segmentation algorithm based on neutrosophic set (NS) and extended the modified method into color image segmentation. The original NS image segmentation approach transformed the images into NS domain, which is described using three membership sets: T, I, and F. Then two operations, -mean and -enhancement operations were employed to reduce the set indeterminacy. Although this method was quite successful in image segmentation application, some drawbacks still exist, such as oversegmentation and fixed and parameters. Thus, a new algorithm is proposed to overcome these limitations of the NS-based image segmentation algorithm. Then, the new modified method is extended into color image segmentation. The NS-based image segmentation algorithm is applied to each color channel independently. Then each channel is moved to a matrix column, respectively, to construct the input matrix to the -means clustering. Experiments are conducted on a variety of images, and our results are compared with those new existing segmentation algorithm. The experimental results demonstrate that the proposed approach can segment the color images automatically and effectively.