Shearography is a very effective non-destructive testing method for detecting the internal defects of composite materials. How to improve the detection efficiency of this technique to help achieve the rapid and automatic recognition of multiple defects has always been a hot research topic for researchers. Based on the results of the shearography, we propose a method to identify multiple defects in the phase map using the spatial characteristics of Freeman chain code. First of all, we use Canny operator to binarize the phase map, then refine the fringe by morphological means, and finally identify the defects by the spatial characteristics chain code. In addition, Otsu is used to assist in judging the fracture or incomplete defects. For the fringes under vacuum loading or thermal loading, this method can detect the defects quickly and successfully. The theoretical description is given and the effectiveness and practicability of the method are verified by experiments.