Performing high-accuracy object detection under low-illumination conditions is a difficult task. The fusion of infrared (IR) and visible images can provide a great reference value for human observation or other computer vision tasks. Therefore, we propose an image fusion-based method to detect objects under poor lighting conditions. It mainly proposes a whole set of process improvements to enable the detection of medium-sized and small objects in poorly illuminated images with high-accuracy. First, the saliency detection of the IR image extracts the salient area in the image to obtain the main objects. Second, the IR and visible-light images undergo multiscale decomposition and fusion to obtain the background information of the visible-light image. Finally, the salient area is dot-multiplied with the fused image, so it can remove unnecessary background information and interference information while acquiring the main objects. Then, the ablation experiment is used to detect the objects. Experimental results demonstrate the effectiveness and robustness of the method. Moreover, lower false detection efficiency and better detection accuracy were obtained, especially in the detection of objects under poor lighting conditions.