We present an image fusion algorithm for a visible image and a near-infrared image. The proposed algorithm synthesizes a fused image that includes high-visibility information of both images while reducing artifacts caused by geometric and illumination inconsistencies. In the proposed fusion, the high-visibility area is labeled at each pixel by global optimization based on the local visibility and inconsistency. The local visibility is evaluated using a local contrast. The inconsistency is also locally estimated based on a learning-based approach. The fused luminance is constructed using Poisson image reconstruction that preserves the gradient of the selected high-visibility areas. The proposed fusion framework has various applications, which include denoising, haze removal, and image enhancement. Experimental results show that the proposed method has comparable or even superior performance to existing methods designed for specific applications.