Objective assessment of visual comfort for stereoscopic video is of great importance for stereoscopic image safety issue. We propose a novel visual comfort assessment metric framework that systematically exploits human visual attention models. In a stereoscopic video shot, perceptually significant regions where human subjects pay more attention are likely to play an essential role in determining the overall level of visual comfort. As a specific example of this concept, we develop a visual comfort metric that quantifies the level of visual discomfort caused by fast salient object motion. The performance of the proposed visual comfort metric has been evaluated using natural stereoscopic videos. The experimental results show that the proposed visual comfort metric significantly improves the correlations with subjective judgment.