This work presents a vision-based data collection system for pedestrian behavior analysis at intersections that include crossing counts, waiting time, crossing speed, and facility utilization. The tracking system uses contextual fusion of motion and appearance cues to more reliably track pedestrians during stop-and-go movements at intersections. Moreover, the pedestrian tracking system is improved through cooperation of two different tracking algorithms: bipartite graph match and optical flow algorithms. The performance of the proposed system is evaluated separately at the detection and tracking steps followed by behavior analyses of pedestrians for three different intersection videos of Las Vegas. The experimental results show the efficiency of the proposed system and intersection utilization is depicted through heat maps.