Object tracking is considered to be a key technique in many computer vision applications, such as video surveillance, object recognition, and robotics. We propose a method that improves the performance of scale-invariant feature transform (SIFT)-based object tracking algorithm to track the object in the subsequent video frames. Recently, many feature-based tracking methods have been proposed. An efficient and improved SIFT feature matching-based tracking method via neural network is provided and compares the outcome of this method with other tracking method outcomes. The tracked object is assigned a distance with the Kinect sensor to determine the depth of the detected object. The experimental results show that the proposed method can track the target object under different situations such as rotation, scaling, and many others with less computation time. Self-organizing map-based improved object tracking method can also estimate the distance between the tracked object and the image sensor. The proposed tracking technique will be useful for the development of many computer vision and robot navigation applications.