Accurate segmentation of the blood vessels from a retinal image plays a significant role in the prudent examination of the vessels. A supervised blood vessel segmentation technique to extract blood vessels from a retinal image is proposed. The uniqueness of the work lies in the implementation of feature-oriented dictionary learning and sparse coding for the accurate classification of the pixels in an image. First, the image is split into patches and for each patch, Gabor features are extracted at multiple scales and orientations to create a set of feature vectors (this is done for the whole training set). Then, an overcomplete feature-oriented dictionary is trained from the extracted Gabor features (selected on the basis of standard deviation) using the generalized K-means for singular value decomposition dictionary learning technique. Sparse representations are subsequently calculated for the corresponding features from the dictionary. The combination of feature vectors and sparse representations constitutes the final feature vector. This feature vector is then fed into the ensemble classifier for the classification of pixels into either blood vessel pixels or nonblood vessel pixels. The method is evaluated on publicly available DRIVE and STARE datasets, as they contain ground truth images precisely marked by experts. The results obtained on both of the datasets show that the proposed technique outperforms most of the state-of-the-art techniques reported in the literature.