In the emerging international standard for scalable video coding (SVC) as an extension of H.264/AVC, a computationally expensive exhaustive mode decision is employed to select the best prediction mode for each macroblock (MB). Although this technique achieves the highest possible coding efficiency, it results in extremely large computation complexity, which obstructs SVC from the practical application. We propose a fast mode decision algorithm for SVC, comprising two fast mode decision techniques: early SKIP mode decision and adaptive early termination for mode decision. They make use of the coding information of spatial neighboring MBs in the same frame and neighboring MBs from base layer to early terminate the mode decision procedure. Experimental results show that the proposed fast mode decision algorithm can achieve the average computational savings of about 70% with almost no loss of rate distortion performance in the enhancement layer.