The projection axes produced by conventional locality preserving projection (LPP) are not orthogonal though many dimension reduction methods favor the use of orthogonal projection axes. Orthogonal LPP (OLPP) has been found to perform well in document indexing but suffers from a much higher computational complexity than conventional LPP. This is somewhat because current OLPP algorithm must solve the same number of eigen equations as the number of required projection axes. In contrast, conventional LPP obtains all of the projection axes by solving just one eigenequation. A further drawback of current OLPP algorithm is that, since it requires a number of matrix operations, it also produce more rounding errors than conventional LPP. Four main theoretical contributions are presented. First, a new, more computationally efficient algorithm for implementing OLPP is proposed. Second, for the first time the solution property of conventional LPP is shown. Third, another form of current OLPP algorithm is described. Finally, it is shown that if the projection axes of conventional LPP and the OLPP are paired in sequence, the projection axis of the OLPP has a greater ability to preserve locality than the paired projection axis of conventional LPP.