Solving a large sparse linear system of equations is one of the most important problems in scientific and engineering computations. The Krylov subspace methods are widely used in this respect. This paper first reviews the Krylov subspace methods and their various types, such as, the orthogonal projection method (Ritz-Galerkin method), the orthogonalization method (or the minimal residual method), the bi-orthogonalization method (Petrov-Galerkin method), and the CGNE and CGNR methods for normal systems. The advantages and shortcomings of these methods are analyzed. Especially, we focus on the parallel computation of the sparse matrix-vector multiplication and the inner product. Then, this paper discusses the development of the preconditioning and the parallel preconditioning technique, the residual smoothing technology with its parallel implementation, the reasonable distribution of data, the bottleneck problem of the inner product.