We consider a problem of computing a number of extreme singular values and the corresponding left and right singular vectors of a large possibly sparse matrix. We, first, review several existing methods which are currently available in the literature and discuss the complications which appear, e.g., when the triple corresponding to the smallest singular value is required. Next, we describe how gradient-type methods can be applied to the problem and suggest an iterative scheme for computing the extreme singular triples. Finally, we discuss possible ways of introducing a preconditioner to improve the convergence to the smallest singular values and the corresponding singular vectors.