In this work we construct multigrid preconditioners to speed up the solution process of a linear-quadratic optimal control problem constrained by the Stokes system. The first order optimality conditions of the control problem form a linear system (the KKT system) connecting the state, adjoint, and control variables. Our approach is to eliminate the state and adjoint variables by essentially solving two Stokes systems, and to construct efficient multigrid preconditioners for the Schur-complement of the block associated to the state and adjoint variables. These multigrid preconditioners are shown to be of optimal order with respect to the convergence properties of the discrete methods used to solve the Stokes system. In particular, the number of conjugate gradient iterations is shown to decrease as the resolution increases, a feature shared by similar multigrid preconditioners for elliptic constrained optimal control problems.