Parameter optimization problems constrained by partial differential equations (PDEs) appear in many science and engineering applications. Solving these optimization problems may require a prohibitively large number of computationally expensive PDE solves, especially if there are many variable parameters. It is therefore advantageous to replace expensive high-dimensional PDE solvers (e.g. finite element) with lower-dimension surrogate models. In this paper, we use the reduced basis (RB) model reduction method in conjunction with a trust region optimization framework to accelerate PDE-constrained parameter optimization. New a posteriori error bounds on the RB cost and cost gradient for quadratic cost functionals are presented, and used to guarantee convergence to the optimum of the high-fidelity model. The proposed certified RB trust region approach thus requires only a minimal number of high-order solves, used to update the RB model if the approximation is no longer sufficiently accurate. We consider problems governed by elliptic PDEs and present numerical results for a thermal fin model problem with six parameters.