A powerful technique for modeling kinetic simulations of plasmas is the Particle in Cell method (PIC). In a PIC algorithm a number of macro-particles move through a computational grid due to the electromagnetic fields. These fields are calculated in a self-consistent manner using the distribution of the charge captured by the macro-particles and external boundary conditions. Traditionally, PIC codes used a uniform Cartesian grid to establish the connection between macro-particles and the field. However, interest in complex geometries and multiscale problems led to development of PIC methods that use block-structured adaptive meshes and logically structured curvilinear body fitted grids. Since emerging multi-core and many-core architectures appear to favor structured data and access patterns, we are very interested in exploring the potential of this curvilinear PIC (CPIC) method.
In this work we explore parallelization of CPIC using both traditional MPI everywhere and on-node techniques such as OpenMP. In addition, we discuss the potential of combining these approaches in the emerging but uncertain MPI+X paradigm. Since we consider the electrostatic case, we use the structured variational-coarsening based Black Box Multigrid (BoxMG) to solve Poisson's equation in the transformed space. We comment on updates that were required in BoxMG to better handle periodic boundary conditions on large grids non-power of two grids, as well as its parallel versions. This study highlights that an important challenge in PIC methods is balancing the data needs of macro-particles and the field solves, a challenge shared with most multi-physics PDE applications.