The canonical technique for Monte Carlo simulations in statistical physics is importance sampling via a suitably constructed Markov chain. While such approaches are quite successful, they are not particularly well suited for parallelization as the chain dynamics is sequential, and if replicated chains are used to increase statistics each of them relaxes into equilibrium with an intrinsic time constant that cannot be reduced by parallel work. Population annealing is a sequential Monte Carlo method that simulates an ensemble of system replica under a cooling protocol. The population element makes it naturally well suited for massively parallel simulations, and bias can be systematically reduced by increasing the population size. We present an implementation of population annealing on graphics processing units and discuss its behavior for different systems undergoing continuous and first-order phase transitions.