Monte Carlo simulations are used to study simple systems where the underlying Markov chain satisfies the necessary condition of global balance but does not obey the more restrictive condition of detailed balance. Here, we show that non-reversible Markov chains can be set up that generate correct stationary distributions, but reduce or eliminate the diffusive motion in phase space typical of the usual Monte Carlo dynamics. Our approach is based on splitting the dynamics into a set of replicas with each replica representing a biased movement in reaction-coordinate space. This introduction of an additional bias in a given replica is compensated for by choosing an appropriate dynamics on the other replicas such as to ensure the validity of global balance. First, we apply this method to a mean-field Ising model, splitting the system into two replicas: one trying to increase magnetization and the other trying to decrease it. For this simple test system, our results show that the altered dynamics is able to reduce the dynamical critical exponent. Generalizations of this scheme to simulations of the Ising model in two dimensions are discussed.