Evolutionary Sampling: A Novel Way of Machine Learning within a Probabilistic Framework

In many traditional machine learning methods, sampling is only a process of acquiring training data. However, some studies (on sequential Markov chains and particle filters) have demonstrated that sampling can be used for solving some intractable optimization problems in classical learning methods. Along this line of thinking, the relationships between sampling and learning are theoretically exploited in this paper, wherein the key feature of the sampling process is selecting representative samples from original data that can be modeled by a probability distribution. In theory, acquiring reliable samples is not an easy task for an arbitrary probability distribution. Motivated by approaches in evolutionary computation, rejection sampling and function approximation, a novel sampling strategy, called the evolutionary sampling, is proposed in this paper, and a machine learning method, called the evolutionary sampling approach (ESA), is put forward afterwards. Within ESA, a computing model, called the support sample model (SSM), is presented as well and is used to approximate an original density function. Accordingly, a concrete implementation of an evolutionary sampling approach (ESA) is proposed to seek the optimal model parameters of the SSM. Benefiting from the combination of rejection sampling and evolutionary searching, the ESA can theoretically converge to the optimal solution by minimizing the total variation distance, and can do this with high computational efficiency. Moreover, the normalized factor of a density function can be automatically estimated with high precision within the ESA. As a result, the ESA may be suitable for machine learning problems that could be transformed into density function approximation problems within a probabilistic framework. In addition, derived from the rejection sampling strategy, the ESA can also have online learning abilities required by large-scale data stream processing tasks. Theoretical analyses and application studies are carried out in this paper, and the results demonstrate that the ESA, as a novel way of machine learning, has several prominent merits aspired by past researches in machine learning.