Parallel dynamic data-driven model for concept drift detection and prediction

S-Y. Lin, Y-C. Chiu, Jacek Lewandowski, Kuo-Ming Chao

Research output: Contribution to journalArticle

36 Downloads (Pure)

Abstract

The traditional data analysis and prediction method assumes that data distribution is normal and will not change. Therefore, it can predict unlabeled data by analyzing the static and historical data. However, in today’s big-data environment, which is changing frequently, the traditional approaches can no longer be effective, as they cannot handle concept drift problems in a Dynamic Data Driven Application System (DDDAS). This study proposes a parallel detection and prediction method for concept drift problems in DDDAS. The proposed method can detect dynamic and changing data, and then feedback to the prediction model to revise for better subsequent predictions. Furthermore, this method computes a global prediction result by aggregating local predictions in the resource bounded environment. Therefore, the prediction accuracy increases, and the computation time decreases. In the simulation, the Map-Reduce technology is used for parallel processing. The simulation results show that the prediction accuracy is raised by 14%, and the execution time is improved by almost 45%.
Original languageEnglish
Pages (from-to)1413-1426
JournalJournal of Intelligent & Fuzzy Systems
Volume32
Issue number2
Early online date23 Dec 2016
DOIs
Publication statusPublished - 30 Jan 2017

Bibliographical note

The final publication is available at IOS Press through http://dx.doi.org/10.3233/JIFS-169138

Keywords

  • Dynamic data-driven application system
  • concept drift
  • Map-Reduce

Fingerprint Dive into the research topics of 'Parallel dynamic data-driven model for concept drift detection and prediction'. Together they form a unique fingerprint.

  • Cite this