Parallel dynamic data-driven model for concept drift detection and prediction

S-Y. Lin, Y-C. Chiu, Jacek Lewandowski, Kuo-Ming Chao

    Research output: Contribution to journalArticlepeer-review

    103 Downloads (Pure)

    Abstract

    The traditional data analysis and prediction method assumes that data distribution is normal and will not change. Therefore, it can predict unlabeled data by analyzing the static and historical data. However, in today’s big-data environment, which is changing frequently, the traditional approaches can no longer be effective, as they cannot handle concept drift problems in a Dynamic Data Driven Application System (DDDAS). This study proposes a parallel detection and prediction method for concept drift problems in DDDAS. The proposed method can detect dynamic and changing data, and then feedback to the prediction model to revise for better subsequent predictions. Furthermore, this method computes a global prediction result by aggregating local predictions in the resource bounded environment. Therefore, the prediction accuracy increases, and the computation time decreases. In the simulation, the Map-Reduce technology is used for parallel processing. The simulation results show that the prediction accuracy is raised by 14%, and the execution time is improved by almost 45%.
    Original languageEnglish
    Pages (from-to)1413-1426
    JournalJournal of Intelligent & Fuzzy Systems
    Volume32
    Issue number2
    Early online date23 Dec 2016
    DOIs
    Publication statusPublished - 30 Jan 2017

    Bibliographical note

    The final publication is available at IOS Press through http://dx.doi.org/10.3233/JIFS-169138

    Keywords

    • Dynamic data-driven application system
    • concept drift
    • Map-Reduce

    Fingerprint

    Dive into the research topics of 'Parallel dynamic data-driven model for concept drift detection and prediction'. Together they form a unique fingerprint.

    Cite this