Sensors, Vol. 19, Pages 4400: PEnBayes: A Multi-Layered Ensemble Approach for Learning Bayesian Network Structure from Big Data

Sensors, Vol. 19, Pages 4400: PEnBayes: A Multi-Layered Ensemble Approach for Learning Bayesian Network Structure from Big Data Sensors doi: 10.3390/s19204400 Authors: Yan Tang Jianwu Wang Mai Nguyen Ilkay Altintas Discovering the Bayesian network (BN) structure from big datasets containing rich causal relationships is becoming increasingly valuable for modeling and reasoning under uncertainties in many areas with big data gathered from sensors due to high volume and fast veracity. Most of the current BN structure learning algorithms have shortcomings facing big data. First, learning a BN structure from the entire big dataset is an expensive task which often ends in failure due to memory constraints. Second, it is quite difficult to select a learner from numerous BN structure learning algorithms to consistently achieve good learning accuracy. Lastly, there is a lack of an intelligent method that merges separately learned BN structures into a well structured BN network. To address these shortcomings, we introduce a novel parallel learning approach called PEnBayes (Parallel Ensemble-based Bayesian network learning). PEnBayes starts with an adaptive data preprocessing phase that calculates the Appropriate Learning Size and intelligently divides a big dataset for fast distributed local structure learning. Then, PEnBayes learns a collection of local BN Structures in parallel using a two-layered weighted adjacent matrix-based structure ensemble method. Lastly, PEnBaye...
Source: Sensors - Category: Biotechnology Authors: Tags: Article Source Type: research