Feature Selection using Multi-objective Genetic Algorith m: A Hybrid Approach

Abstract

Feature selection is an important pre-processing task for building accurate and comprehensible classification models. Several researchers have applied filter, wrapper or hybrid approaches using genetic algorithms which are good candidates for optimization problems that involve large search spaces like in the case of feature selection. Moreover, feature selection is an inherently multi-objective problem with many competing objectives involving size, predictive power and redundancy of the feature subset under consideration. Hence, Multi-Objective Genetic Algorithms (MOGAs) are a natural choice for this problem. In this paper, we propose a hybrid approach (a wrapper guided by filter approach) for feature selection which employs a MOGA at filter phase and a simple GA at the wrapper phase. The MOGA at filter phase provides a non-dominated set of feature subsets optimized on several criteria as input to the wrapper phase. Now, Genetic Algorithm at wrapper phase does the classifier dependent optimization. We have used support vector machine (SVM) as the classification algorithm in the wrapper phase. The proposed hybrid approach has been validated on ten datasets from UCI Machine learning repository. A comparison is presented in terms of predictive accuracy, feature subset size and running time among the pure filter, pure wrapper, an earlier hybrid approach based on genetic algorithm and the proposed approach.