SCITFS folds adaptive redundancy penalization and bootstrap stability regularization into one objective, lifting SVM accuracy by 3.7% and Random Forest by 4.2% on eight benchmarks while cutting 92.3% of features.
The work proposes Stability-Constrained Information-Theoretic Feature Selection (SCITFS), which integrates conditional entropy, normalized mutual information maximization, and a stability term penalizing feature-ranking variance across bootstrap samples into a single formally defined objective, implemented via a greedy forward-selection strategy with proven monotonicity guarantees at O(B n2 m d4 + k n2); across eight benchmark datasets and five classifiers (SVM, Random Forest, k-NN, XGBoost, Logistic Regression), SCITFS outperforms Information Gain, Mutual Information, mRMR, ReliefF, Fisher Score, and JMI, achieving a 3.7% average accuracy improvement on SVM and 4.2% on Random Forest with 92.3% feature reduction while maintaining performance; Friedman test (χ² = 127.4, p < 0.