Fixed multiclass weighted feature-importance baseline
Issue:
The weighted composite feature-importance plot used a hard-coded balanced accuracy no-skill cutoff of 0.5 for all classification tasks.
This is correct for binary classification, where random/no-skill balanced accuracy is 0.5, but it is not correct for multiclass classification. For multiclass datasets, the expected no-skill balanced accuracy should be 1 / number_of_classes.
Impact:
In multiclass runs, models with balanced accuracy above random chance but below 0.5 were incorrectly given a weight of 0 in the normalized and weighted composite FI plot. This could make useful model-specific feature-importance contributions disappear from the weighted FI visualization.
Example:
For a 3-class dataset, the no-skill baseline should be 0.333, not 0.5. A model with balanced accuracy 0.45 is better than random for that dataset, but the old weighting logic treated it as no-skill and zeroed out its FI contribution.
Fix:
Updated the P8 FI weighting logic so the balanced-accuracy baseline is task-aware:
- Binary classification: baseline remains 0.5.
- Multiclass classification: baseline is now 1 / number_of_classes.
- Regression explained-variance weighting is unchanged.
The number of classes is inferred from the dataset outputs, including ClassCounts.csv and CV datasets.
Result:
Weighted composite FI plots now use the correct no-skill threshold for multiclass classification and no longer incorrectly zero out above-chance multiclass models below 0.5 balanced accuracy.
Fixed multiclass weighted feature-importance baseline
Issue:
The weighted composite feature-importance plot used a hard-coded balanced accuracy no-skill cutoff of 0.5 for all classification tasks.
This is correct for binary classification, where random/no-skill balanced accuracy is 0.5, but it is not correct for multiclass classification. For multiclass datasets, the expected no-skill balanced accuracy should be 1 / number_of_classes.
Impact:
In multiclass runs, models with balanced accuracy above random chance but below 0.5 were incorrectly given a weight of 0 in the normalized and weighted composite FI plot. This could make useful model-specific feature-importance contributions disappear from the weighted FI visualization.
Example:
For a 3-class dataset, the no-skill baseline should be 0.333, not 0.5. A model with balanced accuracy 0.45 is better than random for that dataset, but the old weighting logic treated it as no-skill and zeroed out its FI contribution.
Fix:
Updated the P8 FI weighting logic so the balanced-accuracy baseline is task-aware:
The number of classes is inferred from the dataset outputs, including ClassCounts.csv and CV datasets.
Result:
Weighted composite FI plots now use the correct no-skill threshold for multiclass classification and no longer incorrectly zero out above-chance multiclass models below 0.5 balanced accuracy.