IR-Lab Project of Yanjun Qi

---

Some further analysis of each data set

From the classification performance on selected 7 UCI date set with different class ratio, it seems that the performance is not mainly related with the imbalance ratio. To make this point clear, we make some investigation of the distribution of each set. So we use SVD to do the feature dimensional reduction and visualize these 7 sets on their first 3 principle components.

From the following figures, and together with the classification performance summary, we would still keep our conclusion that:

• The classification performance is not very related with the imbalance distribution, but rather highly related with the small training size problem and the class data distribution.
• Harmonic methods would do pretty well if the data has rough “manifold structure”
• Our Gaussian mixture model did not implement well. From the following data distribution, some of them are indeed could be represented by Gaussian Mixtures.

1.Data Set 1

Dataset	% Minority Examples	Dataset Size	FEATURE / Class Situation	CLASS USEd	Unlabel data size in EAch Experimental Run
Letter-a	3.9	20000	16 numeric (integer) features 17 classes	Letter “A” against all other letter	2000

• We could see that the two classes actually are not hard to distinguish, though the class unbalance ratio is high
• From this distribution, not very easy to conclude why Harmonic did better than the transductive SVM. ?

2.Data Set 2

Dataset	% Minority Examples		Dataset Size		FEATURE / Class Situation	CLASS USEd	Unlabel data size in EAch Experimental Run
Pendigits		8.3		7494	16 attributes (All input attributes are integers 0..100) 10 classes	Digits “0” against all other digits	2000

• We could see that the two classes actually are also well separated, though the class unbalance ratio is high
• From this figure’s distribution, we could see the rough “manifold structure”, which is the intuition of Harmonic method. Then it is understandable that Harmonic did best here.

3.Data Set 3

Dataset	% Minority Examples	Dataset Size	FEATURE / Class Situation	CLASS USEd	Unlabel data size in EAch Experimental Run
Letter-a-subset	17.0	4639	16 numeric (integer) features 17 classes	Letter “A” against Letter “BCDEF”	2000

• We could see that the two classes are not well separated, but still has good dense within class distribution
• From this figure’s distribution, we could also see a rough “manifold structure”, which is the intuition of Harmonic method. Then it is understandable that Harmonic did best here.

4.Data Set 5

Dataset	% Minority Examples	Dataset Size	FEATURE / Class Situation	CLASS USED	Unlabel data size in EAch Experimental Run
Yeast	28.9	1484	8 attributes (numerical ) 10 classes	“NUC” against all the other localizations (429 positive)	1350

• We could see that the two classes are not well separated, but still has good dense within class distribution
• From our performance comparison, this one did get worse performance compared to the above three sets, though this one has more balanced class distribution.
• From this distribution, not very easy to conclude why Harmonic method better than transductive SVM?

5.Data Set 6

Dataset	% Minority Examples	Dataset Size	FEATURE / Class Situation	CLASS USED	Unlabel data size in EACH Experimental Run
Pima	34.7	768	8 attributes ( numerical ) 2 classes	( 268 positive)	650

• We could see that the two classes are really bad overlapped with each other
• From our performance comparison, this one did worse compared to the first three sets, though this one has more balanced class distribution
• From this distribution, not very easy to why SVM-transductive did better than the Harmonic method?

6.Data Set 7

Dataset	% Minority Examples	Dataset Size	FEATURE / Class Situation	CLASS USED	Unlabel data size in Each Experimental Run
Bupa	42.0	345	6 attributes (numerical ) 2 classes	(145 positive)	240

• We could see that these two classes are also really bad overlapped with each other
• It also has few data points
• From our performance comparison, this one did much worse compared to the first three sets, though this one has more balanced class distribution
• Here, clearly, this data does not have “manifold structure” at all, this might be the reason why transductive-SVM did better than Harmonic here.

7.Data Set 8

Dataset	% Minority Examples	Dataset Size	FEATURE / Class Situation	CLASS USED	Unlabel data size in EAch Experimental Run
Pendigits -Subset	50.0	1438	16 numeric (integer) features 17 classes	Digit “3” against digits “9” (719 positive)	1300

• We could see that these two classes are so well separated with each other
• Its good classification performance is a natural result based on its distribution.
• Here clearly, this is a very simple classification task.
• In the very small of training size case, the neighborhood based method seems better than the margin based method. But when the training size is a little more, these methods both perform perfectly well.