Statistics: Results

Experiment 1

Agreements Disagreements Kappa
Detector Projects Synthetics Misuses Potential Hits Open Reviews Open '?' Yes No Total Yes/No No/Yes Total p0 pe Score Hits Recall
DMMC 16 27 47 26 52 0 0 0 0 0 0 0 0 0 0 0 0%
Findbugs 29 27 66 17 0 0 11 6 17 0 0 0 1 0.228 1 11 16.7%
GrouMiner 16 27 47 30 60 0 0 0 0 0 0 0 0 0 0 0 0%
Jadet 16 27 47 14 28 0 0 0 0 0 0 0 0 0 0 0 0%
Tikanga 16 27 47 18 36 0 0 0 0 0 0 0 0 0 0 0 0%
Total 29 28 67 105 176 0 11 6 17 0 0 0 0.162 0.006 0.157 11 3.3%

Experiment 2

:
Agreements Disagreements Kappa
Detector Projects Potential Hits Open Reviews Open '?' Yes No Total Yes/No No/Yes Total p0 pe Score Hits Precision
DMMC 10 181 362 0 0 0 0 0 0 0 0 0 0 0 0%
Findbugs 5 82 0 0 37 34 71 5 6 11 0.866 0.192 0.834 43 52.4%
GrouMiner 12 182 364 0 0 0 0 0 0 0 0 0 0 0 0%
Jadet 13 77 154 0 0 0 0 0 0 0 0 0 0 0 0%
Tikanga 13 66 132 0 0 0 0 0 0 0 0 0 0 0 0%
Total 14 588 1012 0 37 34 71 5 6 11 0.121 0.004 0.117 43 7.3%

Experiment 3

Agreements Disagreements Kappa
Detector Projects Misuses Potential Hits Open Reviews Open '?' Yes No Total Yes/No No/Yes Total p0 pe Score Hits Recall
DMMC 23 87 30 8 0 10 16 26 0 0 0 0.867 0.178 0.838 10 11.5%
Findbugs 16 84 37 0 0 32 5 37 0 0 0 1 0.117 1 32 38.1%
GrouMiner 20 85 7 6 0 0 4 4 0 0 0 0.571 0 0.571 0 0%
Jadet 28 95 4 0 0 3 1 4 0 0 0 1 0.188 1 3 3.2%
Tikanga 28 98 11 6 0 6 2 8 0 0 0 0.727 0.099 0.697 6 6.1%
Total 29 103 89 20 0 51 28 79 0 0 0 0.888 0.18 0.863 51 9.9%