Statistics: Results
Experiment 1
download csv
|
|
|
|
|
|
|
Agreements |
Disagreements |
Kappa |
|
|
Detector |
Projects |
Synthetics |
Misuses |
Potential Hits |
Open Reviews |
Open '?' |
Yes |
No |
Total |
Yes/No |
No/Yes |
Total |
p0 |
pe |
Score |
Hits |
Recall |
DMMC |
26 |
3 |
122 |
62 |
0 |
0 |
13 |
49 |
62 |
0 |
0 |
0 |
1 |
0.166 |
1 |
13 |
10.7% |
GrouMiner |
26 |
3 |
122 |
88 |
0 |
0 |
56 |
29 |
85 |
1 |
2 |
3 |
0.966 |
0.21 |
0.957 |
57 |
46.7% |
Jadet |
26 |
3 |
122 |
23 |
0 |
0 |
14 |
9 |
23 |
0 |
0 |
0 |
1 |
0.238 |
1 |
14 |
11.5% |
MuDetect |
55 |
4 |
171 |
149 |
0 |
0 |
119 |
25 |
144 |
1 |
4 |
5 |
0.966 |
0.134 |
0.961 |
123 |
71.9% |
Tikanga |
26 |
3 |
122 |
15 |
0 |
0 |
3 |
10 |
13 |
0 |
2 |
2 |
0.867 |
0.133 |
0.846 |
3 |
2.5% |
Total |
55 |
4 |
171 |
337 |
0 |
0 |
205 |
122 |
327 |
2 |
8 |
10 |
0.97 |
0.22 |
0.962 |
210 |
24.6% |
Experiment 2
download csv
|
|
|
|
|
Agreements |
Disagreements |
Kappa |
|
|
Detector |
Projects |
Potential Hits |
Open Reviews |
Open '?' |
Yes |
No |
Total |
Yes/No |
No/Yes |
Total |
p0 |
pe |
Score |
Hits |
Precision |
DMMC |
5 |
80 |
0 |
0 |
4 |
76 |
80 |
0 |
0 |
0 |
1 |
0.048 |
1 |
4 |
5% |
GrouMiner |
5 |
90 |
0 |
1 |
1 |
86 |
87 |
2 |
0 |
2 |
0.967 |
0.011 |
0.966 |
3 |
3.3% |
Jadet |
5 |
52 |
0 |
0 |
2 |
47 |
49 |
2 |
1 |
3 |
0.942 |
0.036 |
0.94 |
4 |
7.7% |
MuDetect |
10 |
146 |
0 |
0 |
29 |
112 |
141 |
2 |
3 |
5 |
0.966 |
0.153 |
0.96 |
32 |
21.9% |
MuDetectXP |
10 |
91 |
0 |
3 |
30 |
53 |
83 |
0 |
5 |
5 |
0.912 |
0.192 |
0.891 |
30 |
33% |
Tikanga |
5 |
41 |
0 |
0 |
2 |
33 |
35 |
0 |
6 |
6 |
0.854 |
0.039 |
0.848 |
2 |
4.9% |
Total |
10 |
500 |
0 |
4 |
68 |
407 |
475 |
6 |
15 |
21 |
0.95 |
0.111 |
0.944 |
75 |
15% |
Experiment 3
download csv
|
|
|
|
|
|
Agreements |
Disagreements |
Kappa |
|
|
Detector |
Projects |
Misuses |
Potential Hits |
Open Reviews |
Open '?' |
Yes |
No |
Total |
Yes/No |
No/Yes |
Total |
p0 |
pe |
Score |
Hits |
Recall |
DMMC |
30 |
177 |
42 |
0 |
0 |
13 |
27 |
40 |
0 |
2 |
2 |
0.952 |
0.199 |
0.941 |
13 |
7.3% |
GrouMiner |
29 |
173 |
14 |
0 |
0 |
7 |
7 |
14 |
0 |
0 |
0 |
1 |
0.25 |
1 |
7 |
4% |
Jadet |
29 |
173 |
12 |
0 |
0 |
11 |
0 |
11 |
1 |
0 |
1 |
0.917 |
0 |
0.917 |
12 |
6.9% |
MuDetect |
56 |
224 |
72 |
0 |
0 |
46 |
26 |
72 |
0 |
0 |
0 |
1 |
0.231 |
1 |
46 |
20.5% |
MuDetectXP |
51 |
215 |
151 |
0 |
0 |
92 |
54 |
146 |
2 |
3 |
5 |
0.967 |
0.218 |
0.958 |
95 |
44.2% |
Tikanga |
30 |
177 |
11 |
0 |
0 |
8 |
1 |
9 |
2 |
0 |
2 |
0.818 |
0.066 |
0.805 |
10 |
5.6% |
Total |
56 |
224 |
302 |
0 |
0 |
177 |
115 |
292 |
5 |
5 |
10 |
0.967 |
0.223 |
0.957 |
183 |
13.6% |