Statistics: Results

Experiment 1

Agreements Disagreements Kappa
Detector Projects Synthetics Misuses Potential Hits Open Reviews Open '?' Yes No Total Yes/No No/Yes Total p0 pe Score Hits Recall
MuDetectOPAL_180312 28 2 64 43 37 0 6 0 6 0 0 0 0.14 0 0.14 6 9.4%
MuDetectOPAL_180320 29 2 65 45 0 0 36 9 45 0 0 0 1 0.16 1 36 55.4%
MuDetectOPAL_180324 29 2 65 44 0 0 36 8 44 0 0 0 1 0.149 1 36 55.4%
MuDetectOPAL_180330 29 2 65 54 0 0 48 6 54 0 0 0 1 0.099 1 48 73.8%
Total 29 2 65 186 37 0 126 23 149 0 0 0 0.801 0.084 0.783 126 48.5%

Experiment 2

:
Agreements Disagreements Kappa
Detector Projects Potential Hits Open Reviews Open '?' Yes No Total Yes/No No/Yes Total p0 pe Score Hits Precision
MuDetectOPAL_180227 1 20 20 0 0 0 0 0 0 0 0 0 0 0 0%
MuDetectOPAL_180312 5 60 29 1 3 27 30 0 0 0 0.5 0.023 0.488 3 5%
MuDetectOPAL_180320 5 80 0 0 24 56 80 0 0 0 1 0.21 1 24 30%
MuDetectOPAL_180324 5 80 0 0 12 68 80 0 0 0 1 0.128 1 12 15%
MuDetectOPAL_180330 5 80 0 0 15 65 80 0 0 0 1 0.152 1 15 18.8%
Total 5 320 49 1 54 216 270 0 0 0 0.844 0.114 0.824 54 16.9%

Experiment 3

Agreements Disagreements Kappa
Detector Projects Misuses Potential Hits Open Reviews Open '?' Yes No Total Yes/No No/Yes Total p0 pe Score Hits Recall
MuDetectOPAL_180312 5 39 22 22 0 0 0 0 0 0 0 0 0 0 0 0%
MuDetectOPAL_180320 5 39 22 0 0 17 5 22 0 0 0 1 0.176 1 17 43.6%
MuDetectOPAL_180324 5 39 23 0 0 18 5 23 0 0 0 1 0.17 1 18 46.2%
MuDetectOPAL_180330 5 39 28 0 0 20 8 28 0 0 0 1 0.204 1 20 51.3%
Total 5 39 95 22 0 55 18 73 0 0 0 0.768 0.11 0.74 55 35.3%