Cohen’s Kappa
Cohen’s Kappa is one of the statistical analyses applied automatically in MerQur. This page illustrates — on a Natural Sciences & Mathematics sample dataset — how the analysis is run, what the MerQur output looks like, and how the result is reported in APA 7 format.
🎯 What is it for?
Cohen’s Kappa automatically performs all required assumption checks (normality, homogeneity of variance, etc.) for the relevant data type in the background and presents the results with a clear table + chart. Automatic interpretation in the APA 7 standard, effect sizes such as Cohen’s d/η²/R², and 95% confidence intervals are reported.
📌 When is it used?
- Statistical analysis of measurements in the Natural Sciences & Mathematics domain
- To produce APA 7-compatible result tables for academic publications
- Hypothesis testing and decision-making processes
- Undergraduate / master’s / PhD theses after the appropriate method has been selected
📐 Assumptions
- Appropriate scale — Variables must be at the measurement level required by the analysis (nominal/ordinal/interval/ratio)
- Independent observations — Observations must come from individuals independent of one another
- Sufficient sample size — The minimum n requirement for the analysis must be met
- Outlier check — Outliers must be detected and evaluated
If assumptions are violated, MerQur automatically suggests a non-parametric or robust alternative.
🛠 How to do it in MerQur
Load the data. Select the sample file from File → Open. MerQur auto-detects column types.
Select the analysis. From the left side select Cohen’s Kappa.
Panel assignments (form fields in the program):
- error:
cats() got an unexpected keyword argument 'max_unique'
Optional settings. Effect size ✓ · 95% confidence interval ✓ · Assumption checks (automatic).
▶ Run — click the button. Results are produced automatically as a table + chart.
📄 Export to Word. APA 7-formatted report with italic statistical symbols.
📊 Sample Dataset — Natural Sciences & Mathematics
ℹ Note: The scenario, MerQur output and interpretation below were produced by actually running the real example dataset in MerQur. Numeric results on your own data will differ; the goal is to show how the analysis is set up and interpreted end-to-end.
🎬 Example File
This analysis is demonstrated on the following example dataset for Natural Sciences & Mathematics:
Fen_Matematik/25_kappa_observer.xlsx
🎬 Scenario
We measure the agreement of two observers classifying the same samples. For categorical agreement, kappa is appropriate.
⚙️ Variable Selection
- 1st measure / group: observer_a
- 2nd measure / group: observer_b
Data Preview (First 5 Rows)
| sample_id | observer_a | observer_b |
|---|---|---|
| 1 | high high | low |
| 2 | high | high |
| 3 | medium | medium |
| 4 | low | low |
| 5 | high | high |
n = 70 · Columns: sample_id, observer_a, observer_b
📈 MerQur Output
💬 Interpretation
We measured the agreement of two observers (observer_a vs observer_b) classifying the same samples into the same categories — excluding chance agreement: kappa = 0.76, substantial. Unlike raw percent agreement, kappa reports categorical agreement after removing the chance-agreement share, so it is more honest. In the lab it is the standard index for measuring how consistent two observers’ classifications (species ID, quality class) are.
⚠ Common Mistakes
- Misidentifying the data type (e.g., loading a categorical variable as numeric)
- Skipping assumption checks and going straight to the p-value
- Failing to report effect size — APA 7 requires both p and effect size
- Failing to apply a Type I error correction (Bonferroni/Tukey) in multiple comparisons
- Not switching to a non-parametric alternative when n is insufficient
📹 Video Walkthrough
Watch the video below for an end-to-end walkthrough of this analysis on a Natural Sciences & Mathematics file.
▶ Cohen’s Kappa — video walkthrough
This section is part of the Kategorik Veri Analizleri video (8 analyses in one video). The link below jumps straight to 6:35, where this analysis begins. Narration is in Turkish.
🎓 Education Sciences · 🏛 Architecture, Planning & Design · ⚙ Engineering · 🏥 Health Sciences · 📊 Social, Humanities & Admin Sciences · 🏃 Sport Sciences · 🌾 Agriculture, Forestry & Aquatic
📚 If You Used This Analysis, Cite MerQur
If you performed this analysis using MerQur in a scientific study, please use the citation below as part of your academic citation obligations (APA 7):
Örücü, Ö. K. (2026). MerQur: Integrated Academic Data Analysis & Reporting Platform [Computer software] (Version 1.0.0). https://doi.org/10.53463/merqur.2026001
For BibTeX, RIS, EndNote and the English citation form: all citation formats →
- American Psychological Association. (2020). Publication manual of the American Psychological Association (7th ed.).
- Field, A. (2018). Discovering statistics using IBM SPSS Statistics (5th ed.). Sage.
- Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Lawrence Erlbaum.