Statistics Corner: Chi-squared Test

Disclaimer

For demonstration, Excel^® was used to generate the data for the analysis. However, the SATS-36 questionnaire genuinely assesses statistics anxiety.

Chi-squared Test (χ2)

Chi-square is a test of significance when the dependent variable is nominal—it does not tell the strength of the association. Further, the order of categories does not affect Chi-square—it is affected only by differences between groups. The Chi-squared test can be extended to interval or ratio data that researchers collapse into ordinal categories. The distribution of Chi-square is continuous, whereas the test applies to nominal data. As continuous distribution estimates the discrete probability of observed binomial (yes vs no) frequencies, the same overestimates the Chi-square value, which results in a smaller p-value than expected—a crucial reason for the increase in type-I error. The continuity correction for large samples does not make a significant difference—however, a small, expected value in a few cells for a small sample can skew the calculation of Chi-square statistics and hence the p-value. Yates’ continuity correction is applied to reduce error in the approximation. For a 2 × 2 table, it assumes to have at least an expected value—not observed value of five or more. The Chi-squared test for more than two rows and columns assumes that not >20% of cells have an expected value less than five and no cell has a value less than one. The researchers don’t have to apply Yates’ correction for more than two columns and rows or when the test result is not statistically significant. Unlike bidirectional tests such as Z and t-tests, Chi-square is a one-tailed (right side) test—only positive and large values can reject the null hypothesis.⁴

In general, there are three types of Chi-squared tests:

Assumptions

• The patients are randomly selected.

• Patients in the sample are independent of each other.

• The data in the cells are frequencies or counts—no percentages or proportions.

• The variable levels are mutually exclusive—a single subject contributes data to one and only one group.

• Independent and dependent variables should be categorical (nominal or ordinal).

Limitations

• Small sample size—overly sensitive, give smaller p-value. Apply Yates correction or Fisher’s exact test.

• Interpretation is challenging for large numbers of categories (20 or more).

• May obtain low association despite significant results.

Fisher’s Exact Test

When the sample size is small, or some cells have a frequency zero or expected value <5—apply Fisher’s exact test. The test is more precise than the Chi-square. Still, the same is applicable only for 2 × 2 contingency tables—no Fisher test for more than two categories of the independent and dependent variable. Fisher’s exact test avoids flaws of Chi-square by not approximating the p-value from continuous distribution—it directly calculates the p-value from the data.

Post hoc Test

The Chi-squared test for three or more groups is a global test—it does not tell what rows and columns are significantly associated. The practitioner is often interested in knowing the association between specific groups. Many researchers undertake subjective inspection by eyeballing the data and then decide whether specific cells are different—not a reliable and valid method. Sharpe discussed four variations of post hoc tests after obtaining statistically significant omnibus Chi-squared test results; these are—(1) residual analysis, (2) comparing cells, (3) ransacking, and (4) partitioning.⁵ We will discuss only “partitioning” for its ease of understanding and application; interested readers can consult Sharpe for more detail about other methods.⁵ In the partition test, the researcher systematically partitions the original r × c contingency table into an orthogonal set of 2 × 2. (Table 1) displays the partitioning of a 3 × 3, and (Table 2) exhibits a 3 × 2 table. The number of partitions depends on the degree of freedom (df) for the original contingency table; a 3 × 2 contingency table with two df allows two partitions, and a 3 × 3 table with four df allows for four divisions. The sum of the likelihood ratio Chi-square value for partitioned tables will be equal to the original contingency table. A vital rule is that each marginal total of the original table must be a marginal total for one and only one sub-table.

Strength of Association

The Chi-squared test, Yates’ correction, and Fisher’s exact test only give statistical significance (p-value)—it does not tell the researcher about the strength of the relationship between variables. Φ and Cramer’s V are alternatives to the correlation coefficient (continuous data) for two nominal variables. The Φ coefficient for 2 × 2 and Cramer’s V for more than two rows and columns determine strengths of association––how strongly two categorical variables are associated. It ranges from 0 to 1, where, 0 indicates no association, and 1 indicates a perfect association between the two variables. The heuristic to interpret the association is—(1) <0.20, weak association, (2) 0.2–0.6 moderate association, and (3) >0.6, strong association.⁶ The Cramer’s V statistic doesn’t show direction. On a 2 × 2 table, Φ shows direction with a positive or negative sign, but directionality doesn’t make much sense in a larger table of nominal categories. Rcmdr does not calculate Φ and Cramer’s V—the interested reader can calculate the same from VASSAR university online calculator. (VassarStats: Website for Statistical Computation)

Chi-squared Test in Rcmdr

After opening Rcmdr, researchers can access the Chi-squared test in the menu “statistical analysis < discrete variables < enter and analyze two-way table (option 1) and create a two-way table and compare two proportions (Fisher’s exact test—option 2).” Option 1, displayed in (Fig. 1), needs summarized data to run a Chi-square comparison, and option 2, depicted in (Fig. 2), works with raw data. By default, option 1 displays two rows and columns—the user can change the same by dragging the square grid, as demonstrated in (Fig. 1). Bonferroni and Holm’s correction for pairwise (2 × 2) comparison is available only with raw data (option 2). Further, there is flexibility to run Chi-square with and without continuity correction with option 2, compared to mandatory continuity correction with option 1.

Reporting and Interpretation

We intend to find an association between designation (faculty, students, and research staff) and statistical anxiety (mild, moderate, and severe). For parsimony, we are reporting only Chi-square for the 3 × 3 table. On inspection, we found that all expected cell frequencies are more than five, but the sample size is relatively small—therefore, we applied the Chi-squared test with continuity correction. The result shows no statistically significant association between designation and statistical anxiety (p = 0.75). There was a weak association (Φ = 0.09) between designation and statistical anxiety—it is for demonstration only, and there is no need to report if the researcher does not find a statistically significant association.