Analysis of sensory ratings data with cumulative link models

Christensen, Rune Haubo Bojesen; Brockhoff, Per Bruun

Journal de la société française de statistique, Tome 154 (2013) no. 3, pp. 58-79.

Les données issues d’une étude hédonique ou de préférence sont généralement représentées avec une échelle à catégories ordonnées. Elles sont souvent analysées par des méthodes de régression linéaire ou des tests omnibus de Khi-deux de Pearson. Nous proposons dans cet article le recours à des modèles de régression à fonction de lien cumulée qui respectent la nature ordinale des observations. Nous décrivons comment ces modèles sont liés aux tests omnibus de Khi-deux, et comment ils peuvent conduire à des tests plus puissants en l’absence de répétitions. Pour les notations sur une échelle ordinale, nous présentons une approche de type maximum de quasi-vraisemblance et une approche de type « modèles mixtes » qui sont en fait des extensions du modèle à fonction de lien cumulée. Avec ces modèles nous comparons les interprétations de l’effet moyen et de l’effet spécifique du sujet, et nous discutons comment les différentes approches conduisent à différents tests. En présence de répétitions, les tests « naïfs » qui ignorent celles-ci sont souvent trop permissifs à cause de la sur-dispersion. Nous discutons aussi de la dépendance du plan expérimental, randomisé ou en blocs. Pour ces plans en blocs nous abordons la question de savoir comment les tests naïfs peuvent être plus puissants que les approches qui prennent en compte la sur-dispersion et comment les modèles mixtes peuvent fournir des tests encore plus puissants que des tests naïfs. Des exemples sont présentés tout au long de l’article. Les procédures d’analyse sont implémentées par les auteurs dans l’environnement R.

Examples of categorical rating scales include discrete preference, liking and hedonic rating scales. Data obtained on these scales are often analyzed with normal linear regression methods or with omnibus Pearson ${\chi }^2$ tests. In this paper we propose to use cumulative link models that allow for regression methods similar to linear models while respecting the categorical nature of the observations. We describe how cumulative link models are related to the omnibus ${\chi }^2$ tests and how they can lead to more powerful tests in the non-replicated setting. For replicated categorical ratings data we present a quasi-likelihood approach and a mixed effects approach both being extensions of cumulative link models. We contrast population-average and subject-specific interpretations based on these models and discuss how different approaches lead to different tests. In replicated settings, naive tests that ignore replications are often expected to be too liberal because of over-dispersion. We describe how this depends on whether the experimental design is fully randomized or blocked. For the latter situation we describe how naive tests can be stronger than over-dispersion adjusting approaches, and that mixed effects models can provide even stronger tests than naive tests. Examples will be given throughout the paper and the methodology is implemented in the authors’ free R-package ordinal.

Keywords: Cumulative link models, ordinal regression models, mixed effects models, R software
Mot clés : modèle à fonction de lien cumulée, modèle de régression ordinale, modèle mixte, logiciel R

[1] Agresti, Alan Categorical Data Analysis, Wiley, 2002 | MR | Zbl

[2] Agresti, Alan Modelling ordered categorical data: Recent advances and future challanges, Statistics in Medicine, Volume 18 (1999), pp. 2191-2207

[3] Agresti, Alan; Natarajan, Ranjini Modeling Clustered Ordered Categorical Data: A Survey, International Statistical Review, Volume 69 (2001), pp. 345-371 | Zbl

[4] Brockhoff, Per Bruun; Christensen, Rune Haubo Bojesen Thurstonian models for sensory discrimination tests as generalized linear models, Food Quality and Preference, Volume 21 (2010), pp. 330-338

[5] Brazzale, A R; Davison, A C; Reid, N Applied Asymptotics—case studies in small-sample statistics, Cambridge University Press, 2007 | MR | Zbl

[6] Bi, Jian Statistical models for the degree of difference test, Food Quality and Preference, Volume 13 (2002), pp. 13-37

[7] Bi, Jian Sensory Discrimination Tests and Measurements—Statistical Principles, Procedures and Tables, Blackwell Publishing, 2006 | Zbl

[8] Boyles, Russell A The Role of Likelihood in Interval Estimation, The American Statistician, Volume 62 (2008) no. 1, pp. 22-26 | MR

[9] Brockhoff, Per Bruun The statistical power of replications in difference tests, Food Quality and Preference, Volume 14 (2003), pp. 405-417

[10] Christensen, Rune Haubo Bojesen; Brockhoff, Per Bruun Estimation and Inference in the Same Different Test, Food Quality and Preference, Volume 20 (2009), pp. 514-524

[11] Christensen, Rune Haubo Bojesen; Cleaver, Graham; Brockhoff, Per Bruun Statistical and Thurstonian models for the A-not A protocol with and without sureness, Food Quality and Preference, Volume 22 (2011), pp. 542-549

[12] Christensen, Rune Haubo Bojesen ordinal—Regression Models for Ordinal Data, 2012 (R package version 2012.09-12 http://www.cran.r-project.org/package=ordinal/)

[13] Collett, David Modelling binary data, London: Chapman & Hall/CRC, 2002 | MR | Zbl

[14] Cox, Christopher Location-scale cumulative odds models for ordinal data: A generalized non-linear model approach, Statistics in medicine, Volume 14 (1995), pp. 1191-1203

[15] Cressie, Noel; Read, R C Pearson’s $X^2$ and the loglikelihood ratio statistic $G^2$: A comparative review, International Statistical Review, Volume 57 (1989) no. 1, pp. 19-43 | Zbl

[16] Catalano, Paul J; Ryan, Louise M Bivariate Latent Variable Models for Clustered Discrete and Continuous Outcomes, Journal of the American Statistical Association, Volume 87 (1992) no. 419, pp. 651-658 http://www.jstor.org/stable/2290200

[17] Diggle, Peter J; Heagerty, Patrick; Liang, Kung-Yee; Zeger, Scott L Analysis of Longitudinal Data, Oxford university Press, 2002 | MR

[18] Ennis, Daniel M; Bi, Jian The beta-binomial model: Accounting for inter-trial variation in replicated difference and preference tests, Journal of Sensory Studies, Volume 13 (1998), pp. 389-412

[19] Ennis, Daniel M; Bi, Jian The Dirichlet-multinomial model: Accounting for inter-trial variation in replicated ratings, Journal of Sensory Studies, Volume 14 (1999), pp. 321-345

[20] Fitzmaurice, Garrett; Davidian, Marie; Verbeke, Geert; Moleberghs, Geert Longitudinal Data Analysis, Chapman & Hall/CRC, 2009 | MR | Zbl

[21] Fahrmeir, Ludwig; Tutz, Gerhard Multivariate Statistical Modelling Based on Generalized Linear Models, Springer series in statistics, Springer-Verlag New York, Inc., 2001 | MR

[22] Genter, Frederic C; Farewell, Vernon T Goodness-of-link testing in ordinal regression models, The Canadian Journal of Statistics, Volume 13 (1985) no. 1, pp. 37-44 | Zbl

[23] Greene, William H; Hensher, David A Modeling Ordered Choices: A Primer, Cambridge University Press, 2010 | MR | Zbl

[24] Inc., SAS Institute SAS/STAT® 9.2 User’s Guide (2008)

[25] Irwin, R J; Stillman, J A; Hautus, M J; Huddleston, L M The measurement of taste discrimination with the same-different task: a detection-theory analysis, Journal of Sensory Studies, Volume 8 (1993), pp. 229-239

[26] Joe, Harry Accuracy of Laplace approximation for discrete response mixed models, Comput. Stat. Data Anal., Volume 52 (2008) no. 12, pp. 5066-5074 | DOI | MR | Zbl

[27] Lawless, H T; Heymann, H Sensory evaluation of food, Chapman and Hall, London, 1998

[28] Liu, Qing; Pierce, Donald A A note on Gauss-Hermite quadrature, Biometrika, Volume 81 (1994) no. 3, pp. 624-629 | MR | Zbl

[29] McCullagh, Peter Regression Models for Ordinal Data, Journal of the Royal Statistical Society, Series B, Volume 42 (1980), pp. 109-142 | MR | Zbl

[30] Macmillan, Niel A; Kaplan, Howard L; Creelman, C Douglas The Psychophysics of Categorical Perception, Psychological Review, Volume 84 (1977) no. 5, pp. 452-471

[31] McCullagh, Peter; Nelder, John Generalized Linear Models, Chapman & Hall/CRC, 1989 | MR

[32] Nair, V N Testing in Industrial Experiments With Ordered Categorical Data, Technometrics, Volume 28 (1986) no. 4, pp. 283-311 | MR | Zbl

[33] Pawitan, Yudi A reminder of the fallability of the Wald statistic: Likelihood explanation, The American Statistician, Volume 54 (2000) no. 1, pp. 54-56

[34] Pawitan, Yudi In All Likelihood—Statistical Modelling and Inference Using Likelihood, Oxford University Press, 2001 | Zbl

[35] Pinheiro, José C; Bates, Douglas M Approximations to the Nonlinear Mixed-Effects Model, Jounal of Computational and Graphical Statistics, Volume 4 (1995) no. 1, pp. 12-35

[36] R Development Core Team R: A Language and Environment for Statistical Computing (2010) http://www.R-project.org (ISBN 3-900051-07-0)

[37] Rayner, J C W; Best, D J A Contingency Table Approach to Nonparametric Testing, Chapman & Hall/CRC, 2001

[38] Rayner, J C W; Best, D J; Brockhoff, P B; Rayner, G D Nonparametrics for Sensory Science, A More Informative Approach, Blackwell Publishing, 2005 | Zbl

[39] Self, Steven G; Liang, Kung-Yee Asymptotic Properties of Maximum Likelihood Estimators and Likelihood Ratio Tests Under Nonstandard Conditions, Journal of the American Statistical Association, Volume 82 (1987) no. 398, pp. 605-610 | MR | Zbl

[40] Stram, D O; Lee, J W Variance Component Testing in the Longitudinal Mixed Effects Model, Biometrics, Volume 50 (1994), pp. 1171-1177 | Zbl

[41] Skrondal, Anders; Rabe-Hesketh, Sophia Generalized Latent Variable Modeling, Chapman & Hall/CRC, 2004 | MR | Zbl

[42] Thurstone, L L A Law of Comparative Judgment, Psychological Review, Volume 34 (1927), pp. 273-286

[43] Thurstone, L L Psychophysical analysis, American journal of Psychology, Volume 38 (1927), pp. 368-389

[44] Thurstone, L L Three psychological laws, Psychological Review, Volume 34 (1927), pp. 424-432

[45] Ten Have, Thomas R; Landis, J Richard; Hartzel, Jonathan Population-averaged and cluster-specific models for clustered ordinal response data, Statistics in Medicine, Volume 15 (1996), pp. 2573-2588

[46] Venables, W N; Ripley, B D Modern Applied Statistics with S, Springer, New York, 2002 http://www.stats.ox.ac.uk/pub/MASS4 (ISBN 0-387-95457-0) | Zbl

[47] Wasserman, Larry All of statistics: A concise cource in statistical inference, Springer, 2004 | MR | Zbl

[48] Wedderburn, R W M Quasi-likelihood functions, generalized linear models and the Gauss-Newton method, Biometrika, Volume 61 (1974), pp. 439-447 | MR | Zbl

[49] Zeger, S L; Liang, K-Y An overview of methods for the analysis of longitudinal data, Statistics in Medicine, Volume 11 (1992), pp. 1825-1839

[50] Zeger, S L; Liang, K-Y; Albert, P A Models for longitudinal data: a generalized estimating equation approach, Biometrics, Volume 44 (1988), pp. 1049-1060 | MR | Zbl