The Use of Sensory Difference

Tests to Investigate Perceptible

Colour-Difference in a Cosmetic
Product

Roger Whiting, * Shona Murray,

3

Zyrene Caintic, Kirsteen Ellison

AucklandUniversityofTechnology,PrivateBag92006,Auckland,NewZealand

4/43AlverstonSt,Waterview,Auckland,NewZealand

GloversFoodProcessors,P.O.Box97479,SouthAucklandMailCentre,Auckland,NewZealand

Received31August2002;revised27May2003;accepted8October2003

Abstract:Twosensorydifferencetestshavebeen
usedtoassesstheabilityofanuntrained
populationtoperceivecolourdifferenceina
cosmeticproduct.Thetwotestsusedwerethe
triangletestandthetwooutoffivetest.
Participantswerepresentedwithgroupsof
sampleswithvaryingcolourdifferencesand
askedtoidentifytheoddsampleinthetriangle
testandthepairinthetwooutoffivetest.

Fromthesedata,thenumberofcorrect
responseswascorrelatedwiththecalculated
colourdifferenceusingthreecolourdifference
equations(CMC,CIE94,andCIEDE2000).
Thesecorrelationswereoptimizedbyvarying
theparametersinthecolourdifference
equations.Withtheparametersoptimized,each
ofthethreecolourdifferenceequationsgavea
correlationcoefficientof0.97withthetwoout
offivetestandacorrelationcoefficientof0.79
withthetriangletest.Thesecorrelationcoeffi
cientssuggestthatsensorydifferencetestingcan
beusedtoinvestigateperceptionofcolour
difference.However,forthetriangletestthe
correlationbetweenthesensorydataandthe
calculatedcolourdifferenceisweakandthe
twooutoffivetestshouldbepreferred.The
minimumperceptiblecolourdifferencewas
estimatedfromtheregressionplotsbetweenthe

optimisedcolourdifferenceequationsandthe
sensorydata.
INTRODUCTION

2004WileyPeriodicals,Inc.ColResAppl,29,299
304,2004;Pub

lishedonlineinWileyInterScience
(www.interscience.wiley.com).DOI

10.1002/col.20025

*Correspondenceto:RogerWhiting,Auckland
UniversityofTechnology,PrivateBag92006,
Auckland,NewZealand(email:roger.
whiting@aut.ac.nz)

2004WileyPeriodicals,Inc.

Theuseofdifferencetestinghasbeenwell
establishedinthefieldofsensoryscience.1,2
Oneofthemostcommonlyusedofthese
differencetestsisthetriangletest.Inthe
triangletest,theparticipantorpanelistis
presentedwiththreeitems,twoofwhichare
identical,whereasthethirddifferent,andis
askedtoidentifytheodditem.Another
differencetestisthetwooutoffivetest,
wherethepanelistisgivenfivesamplesof
whichthreeformanidenticaltrioandthe
othertwoformanidenticalpairdifferentfrom
thetrio.Thepanelistisaskedtoidentifythe
pair.Thesesensorytestsaresimpleandrapid
toperform.Theycanbeusedwhenthe
differencebetweenthesamplesmaybethe
productofseveralvariablesbutassessingthe
contributionofeachvariabletotheperception
ofdifferencecanbedifficult.

Inrecentyears,therehasbeenasignificant
researchefforttoinvestigatetheperception
andcalculationofcolourdifference
(reviewedbyMelgosa3).Thelatest
developmenthasbeenthepublicationofthe
CIEDE2000colourdifferenceequation.4
Muchoftheresearchinthecolourdifference
fieldhasrevolvedaroundanumberofdata

Keywords:colordifference;perception;
sensorydifferencetests;liquidfoundation;
cosmetics

sets,notablytheRITDupontdataset,5which
isbasedonaluminiumplatescoatedwith
automotivelacquer,andtheLuoRiggdata
set,6whichisbasedondyedtextilesamples.
Themethodsusedtodevelopthedatasets
werethegrayscale7,8andthepair
comparison9methods.Bothofthese

Volume 29, Number 4, August 2004

299

methodsdifferconsiderablyfromthe
triangletestandthetwooutoffivetest.

Theaimofthisstudywastoinvestigate
whethercolourdifferencecouldbetreated
inasimilarwaytoothersensory
phenomena(e.g.,tasteandodor)andto
determineaminimumperceptiblecolour
differenceforthegeneralconsumerthat
couldbeusedtoassistthederivationofa
colourdifferencelimitforqualitycontrol
purposesincosmeticmanufacture.To
obtaindatarelevanttothegeneralpopu
lation,panelswerechosentobeuntrained
incolourmatching.Itshouldbenotedthat
anumberofthepreviousstudieshaveused
observerswithexperienceinthefieldof
colourmatching.

Astheresultsweretobeappliedinthe
cosmeticsindustry,samplesofliquid
foundationmakeupsealedinclearplastic
containerswereusedassamples.No
attemptwasmadetoviewthesamples
appliedtoskinasthisintroducedtoomany
variables.

METHODSANDMATERIALS

Samples

Samplesofliquidfoundationmakeupwere
obtainedfromalocalcosmetic
manufacturer.Althoughtheexactformula
tionoftheproductiscommercially
sensitivethemajorcomponentswerewater,
paraffinoil,emulsifiers,andpigments
(titaniumdioxide,ironoxides,andcarbon
black)formulatedfornonoilyCaucasian
skin.Thecoloursforthesesampleswere
measuredusingaMinoltaCR200bchroma
meter(usingD65illuminationandCIE1931

colourmatchingfunctions)andacuvettein
alighttightcontainertominimizeany
possibleeffectsoftranslucence.Tensam
plepairswithcolourdifferences[
E(CMC1:1)]between0.2and1.6were
selectedplusonepair(pairX)with
E(CMC1:1)5.2.Thecoloursofthesamples
rangedforL*from74to62,fora*from9
to14,andforb*from16to20.

Portionsofthesamplesweredispensedinto
50mmwaterwhiteplasticPetridishes,
whichwerecompletelyfilledandsealed
aroundtheedgewithclingfilm.Thisgave
asampledepthof8mm.FivePetridishes
werefilledfromeachsampletogiveapair
forthetriangletestandatripletforthetwo
outoffivetest.Colourmeasurementmade
onthesealedsamples(usingaLabex
spectrophotometer)indicatedthatthe
colourdifferenceofthesamplepairswas
unchangedbyenclosureinthePetridish.

Panelists

Seventyfourpanelists,whowererecruited
fromstaffandstudentsatAuckland
UniversityofTechnology,completedthe
sensorytests.Agesrangedfrom16to54
withequalnumbersofmalesandfemales.

ViewingConditions

Sampleswereviewedinasensorybooth
withgray(L*78.1,a*2.8,b*2.8)walls
andbase.IlluminationwasD65at1000lux.
Theviewingdistancewas60cmandeach
samplesubtendedanangleof6.Samples
werepresentedtothepanelistsonwhite
plastictrayswithgray(L*62.4,a*3.4,b*
3.1)nonwovenfabricoverthebottomof
thetray.Thesampleswereplacedinaline
touchingeachother(Fig.1)andattachedto
thetrayusingVelcro.Thearrangementof
thesamplesonthetraysisshowninFig.1.
Panelistsweregivenaresponseformthat
listedeachsetofsamplesinthesame
sequenceastheyappearedonthetray.

MechanicsoftheSensoryTests

Theprospectivepanelistswerefirst
requiredtosignaconsentformandthen
theyundertookacolourblindnesstest
(Ishihara).Panelistswithnormalcolour
visionwerethenaskedtoviewthe11
samplepairsthatwerepresentedwithinsets
of3forthetriangletestandthentoview
the11samplepairswithinsetsof5forthe
twooutoffivetest.Forbothtypesoftest,
thefirstsamplepairwaspairX[
E(CMC1:1)5.2].Thishelpedthepanelists
tounderstandthemechanicsofthetest.All
theresultswerediscardedforpanelists
whoscoredincorrectlyforpairX.

Eachpanelistwasgivenanindividual
responseformforthetriangletest.The
responseformshowedeachsetofthreePetri
dishesthatthepanelistwasgoingtoviewin
theorderthatthesetswouldbepresented(the
firstsettobeviewedwasatthetop).Each
Petridishhadanindividualrandomthree
digitnumberandontheresponseformeach
setof

FIG. 1. The arrangement of samples for

sensory tests.

300

COLOR research and application

TABLE 1. Sensory results for sample

pairs.

Two-out-of-five
Pair
The number of correct

E(CMC1:1)
Triangle test
test

choices of 74

1
0.2680
34
4
2
0.3772

33
12
3
0.4379
26
7
4
0.5674
42
25
5
0.7033
41
14
6
0.9077
55

threedisheswasshownastheindividual
numbersonthedishesmakingupthatset.
Theorder(lefttoright)ofthedishesonthe
traywasthesameasthatshownonthe
responseform.Thepanelistscircled,onthe
responseform,thenumberthatappearedon
thedishthatwasthesingleinthatset.On
completingonesetofthreedishes,the
panelistwaspresentedwithanotherset
untileachsamplepairhadbeenviewed.
Thetwooutoffivetestwasconductedina
similarmannerexceptthatthepanelistwas
requiredtocirclethenumbersappearingon
twodishesthatmadeupthepairineachset
of5.Overall,eachpanelistviewed22sets
andthistookapproximately15min.

15
7
0.7972

Randomization

48
21
8
1.0424
61
29
9
0.9765
40
33
10
1.6165

ThearrangementofthePetridishesinthe
triangletestsandinthetwooutoffive
testswasrandomized(usingpublished
tables1,2)toensurethatallpossible
arrangementsappearedwithequal
frequency.Inthetwooutoffivetestthe
numberofpossiblearrangementsexceeded
thenumberofpanelists,soonlysome
arrangementswereviewedandthesewere
viewedonlyonce.Withoneexception
(samplepairX,whichalwaysappeared
first)theorderinwhichthesamplepairs
werepresentedwasrandomized(usingpub
lishedtables)sothateachpanelistsawthe
pairsinadifferentorder.

53
35

StatisticalTreatment

Thevisualcolourdifferencesforthesample
pairswerecalculatedusingCMC,CIE94,and
CIEDE2000witharangeofvaluesforlandc

SensoryResults

intheCMCequationandforkLandkCinthe
CIE94andCIEDE2000equations.Thecor
relationofthevisualcolourdifferenceswith
thesensoryresultswasinvestigatedforeach

Theresultsofthesensorytestsareshownin
TableIalongwiththecolourdifference

setofparameters,landcorkLandkC,using
Minitab13.Thecolourdifferenceequations
wereoptimizedbyfindingthecombinations
ofparametersthatgavethelargest
correlationcoefficient.Thevaluesforlandc
orkLandkCwerevariedfrom0.5to2.0(or
2.5whereappropriate)instepsof0.5.The
valueofkHwaskeptat1.00forbothCIE94
andCIED2000.Aroundthemaxi

calculatedasE(CMC1:1).Thesensorytest
datafromTableIwerethencorrelatedwith
thevisualcolourdifferencescalculatedusing
CMC,CIE94,andCIEDE2000.The
parametersineachofthesecolourdifference
equationswerevariedtofindthecombination
ofparametersthatgavetheoptimum
correlationcoefficients.

CorrelationCoefficients
muminthecorrelationcoefficient,smaller
steps(0.25)weretriedinthevaried
parameterstoensurethatatruemaximum
wasfound.

Foreachcolourdifferenceequation,the
setofparametersthatgavethebest
correlationwiththesensorydatawasthen
investigatedfurther.Theresidualswere
testedtoensurethattherewereno
systematicdifferencesbetweenthe
regressionplotandthedata.Theregression
plotwasinspectedtofindthepointat
whichthesensorydataindicatedthatthere
wasa95%confidencethattherewasa
significanteffectbeingmeasured.

Statistically,with74panelists,thepointat
whicharealeffectisbeingobservedatthe
95%confidencelevelis34correctchoicesfor
thetriangletestand12correctchoicesforthe
twooutoffivetest.Usingtheequationsfor
theregressionplots,thesepoints(34or12
correctchoices)thengavethevisualcolour
differenceatwhichthereisa95%probability
thatsomeobserverscouldperceiveacolour
difference.

Themaximumcorrelationcoefficientforeach
colourdifferenceequationwitheachsensory
differencetestisshowninTableIIalongwith
thevaluesfortheappropriateparameters.

ThedatainTableIIindicatethereisaweak
positivecorrelationbetweenthetriangle
testsensorydataandthecolourdifferences.
However,inthecaseofthetwooutoffive
testthiscorrelationisstrong.Thusthetwo
outoffivetestisamoreappropriatetestto
applytocolourdifferenceperception.The
twooutoffivetesthasamuchlowerlike
lihood(P0.1)ofachievingacorrectchoice
bychance

TABLE II. Maximum correlation

coefficients between sensory test data
and calculated colour difference

Colour-difference
Correlation

RESULTSANDDISCUSSION
equation

coefficient

CIEDE2000

Equation parameters

0.78
kL1 and kC0.5

Two-out-of-five test
CMC(l:c)
0.98
Triangle test
l1.25 and c0.3
CIE94
CMC (l:c)
0.77

0.97
kL2.5 and kC0.4

l0.5 and c0.5

CIEDE2000
CIE94
0.97
0.79
kL1.25 and kC0.3
kl1.5 and kC1

Volume 29, Number 4, August 2004

301

comparedwiththetriangletest(P0.333)
andhenceisnormallyconsideredamore
selectivetest.

FittedLinePlots

Regressionplotsweregeneratedforthe
twooutoffivetestsensorydatawitheach
ofthecolourdifferenceequa

FIG. 2. Regression plots for two-out-of-five

test sensory data with optimized colourdifference equations.

tionsusingtheoptimumvalues(asshown
inTableII)fortheappropriateparameters.
Thesethreeregressionplotsareshownin
Fig.2.Astheoptimumcorrelationcoeffi
cientsbetweentheTriangletestsensory
dataandtheoptimizedcolourdifference
equationsweremuchpoorerthanforthe
twooutoffivetest,regressionplotsarenot
shown.

E
L2

C2
H
2

0.5

AnalysisofResiduals

Therearefourassumptionsimplicitinthe
regressionmodel:namely,alinear
relationshipbetweenxandy,aconstant
variationwithinthedata,anormal
distributionoftheresiduals,and
independentyvalues.Theresidualswere
plottedagainstthecolourdifferencesand
werefoundtoberandomlydistributedand
evenlyspread,therebyprovingthatthey
valueswereindependentwithconstant
variation.Thenormalprobabilityplots
(showninFig.3)gavePvalues0.05,which
indicatethatthedistributionoftheresiduals
isnormal.Withthreeofthefour
assumptionssatisfied,thehighcorrelation
coefficientsindicatethatthereisalinear
relationshipbetweenthecolourdifferences
andthesensorydata.

(1)

kLSL
MinimumPerceptibleColourDifference
kCSC
Minimumperceptiblecolourdifferences
werecalculatedfromtheequationsforthe
regressionplotsforeachoptimizedcolour
differenceequation.Theseareshownin
TableIII.Theoptimizedparametersfor
thesecolourdifferenceequationswere
unexpected.Thethreecolourdifference
equationshavethefollowinggeneralform:

kHSH

productandpurposeforwhichthecolour
differenceisbeingdetermined.Thelarger
thevaluethelessimportancethatparticular
attributehastothecalculatedcolour
withCIEDE2000,includinganangular

difference.ThevaluesforkL,kC,andare
indicatedinexpressionforthecolour

functionaswell.ThetermsSL,SC,andSh
arefunctionsoftheregionin

difference(e.g.,CIE94(1:1)usekL,kC,and

colourspacethatthesamplepairare
situated.TheydependonL,C,andHand
givetherelativeimportancethatdiffer
encesinlightness,chroma,andhuehaveto
havecolourdifferenceinthatregionof
colourspace.

perceptiblecolourdifference.10Theresults
ofthisstudysuggestagreateremphasison
chromaandlessonlightness.

ThetermskL,kC,andkHareweightingsfor
lightness,chroma,andhuefortheparticular

302

kHequalto1).TypicallyCMC(1:1)and
CIE94(1:1)havebeenindicatedas
appropriatecolourdifferenceequationsfor

Alsounexpectedwastherelativelysmall
value(0.46to0.75)oftheminimum
perceptiblecolourdifference.These
unexpectedvaluescouldbedueto
variationsinthewaythetermminimum
perceptiblecolourdifferenceisused.Here

COLOR research and application

thetermisusedtodescribethecolour
differenceatwhichthereisa95%
probabilitythatoneormoreuntrained
observerswilldetectacolourdifference,
whereasother

TABLE III. Minimum perceptible colour

differences for optimized colourdifference equations

Minimum perceptible

Equation
colour difference
Parameters

CMC (l : c)
0.75
l1.25 and c0.3

CIE94

CONCLUSION

0.46
kL2.5 and kC0.4
CIEDE2000
0.67
kL1.25 and kC0.3

workhasrelatedittocomparisonstoan
anchorcolourdifferencepair.6Another
possiblesourceofvariationintheminimum
perceptiblecolourdifferenceisthe
weightingsgiventotheL,C,andH
components.IfkL,kC,andkHwere,for
example,halved,thenthecorrelation
coefficientwouldremainunchangedbutthe
magnitudeoftheminimumperceptible
colourdifferencewouldbedoubled.

Itisalsonotablethatinthisstudythe
parametricfactorsofthetwooutoffive
testaredifferentfromthoseofthegray
scaleandthepaircomparisonmethods.In
thetwooutoffivetestthesamplesare
presentinawaythat,althoughtheytouch,a
clearlydefinedseparationisapparent
betweenthem.Thisclearlydefined
separationwouldnormallybeexpectedto
resultinacolourdifferencebeingmore
difficulttodetect.Henceitwouldbe
expectedthatalargerminimumperceptible
colourdifferencewouldresultfromthese

Fromthecorrelationcoefficientsbetween
thesensorydifferencetestdatawiththe
variouscolourdifferenceequationsitcan
beconcludedthatthetriangletestandthe
twooutoffivetestcanbeusedfortesting
perceptionofcolourdifference.Thetwo
outoffivetestgavemuchbettercorrelation
thanthetriangletestbetweenthesensory
dataandthecalculatedvisualcolour
differenceandwouldthereforebe
consideredthepreferredtestforvisual
colourdifference.Thespeedatwhichthe
testcanbeperformedwouldrecommendit
incomparisonwithmoretraditionaltech
niquesfordeterminingvisualcolour
difference.Inthepresentexperiment,22
colourpairswereassessedin15minwith
noreportedcasesofpanelistfatigue.

Therelativelylargevalueforthe
weightingsofLintheoptimizedcolour
differenceequationssuggeststhatthe
lightnessdifferenceisnotasimportantas
chromaorhuedifferencesintheperception
ofcolourdifferenceundertheconditions
usedintheexperiment.

Thecompanysupportingthisresearchis
nowusingthecolourdifferenceequation
CIEDE2000asameasureofcolour
differenceforqualityassurancepurposes.

differencetests.11

ACKNOWLEDGMENTS

FIG. 3. Normal probability plots for the twoout-of-five test sensory data with optimized
colour-difference equations.

WeacknowledgethesupportofAuckland
UniversityofTechnologythroughresearch
Grant99/21.Wealsothank

Volume 29, Number 4, August 2004

ourpanelistsforthetimeandinterestthey
putintothisstudy.

MeilgaardM,Civille,GV,CarrBT.Sensory
evaluationtechniques.BocaRaton:CRCPress;1991.
p6070.

LawlessHT,HeymannH.Sensoryevaluationoffood
principlesandpractices.NewYork:Chapmanand
Hall;1998.p152161.

303
LuoMR,RiggB.Chromaticitydiscriminationellipses
forsurfacecolours.ColorResAppl1986;11:2542.

LuoMR,RiggB.BFD(l:c)colourdifferenceformula
Part1devel

opmentoftheformula.JSocDyersColourists,
1987;103:8693.

CheungM,RiggB.Colourdifferenceellipsoidsfor
fivecolourcentres.ColorResAppl1986;11:185195.
MelgosaM.TestingCIELABbasedcolordifference
formulas.ColorResAppl2000;25:4955.

LuoMR,CuiG,Rigg,B.Thedevelopmentofthe
CIE2000colourdifferenceformula:CIEDE2000.
ColorResAppl2001;26:340350.

BernsRS,AlmanD,ReniffL,SnyderGD,Balenon
RosenMR.Visualdeterminationofsuprathreshold
colordifferencetolerancesusingprobitanalysis.
ColorResAppl1991;16:297316.

BernsRS.ThemathematicaldevelopmentofCIETC
129proposedcolordifferenceequation:CIELCH.
Colour93;TechnicalUniversityofBudapest,1993p
226.

BernsRS.Industrialapplicationsderiving
instrumentaltolerancesfrompassfailand
colorimetricdata.ColorResAppl1996;21:459472.

WittK.Parametriceffectsonsurfacecolordifference
evaluationatthreshold.ColorResAppl1990;15:189
199.

304

COLOR research and application