Selection, training and calibration of sensory panelists

By MIFLORA M. GATCHALIAN, PhD.

CEO, Quality Partners Company Ltd.

Academician Emeritus, International Academy for Quality (IAQ)

Founding President, Philippine Association of Food Technologists, Inc.

Introduction

MEASUREMENT tools are most important in understanding product characteristics necessary in quality development. Particularly for the food industry, human perceptions that lead towards final judgement about a product under study, provide one of the best measures absolutely necessary in product development and quality monitoring efforts. To achieve accurate and reliable sensory evaluation results, proper selection, training and calibration of sensory panelists (SP) are very important activities which need to be properly executed. The well-trained and maintained SPs serve as the instruments which cannot be replaced by any objective or physical measuring tools utilised especially in food research and quality assurance. Furthermore, these “human instruments” have to be properly maintained and regularly calibrated to ensure that their performance remain accurate and reliable through time. Retention of properly selected, trained and calibrated SPs is thus, an imperative in the food enterprise.

Sensory evaluation concept and applications

The Institute of Food Technologists’ Sensory Evaluation Division defines sensory evaluation (S.E.) as “a scientific discipline used to evoke, measure, analyse and interpret sensations as they are perceived by the senses of sight, smell, taste, touch and hearing” (Prell,1976). As such, progress of the food industry underscores the importance of S.E. as follows: (a) S.E. plays a big role among the four major players in food manufacturing, namely Sales and Marketing (S&M), Research and Development (R&D), Quality Control (Q.C.) and Production; (b) use in development of food product profile through quantitative descriptive analysis (QDA); (c) necessary for measurement of customer acceptability which enables direct comparisons between competitors; and (d) S.E. serves as an important instrument in innovation activities.

Fig. 1. Role of sensory evaluation in the quality development cycle

Figure 1 shows S.E. (centre) as it is linked with the different stages of product quality development (Gatchalian and Brannan, 2011) which generally starts with: (a) “consumer preferences” at top of inner circle; connecting with (b) “product profile”; then with (c) “test procedures”; (d) quality control tests; (e) “system of recording and reporting” and (f) “trouble shooting”. The innermost arrows moves in reverse directions, since consumer complaint, implying “ trouble” requires review of all recorded observations and the other activities in the development cycle. Almost at every stage, measurements are related to product sensory attributes as they are perceived by the trained SPs (Gatchalian, 2018).

The outermost circle displays the four major players in the company engaged continually in product innovation, development and monitoring which require them to be in constant communication with each other, hence the two-way arrows. S & M. (I) are generally the ones in direct contact with the target or actual customers from whom they elicit information about product preferences; which are then reported to R & D (II) - who usually generates innovation

activities starting with “product profiling”. When developing product design, inputs from Q.C. (III) are solicited to facilitate development of process and product specifications. Likewise, Production (IV) gets into the picture to check on feasibility. Hence, the external two-way arrows indicate constant communication between the four major players, from product concept to product market performance. In all these activities, S.E. plays a very important role especially since the final judge of a food product is the consumer. It is the SP who studies the product at various stages of development. They help identify the characteristics desired by the target consumers and finally ensures that these are maintained or improved during product life.

Fig. 2 Sensory evaluation set-up at Singapore Polytechnic Research Centre (Gatchalian, 2018)

Figure 2 shows the basic structure and process necessary for the conduct of a scientific S.E. A major requirement is a well-designed S.E. laboratory with evaluation booths linked to the sample preparation area. To ensure proper focus on the product being evaluated the conditions for doing the S.E. process must be controlled starting from sample preparation set-up to the conditions inside the booths where temperature and light intensity are controlled.  As such, the SPs themselves must have the least variation between each other and this can only be achieved through proper selection, training and calibration.

Selection of sensory panelists

SPs serve as measuring instruments approximating that of an objective equipment having the qualities of accuracy, precesion and reliability. Therefore, there must be enough opportunities to determine if the SPs meet these basic requirements. Most of these can be obtained through proper selection processes which usually start with knowing who are interested to become a panelist. This is done through a survey among the employees that asks basic information about the person’s background and physical health conditions. Also, certain attitudes like “Willingness” to be called upon on scheduled dates plus their Availability whenever called. Once these are established other qualities like Capability to do serious evaluation work, Sensitivity to sensory stimulus and Reliability to respond and provide consistent results or in short, the acronym WACS-R. There are specific tests which can be utilised to determine the existence of these characteristics.

DOER is the acronym for the steps in the final selection process such as: Step1 - Determine discrimination capability; Step 2 – Observe performance in recognition tests for taste, odour, flavour and texture sensations; Step 3 – Examine the relative performance of a potential panelist compared with other candidates; Step 4- Review the potential SP’s personal qualities. There are SE methods useful in the selection process, like for instance, the “triangle test” most useful among “discrimination tests” (Castura and Franczak. 2017). Figure 3 shows blueberry preserves in three coded cups two of which are the standard products or control and the other is with higher sugar level or the odd sample. The potential SP is asked to identify the odd sample in at least three trials. Correct identification means success as far as sweetness perception is concerned. Another tool is “Recognition test” where the potential SP is asked to identify either flavour or odour of coded samples with different levels of stimulus concentration.

Fig.3. Triangle test, an example of “discrimination test”

All SE test results are analysed and scores ranked from the highest to the lowest for all tests undergone by the potential SP. It is recommended that two or more times the number of required SP should be encouraged to participate during the selection process for assurance that the desired number is met. Based on the rankings, identify the highest performers who could fill-up the required number, then check the personal qualities. Those who could not make it to the cut-off point are thanked and if possible given some tokens to keep their interest and should there be another call for applicants, they could be candidates.

Training of sensory panelists

After the completion of the selection process, it is important that plans for enhancing the chosen ones’ capabilities will have to be laid down to ensure continuous improvement as well as maintain their interest. CARE is the acronym for the importance of training as follows: (a) Continuous improvement of one’s performance is imperative in order to sustain interest and active participation; (b) Affords the panelist a greater experience and understanding of his role in the programme; (c) Refresher for “old” or experienced SPs who join the training sessions; and (d) Evaluation of performance can also be done at this time. During training, identification of those with high potentials for more sophisticated tests like the Quantitative Descriptive Analysis or QDA approach can be made (O’Sullivan, 2017).

The 5Ds are activities planned into the SP training programme: (D1) Development of the capability to identify different odour sensations; (D2) Determination of various thresholds for different taste sensations; (D3) Determination of the capability to recognise basic taste sensations at threshold levels of experienced SPs; (D4) Development of the capability to accurately match color sensations with varying concentrations of color stimuli in solution; and (D5) Discovering panelist’s precision in the evaluation of  flavour stimuli with varying intensities.

There are basic guidelines for training SPs which are expected to provide efficiency and to ensure sustainability of interest using acronym PLOT. These are done in two parts as follows: Part I : (a) Plan for a well-organised approach to evaluation of SP performance; (b) Lay down proper conditions for understanding panelist’s role : (3) Observe and identify problem areas associated with the evaluation process; and (4) Train SPs to ensure continual improvement in their sensitivity and efficiency, both of which add to SPs’ self- confidence.

Part II - Analyse and interpret test results from Part I and utilise the outcome to determine SP performance level with focus on those requiring additional training and those who can be candidates for higher level sensory tests. Steps are as follows: (1) Analyse overall performance of SP by ranking them according to their scores in test results; (2) Rank the Panelists from highest to lowest performer and also determine the tests where most Panelists failed; and (3) Use the final outcome of the tests to identify the SPs who would need further training to achieve continual improvement. High performers can be developed for participation in more sophisticated sensory tests

Calibration of sensory panelists

For Sensory evaluation, Calibration is described as the process of making a comparison between the performance of a trained or experienced SP and the performance of a newly recruited SP. In Calibration, activities mostly involve monitoring of new SP performance in comparison with the outputs of the more experienced ones and results are tested statistically to establish significant differences. The “paired difference” test is done, for example, when it is desired to know if a significant difference exists between a new SP performance compared with that of a trained SP. The comparative performance of a new SP versus an existing trained SP could be seen from the difference in their respective scores for the same product in ten replications. Table 1 shows data from a paired comparison test, partially showing an approach to analysis utilising the t-test (Gatchalian and Brannan, 2011).

Table 1. Sample data from results of paired comparison test

Utilising the data in Table 1, a t-test for paired difference was done and results show that the new SP (A) gave a significantly higher score than the trained SP (B), implying the need for further training of the new SP until significant differences will no longer be obtained.

Summary

Measurement of sensory attributes of food products can be done through the use of different sensory evaluation methods. Such measures are most important in product development and quality monitoring activities in the food company. Those responsible for performing the evaluation processes are trained sensory panelists who had undergone the stages of selection, training and calibration. They would then compose the food Company’s pool of Sensory Panelists, very important “instruments” in the measurement of product sensory qualities and whose capabilities must be maintained through time employing regular training and calibration programmes.

Literature cited

Castura, John C. and Brian C. Franczak. 2017. Statistics for Use in Discrimination Testing, Discrimination Testing in Sensory Science, 10.1016/B978-0-08- 10100 9.000022.

Gatchalian, M.M. 2018. Innovation management in food product development. Food Pacific Manufacturing Journal. Vol. XVIII NO. 2, (March) ISSN 1608-7100. Ringier Trade Media Ltd. Hongkong

Gatchalian, M.M. 2019. Sensory Evaluation: A Must in Food Innovation. Proceedings. Food and Beverage Asia Summit. August 29, 2019. Ringier Trade Media Ltd. Hongkong

Gatchalian, M. M. and Brannan, Grace D. 2011. Sensory Quality Measurement: Statistical Analysis of Human Responses, 3^rd edition. Quality Partners Company, Ltd. 283pp. Quezon City Philippines; ISBN 978-971-691-921-9.  

O’Sullivan, Maurice G. 2017. A Handbook for Sensory and Consumer -Driven New Product Development: Innovative Technologies for the Food and Beverage Industry. Woodhead Publishing Series in Food Science, Technology and Nutrition. Elsevier Ltd.

Prell, Patricia. 1976. Preparation of Reports and Manuscripts which include Sensory Evaluation Data. Food Technology. Reprint from P. Prell.