Comparison of Mental Workload with N-Back Test A New Design for NASA-Task Load Index Questionnaire

Document Type : Original Article


1 Department of Occupational Health, Kashan University of Medical Sciences, Kashan, Iran

2 Department of Ergonomics, School of Health, Shiraz University of Medical Sciences, Shiraz, Iran

3 Department of Occupational Health Engineering, School of Health, Shahid Beheshti University of Medical Sciences, Tehran, Iran

4 Department of Industrial Management, Faculty of Economic, Management and Social Science, Shiraz University, Shiraz, Iran

5 Department of Industrial Management, Faculty of Economic, Management and Social Science, Shiraz, Iran

6 Cardiovascular Diseases Research Center, Department of Cardiology, Heshmat Hospital, School of Medicine, Guilan University of Medical Sciences, Rasht, Iran



A most widely used tool for measuring the workload is the NASA-task load index (TLX) questionnaire, in which various studies have reported numerous problems. The present study aimed to improve the NSAS-TLX mental workload questionnaire using the fuzzy linguistic variables instead of the virtual rating scale, and multicriteria decision-making Fuzzy Best-Worst method (FBWM) instead of pair-wise comparison.
Materials and Methods: 
This cross-sectional study was carried out among students of Shiraz University of Medical Sciences. In order to compare the traditional NASA-TLX and FBWM NASA-TLX questionnaire, participants performed a standard N-Back task with three low, medium, and high workload, subsequently, they completed the two questionnaires. Finally, results were examined using the researcher-made software and SPSS 16.
With regard to the reaction time to response the N-Back test, the mean of correct responses were 107.43 and 85.86 responses at levels 1 and 3, respectively. The mean score of the subscales and the final score in the two questionnaires at different levels of the N-Back test were independent as follows: the two questionnaires had a significant difference on mental demand at level 3 with a mean (standard deviation [SD]) of 18.09 (6.39) in the FBWM-NASA-TLX questionnaire and mean (SD) of 22.64 (8.15) in the NASA-TLX questionnaire (P < 0.05).
In this study, the FBWM-NASA-TLX questionnaire was designed and studied with regard to the problems and limitations of the NASA-TLX questionnaire. Results of this study showed that the FBWM-NASA-TLX questionnaire can estimate more realistic scores and decisions of workload in the studied task.


1.     Backs RW, Ryan AM, Wilson GF. Psychophysiological measures of workload during continuous manual performance. Hum Factors 1994;36:514‑31.
2.     Hart SG, Staveland LE. Development of NASA‑TLX (Task Load Index): Results of empirical and theoretical research. Adv Psychol 1988;52:139‑83.
3.     Hart SG, editor NASA‑task load index (NASA‑TLX); 20 years laterProceedings of the Human Factors and Ergonomics Society Annual Meeting. Los Angeles, CA: SAGE Publications; 2006.
4.     Delice EK, Can GFJEOÜMvMFD. An Integrated mental workload assessment approach based on Nasa-TLX and SMAA-2: A case study. Eskişehir Osmangazi Üniversitesi Mühendislik ve Mimarlık Fakültesi Dergisi. 2018;26:88-99.
5.     Rubio S, Díaz E, Martín J, Puente JM. Evaluation of subjective mental workload: A comparison of SWAT, NASA-TLX, and workload profile methods. Appl Psychol 2004;53:61‑86.
6.     Parasuraman R, Hancock PA. Adaptive Control of Mental Workload. Stress, Workload, and Fatigue; 2001.
7.     Flemisch FO, Onken RJC, Technology, Work. Open a Window to the Cognitive Work Process! Pointillist Analysis of Man–Machine Interaction. 2002;4:160-70.
8.     Loft S, Sanderson P, Neal A, Mooij M. Modeling and predicting mental workload in en route air traffic control: Critical review and broader implications. Human Factors 2007;49:376‑99.
9.     Vidulich MA, Tsang PS. Methodological and theoretical concerns in multitask performance: A critique of boles, bursk, phillips, and perdelwitz. Hum Factors 2007;49:46‑9.
10.     Wickens CD. Multiple resources and mental workload. Hum Factors 2008;50:449‑55.
11.     Young MS, Brookhuis KA, Wickens CD, Hancock PA. State of science: Mental workload in ergonomics. Ergonomics 2015;58:1‑17.
12.     Yurko YY, Scerbo MW, Prabhu AS, Acker CE, Stefanidis D. Higher mental workload is associated with poorer laparoscopic performance as measured by the NASA‑TLX tool. Simul Healthc 2010;5:267‑71.
13.     Zheng B, Jiang X, Tien G, Meneghetti A, Panton ON, Atkins MS. Workload assessment of surgeons: Correlation between NASATLX and blinks. Surg Endosc 2012;26:2746‑50.
14.     Wiebe EN, Roberts E, Behrend TS. An examination of two mental workload measurement approaches to understanding multimedia learning. Comput Hum Behav 2010;26:474‑81.
15.     Malekpour F, Malekpour A, Mohammadian Y, Mohammadpour Y, Shakarami A, Sheikh Ahmadi A. Assessment of mental workload in nursing by using NASA‑TLX. J Urmia Nurs Midwif Facul2014;11:892‑9.
16.     Amady MM, Raufaste E, Prade H, Meyer JP, Fuzzy‑TL. Using fuzzy integrals for evaluating human mental workload with NASA‑Task Load indeX in laboratory and field studies. Ergonomics 2013;56:752‑63.
17.     Bridger RS, Brasher K. Cognitive task demands, self‑control demands and the mental well‑being of office workers. Ergonomics 2011;54:830‑9.
18.     Nygren TE. Psychometric properties of subjective workload measurement techniques: Implications for their use in the assessment of perceived mental workload. Human Factors 1991;33:17‑33.
19.     Hart SG. NASA Task load Index (TLX). Volume 1.0; Paper and pencil package. National Aeronautics and Space Administration 1986;2:10-5.
20.     Ruiz‑Rabelo JF, Rodriguez NE, Di‑Stasi LL, Jimenez ND, Bermon CJ, Iglesias DC, et al. Validation of the NASA‑TLX score in ongoing assessment of mental workload during a laparoscopic learning curve in bariatric surgery. Obes Surg 2015;25:2451‑6.
21.     Kane MJ, Conway AR, Miura TK, Colflesh GJ. Working memory, attention control, and the N‑back task: A question of construct validity. J Exp Psychol Learn Mem Cogn 2007;33:615‑22.
22.     Jaeggi SM, Buschkuehl M, Perrig WJ, Meier B. The concurrent validity of the N‑back task as a working memory measure. Memory 2010;18:394‑412.
23.     Li RJ. Fuzzy method in group decision making. Comput Math Appl 1999;38:91‑101.
24.     Guo S, Zhao H. Fuzzy best‑worst multi‑criteria decision‑making method and its applications. Knowl‑Based Syst 2017;121:23‑31.
25.     Rezaei J. Best‑worst multi‑criteria decision‑making method. Omega 2015;53:49‑57.
26.     Verhaeghen P, Basak C. Ageing and switching of the focus of attention in working memory: Results from a modified N‑Back task. Q J Exp Psychol 2005;58:134‑54.
27.     Straker L, Mathiassen SE. Increased physical work loads in modern work–a necessity for better health and performance? Ergonomics 2009;52:1215‑25.
28.     Choi B, Schnall PL, Yang H, Dobson M, Landsbergis P, Israel L, et al. Sedentary work, low physical job demand, and obesity in US workers. Am J Ind Med 2010;53:1088‑101.