This is an excerpt from Measurement in Sport and Exercise Psychology With Web Resource by Gershon Tenenbaum,Robert Eklund & Aki Kamata.
Evaluation of Measures of Intrinsic and Extrinsic Motivation in Sport and Exercise
In this section, a critical review of the different measures used to assess intrinsic and extrinsic motivation in sport and exercise research is conducted. Certain criteria have guided the selection of the measures presented in this section. First, we have selected measures that are fully developed instruments that have gone through extensive validation steps. Second, we have chosen scales that have been used in research, published or unpublished, during the past 10 years. Scales that have not been used during that time frame are considered to be obsolete and are not reviewed. Finally, in light of recent theoretical development and because of space limitation, we have focused on motivation scales that assess intrinsic and extrinsic motivation independently of determinants and outcomes, while focusing on the perceived reasons of behavior. Our earlier discussion on the definitions of intrinsic and extrinsic motivation makes it possible to classify the different measures. The measures can vary in terms of the level of generality (situational versus contextual level) and the area (sport versus exercise). This classification appears in table 25.1. Table 25.2 (see p. 291) provides additional information on the concept of, dimensions of, publication source of, and where to obtain the scale. As can be seen, seven measures are reviewed. For each one, we present (a) a description of the instrument, (b) the conceptual and theoretical rationale underlying its scale development, (c) the available evidence concerning its psychometric properties (e.g., factorial validity, reliability, and construct validity), and (d) a broad assessment of the strengths and weaknesses associated with each measure.
Measures Used in Sport
In this section, we review the SMS (Brière et al., 1995; Pelletier et al., 1995), the Sport Motivation Scale-6 (SMS-6; Mallett, Kawabata, Newcombe, Otero-Rorero, & Jackson, 2007), the Behavioral Regulation in Sport Questionnaire (BRSQ; Lonsdale, Hodge, & Rose, 2008), the Pictorial Motivation Scale (PMS; Reid, Vallerand, Poulin, Crocker, & Farrell, 2009), and the SIMS (Guay et al., 2000).
Sport Motivation Scale
The SMS was developed (Brière et al., 1995; Pelletier et al., 1995) in order to assess contextual intrinsic and extrinsic motivation from a multidimensional perspective, as well as amotivation. The SMS has been the most often used motivation measure in sport, being employed with a variety of athletes (recreational to elite), age groups (adolescent to senior), and cultures (e.g., Canada, United States, United Kingdom, Bulgaria, Australia, Spain, and New Zealand). In fact, the SMS has been translated and validated in several languages (see Pelletier & Sarrazin, 2007). The SMS is based on SDT (Deci & Ryan, 1985) and is made up of seven subscales assessing amotivation; external, introjected, and identified regulation; and intrinsic motivation to know, to experience stimulation, and to accomplish. In line with SDT, motivation is assessed as the perceived reasons for participation, or the why of behavior. At the beginning of the scale, participants are asked, “In general, why do you practice your sport?” The items represent the perceived reasons for engaging in the activity, thus reflecting the different types of motivation.
The original scale was developed in French as L’Échelle de Motivation dans les Sports (Brière, Vallerand, Blais, & Pelletier, 1995) and was validated in three steps. The first step involved generating a pool of items explaining various reasons for sport participation through interviews with French Canadian athletes (aged 17-20 y). These reasons were then used to formulate items for the seven subscales of the French SMS. In the second step, a committee of experts evaluated the content validity of the items and eliminated those that were thought to be inadequate. Another sample of athletes from various sports completed the scale. Results from an exploratory factor analysis (EFA) provided support for a seven-factor structure with 4 items per subscale; this second step thus resulted in a 28-item scale. In the third and final step, two additional studies were conducted to further validate the scale. These studies included approximately 500 individuals, most of whom were involved in recreational sports. Results from confirmatory factor analyses (CFA) and correlational analyses confirmed the seven-factor structure, the subscale internal consistency (ranging from .65-.96), and moderate to high indexes of temporal stability (ranging from .54-.82) over 1 month. Furthermore, inspection of correlations among the seven SMS subscales provided support for the simplex pattern proposed by SDT. Results of correlations also showed that (in line with SDT) the most self-determined forms of motivation (intrinsic motivation and identified regulation) were related more strongly to determinants such as autonomy support from coaches and feelings of competence than to other forms of motivation (external and introjected regulation) and amotivation. Similar results were obtained with motivational outcomes such as positive affect, concentration, and intentions to pursue engagement in sport. In sum, adequate construct validity was obtained for the French form of the SMS.
The translation of the French SMS into English involved back-translation and committee procedures as suggested by Vallerand (1989). Pelletier and colleagues (1995) conducted two studies involving college athletes from various sports in order to assess the psychometric properties of the English form of the SMS. Results from CFA with a sample of 593 Canadian university athletes revealed adequate fit indices or the hypothesized seven-factor model (see the Adjusted Goodness of Fit Index and the Normed Fit Index both > .90 and the Root Mean Square Residual < .08), and correlations with determinants and outcomes supported the simplex model. Moreover, internal consistency above .70 was obtained on all of the subscales except the identified subscale (.63). Test-retest correlations were acceptable and very similar to those obtained with the French SMS, as was the scale construct validity.
Since 1995, the SMS has been used extensively in sport psychology research. The seven-factor structure has been supported repeatedly (e.g., Doganis, 2000; Gillet, Vallerand, & Rosnet, 2009; Li & Harmer, 1996; Shaw, Ostrow, & Beckstead, 2005; Standage, Duda, & Ntoumanis, 2003). In addition, Hu and Bentler (1999) obtained support for a five-factor model by combining the three types of intrinsic motivation into one factor. Similar results were obtained by Gillet and colleagues (2009) with the French SMS. However, some studies have not supported the seven-factor model (Hodge, Allen, & Smellie, 2008; Mallett, Kawabata, & Newcombe, 2007; Mallett, Kawabata, Newcombe, & Otero-Rorero, 2007; Martens & Webber, 2002). Why is there such a discrepancy between these two sets of studies? One possibility lies in the populations from which the different samples were taken. Specifically, the SMS was validated using adolescent and young adult athletes and not older athletes. Because of this specific focus, some of the items may reflect a participation rather than an elite orientation, which is more in line with the younger population. For instance, an identified regulation item reads, “Because sport is one of the best ways to maintain good relationships with my friends.” Such an item seems more relevant for a younger population. An older, high-level athlete may disagree with this item but still display a high level of identified regulation for a sport (but not for relationship reasons). Future research using the SMS with different age groups and proficiency levels is needed to clarify this issue.
Whereas the internal consistency of the SMS has systematically shown adequate values, some values below .70 have been found. This is especially the case for the identified regulation subscale (Brière et al., 1995; Kingston, Horrocks, & Hanton, 2006; Li & Harmer, 1996; Pelletier et al., 1995), although some lower values (below .70) have been obtained with the introjected (McNeill & Wang, 2005; Perreault & Vallerand, 2007; Riemer, Fink, & Fitzgerald, 2002; Standage, Duda, & Ntoumanis, 2003) and external regulation (Standage, Duda, & Ntoumanis, 2003) and amotivation subscales (Standage, Duda, & Ntoumanis, 2003). However, very few instances of values below .60 have been obtained. It should be noted that a Cronbach alpha of .60 with only 4 items is acceptable because, as noted by Cronbach (1951), the coefficient alpha underestimates the internal consistency of scales with a low number of items. This is because the coefficient alpha includes the number of items in the formula. For instance, given the same average interitem correlation, a 3-item scale coefficient alpha value of .56 is equivalent to an alpha value of .81 on an 8-item scale!
In line with the original work of Ryan and Connell (1989) and the initial SMS validation procedures (Brière et al., 1995; Pelletier et al., 1995), construct validity has been assessed by other authors in two fashions: (1) with the simplex pattern of correlations among the subscales and (2) with correlations between motivational factors and their determinants and consequences. We do not have space to review all studies. However, overall, there is overwhelming support for the construct validity of the SMS both in French and English. For instance, in addition to finding support for the simplex pattern, Pelletier and Sarrazin (2007) concluded in their review of the evidence that the SMS has been used with success to predict a great variety of specific outcomes and consequences (such as burnout, exercise dependence among endurance athletes, fear of failing, adaptive coping skills, perceptions of constraints, flow, vitality and well-being, sporting behavior orientations, aggression, and performance) in a manner that is consistent with SDT. These findings provide strong support for the construct validity of the SMS.
In sum, the SMS has some positive features. First, it is a multidimensional instrument that assesses different types of intrinsic and extrinsic motivation as well as amotivation. Second, the scale focuses on the why of behavior and thus items are not confounded with determinants and consequences. Finally, it has some excellent psychometric properties. Nevertheless, some limitations should be underscored. First, although internal consistency levels have been acceptable overall, some subscales, especially the identified regulation subscale, have yielded relatively low coefficient alphas at times. Second, the SMS does not assess integrated regulation. Third, the seven-factor structure has not always been supported by CFAs. According to Pelletier, Vallerand, and Sarrazin (2007), this may be explained by a host of factors, including differences in sample sizes, variations in the way the instrument is administrated, or some other characteristics specific to the context of the study. However, as already indicated, it is also possible that the SMS is better suited for a younger, nonelite athlete population. Clearly, future research on this issue is in order.
Sport Motivation Scale-6
Mallett, Kawabata, Newcombe, and Otero-Rorero (2007) developed another version of the SMS, the SMS-6. This scale has the same underlying rationale that the original SMS scale but was designed to improve the original version of the SMS by including an integrated regulation subscale and attempting to solve some of the inconsistencies with the factor structure and some of the relatively low internal consistency values (below .70). The SMS-6 comprises 24 items, 4 for each of the six subscales, which include amotivation; external, introjected, identified, and integrated regulation; and general intrinsic motivation. Mallett, Kawabata, Newcombe, and Otero-Rorero (2007) developed 5 items for the integrated regulation subscale as well as 7 other items (4 of which were kept in the final scale) to replace some items in the original SMS. Two samples were used to validate the SMS-6. Sample 1 was composed of 501 first-year university students participating in competitive sport at least twice per week and 113 elite athletes representing Australia at the international level (for a total of 614 participants). Sample 1 was used to derive a factor structure that included the SMS items as well as the reformulated and integrated regulation items. Sample 2 was composed of 557 university students who were engaged in a variety of sports or physical activities twice per week. The second sample was used to confirm the structure of the SMS-6. Participants also completed the Dispositional Flow Scale (DFS).
Results of a CFA with the SMS-6 (with sample 2) provided support for the factor structure as well as for the internal consistency values (all above .70). Concerning the construct validity of the SMS-6, Mallett, Kawabata, Newcombe, and Otero-Rorero (2007) reported a rather weak simplex pattern of correlations among the subscales. More specifically, external regulation correlated highly with intrinsic motivation (r = .54), while the correlation between identified regulation and intrinsic motivation was very high (r = .91) and was higher than the one between integrated regulation and intrinsic motivation (r = .75). The construct validity of the SMS-6 was not fully supported, as some of the correlations involving the SMS and flow were not as expected by SDT. For instance, the distinctions among integrated regulation, identified regulation, and intrinsic motivation were not always clear. Furthermore, external regulation revealed some positive and sometimes strong correlations with flow, contrary to hypotheses derived from SDT.
In sum, the SMS-6 contains some nice features. First, it contains an integrated regulation subscale. Furthermore, the addition of 4 new items may make the SMS more acceptable for older and more experienced athletes. Second, Mallett, Kawabata, Newcombe, and Otero-Rorero (2007) presented results supporting the validity of a variation of the SMS-6, the SMS-8. The SMS-8 contains the same items that the SMS-6 contains but assesses the three types of intrinsic motivation rather than general intrinsic motivation. The SMS-6 also shows some limitations. First, Mallett, Kawabata, Newcombe, and Otero-Rorero (2007) proposed 7 new items to replace those that were presumably problematic in the original SMS. However, only 4 of these items made it to the final version. Thus, it appears that the SMS-6 retained much of the original SMS. Second, even some of the new items appear problematic and may not assess the desired construct (see Pelletier et al., 2007). For instance, a new amotivation item (“I don’t seem to be enjoying my sport as much as I previously did”) seems to reflect a decrease in intrinsic motivation rather than amotivation. Finally, results from Mallett, Kawabata, Newcombe, and Otero-Rorero (2007) demonstrated that the integrated regulation subscale may lack discriminant validity, leading to results with flow highly similar to identified regulation and intrinsic motivation.
Behavioral Regulation in Sport Questionnaire
Lonsdale and colleagues (2008) developed the BRSQ to create an alternative measure of elite sport motivation as conceptualized by SDT. However, in contrast to Mallett, Kawabata, Newcombe, and Otero-Rorero (2007), these authors used a complete new pool of items developed by SDT experts and competitive athletes. There are two versions of the BRSQ. The BRSQ-8 contains 32 items assessing integrated, identified, introjected, and external regulation; amotivation; and the three forms of intrinsic motivation (knowledge, experience stimulation, and accomplishment) identified by Vallerand (1997). The BRSQ-6 contains the same items but assesses general intrinsic motivation rather than all three types of intrinsic motivation, for a total of 24 items.
Lonsdale and colleagues (2008) conducted a series of three studies to validate the scale. In the first study, the factorial validity and the internal consistency were assessed with 382 New Zealand elite athletes. Results from a CFA on the 32 items supported the factor structure of the BRSQ. Specifically, fit indexes were acceptable and all items loaded significantly on the appropriate factors (they ranged from .58-.91). Finally, internal consistency of the eight subscales, measured with the Cronbach alpha, showed high values ranging from .71 to 91. Additionally, 1 wk test-retest reliability was tested with 34 competitive adult athletes. Coefficient alphas for all subscales supported the temporal reliability (values ranged from .73-.90).
In a second study with 343 athletes from New Zealand, the results of a CFA on the BRSQ-8 supported once more the factor structure as well as the subscale internal consistency. Lonsdale and colleagues (2008) also showed that the factor structure of the BRSQ-6 model fit the data very well and that subscale coefficient alphas all exceeded .78. Moreover, the construct validity of the BRSQ-6 was assessed by testing for a simplex pattern of correlations among the six subscales. While some relationships were in line with predictions (e.g., amotivation was negatively related to intrinsic motivation), there was a lack of discrimination between some subscales. More specifically, there was no difference between external and introjected regulation scores in terms of their relationships with amotivation. A similar pattern was evident with the identified and integrated regulation subscales, which both had similar high correlations with intrinsic motivation. These results with the simplex pattern were replicated in a third study conducted with nonelite athletes. In this third study, Lonsdale and colleagues also assessed the relationships between the BRSQ-6 and indexes of burnout (Lemyre, Treasure, & Roberts, 2006; Raedeke & Smith, 2001) and flow (Jackson & Eklund, 2002). Overall, results supported hypotheses in line with SDT. Specifically, amotivation and external and introjected regulation showed negative correlations with flow and positive correlations with burnout. The opposite pattern of correlations was found for the self-determined subscales (intrinsic motivation and identified and integrated regulation). However, there was a lack of discrimination between integrated regulation and general intrinsic motivation. Results of another study on burnout (Lonsdale, Hodge, & Rose, 2009) replicated these findings. Thus, overall, the support for the construct validity of the BRSQ-6 appears to be mixed.
It should be underscored that the BRSQ has some nice features. First, the scale is designed in such a way that the researcher can decide to use a multidimensional (BRSQ-8) or unitary (BRSQ-6) conceptualization of intrinsic motivation. Second, the scale is rather short, with 4 items per subscale. Finally, it assesses integrated regulation. At the same time, the BRSQ also displays some limitations. First, additional research is needed on the construct validity of the scale. Whereas there is support distinguishing the self-determined subscales (intrinsic motivation and identified and integrated regulation) from the non-self-determined subscales (external and introjected regulation), the finer discrimination within each type of category appears to be lacking. Such evidence is crucial, and future research is needed in order to show that this scale does indeed assess the SDT constructs rather than two broad sets of subscales tapping self-determined versus non-self-determined motivation. Second, this scale is designed specifically for older participants in competitive sport; it remains to be seen if the BRSQ can be used with younger participants, for whom the integrated regulation subscale may not have full meaning. Finally, research is needed to test the temporal stability of the scale over a time framed longer than 1 week.
Pictorial Motivation Scale
The PMS was designed to measure intrinsic and extrinsic motivation for sport and exercise in people with an intellectual disability. It assesses participants’ reasons for engaging in sport and exercise. The scale’s main characteristics are drawings depicting each of the 20 items. There are 5 items (pictures) for each of four subscales: intrinsic motivation, self-determined extrinsic motivation (a mixture of integrated and identified regulation), non-self-determined extrinsic motivation (a mixture of introjected and external regulation), and amotivation. These pictures are used to help participants with cognitive difficulties and to help represent the motivational concept depicted in each item.
The original scale was developed in French (Reid, Poulin, & Vallerand, 1994). Results of a study with 62 participants supported the internal consistency, temporal stability, and construct validity, as exemplified by the presence of a simplex pattern among the four subscales. However, the amotivation subscale had poor reliability (α = .52). The French version (Reid et al., 2009) was translated into English according to the back-translation and committee procedures outlined in Vallerand (1989). Then, 6 new items were generated for the less reliable amotivation subscale. Participants in the Special Olympics (n = 160) completed the English version. Results of the CFA confirmed the four-factor structure of the PMS. Furthermore, the internal consistency (Cronbach alphas) ranged from .60 to .71. Finally, the construct validity was assessed by testing for a simplex pattern of correlations among the four subscales. The intercorrelations among latent variables from the CFA provided support for the simplex pattern.
Results from a study conducted with the English version of the PMS involving 80 high school students with mild intellectual disability provided support for the internal consistency, temporal stability (over 3 wk), and construct validity of the PMS with respect to the simplex pattern of correlations among the PMS subscales as well as correlations between the PMS subscales and motivational antecedents (skill and perceived competence) and outcomes (perceived effort) as rated by the physical education teacher. Finally, the internal consistency of each subscale was tested without the pictorial dimension with a subset of 47 high school students with mild intellectual disability. Results indicated poor internal consistency (.91 for intrinsic motivation, .27 for self-determined extrinsic motivation, .20 for non-self-determined extrinsic motivation, and .60 for amotivation). This finding suggests that the scale is not reliable without the drawings.
The preliminary findings with the English version of the PMS are encouraging. Furthermore, this scale is the only one geared for individuals with intellectually disability. The use of drawings to depict the various items makes this scale unique in the field. Nevertheless, the PMS shows some limitations. First, the scale does not differentiate among all forms of intrinsic (knowledge, stimulation, and accomplishment) or extrinsic ( integrated, identified, introjected, and external regulation) motivation. Second, construct validity was tested with only a limited number of variables. Third, it is not known if the scale is usable with children who have severe forms of intellectual disabilities. Clearly additional research is needed on the reliability and validity of the PMS.
Situational Motivation Scale
The SIMS is one of the few scales to assess intrinsic and extrinsic motivation and amotivation at the situational level (Guay et al., 2000). The SIMS is a multidimensional tool that measures four types of motivation: intrinsic motivation, identified regulation, external regulation, and amotivation. The SIMS is made up of 16 items (4 items per subscale) and asks this question: “Why are you currently engaged in this activity?” The items represent potential reasons for task engagement. The scale is worded in such a way that it can be used in most situations (sport and nonsport).
Five studies were reported in the original article. In study 1, the original scale was developed by a committee of experts and completed by 195 French Canadian college students. Results of an EFA revealed a four-factor structure with the final 16 items loading on their respective factor. In study 2, a CFA confirmed the factor structure as well as its invariance across gender. Across the five studies, the internal consistency values of the subscales were acceptable, ranging from .62 to .95 (see Guay et al., 2000). Moreover, across all studies, support was obtained for the construct validity of the SIMS through results from correlations in line with the simplex pattern among the subscales as well as between the SIMS subscales and motivational determinants and consequences. Perhaps of greater interest for the present discussion were the results of study 4, which showed that some subscales (intrinsic motivation and identified regulation) were sensitive enough to detect changes in motivation that took place during two games of a basketball tournament.
Other researchers have also obtained support for the psychometric properties of the SIMS. First, all studies reported acceptable internal consistency values for each subscale (Blanchard, Mask, Vallerand, de la Sablonnière, & Provencher, 2007; Conroy, Coatsworth, & Kaye, 2007; Law & Ste-Marie, 2005; Ntoumanis & Blaymires, 2003; Standage, Treasure, Duda, & Prusak, 2003). The coefficient alpha values of all but the amotivation subscale (α = .58) in the Conroy and colleagues study were above .60. Second, support for the factorial validity of the SIMS was obtained through CFAs with one qualification. Whereas the CFA results with the 16 items yielded acceptable fit indexes, removal of 1 item (Jaakkola, Liukkonen, Laakso, & Ommundsen, 2008) and even 2 items (Gillet, Berjot, & Paty, 2009; Standage, Treasure, et al., 2003) yielded better fit indexes. Moreover, Standage, Treasure, and colleagues (2003) conducted multisample CFAs and showed that the pattern of factor loadings was largely invariant across four different samples.
Construct validity of the SIMS was also assessed in several studies (Blanchard et al., 2007; Conroy et al., 2007; Law & Ste-Marie, 2005; Ntoumanis & Blaymires, 2003; Standage, Treasure, et al., 2003). In addition to supporting the simplex pattern among the SIMS subscales and between the SIMS subscales and need satisfaction (study 2 of Blanchard and colleagues, 2007), results also supported the postulate from the HMIEM (Vallerand, 1997) for the top-down effect, in which contextual sport motivation was found to predict situational sport motivation (studies 1 and 2 of Blanchard et al., 2007; Jaakkola et al., 2008; Ntoumanis & Blaymires, 2003). Specifically, the more self-determined the motivation was found to be in a specific context (in this case, sport), the more self-determined the motivation was found to be in a given situation. Furthermore, Blanchard and colleagues (2007, studies 1 and 2) found support for another postulate from the HMIEM that suggests that over time, situational motivation in the realm of sport (basketball) has recursive effects on contextual motivation. The more that situational motivation is self-determined, the more that contextual motivation becomes self-determined over time. Finally, Jaakkola and coworkers (2008) demonstrated that, as predicted by the HMIEM, situational self-determined motivation was better than contextual motivation in predicting the situational intensity (as assessed by HR) displayed by students in a physical education class. Overall, these findings provide strong support for the reliability and factorial and construct validity of the SIMS.
The SIMS has several positive features, one of them being that it is the only scale to assess intrinsic and extrinsic motivation and amotivation at the situational level. Furthermore, it does so using only 16 items. Nevertheless, it also has some weaknesses. First, the SIMS does not assess the different types of intrinsic motivation and integrated and introjected regulation, because it was designed to be short. Second, while the factor structure has been supported, it is not clear if some items should be replaced (Gillet, Berjot, et al., 2009; Jaakkola et al., 2008; Standage, Treasure, et al., 2003). Third, research so far has not assessed the validity of the scale with high-performance athletes. Thus, additional research is needed to further test the psychometric properties of the SIMS in sport.
Learn more about Measurement in Sport and Exercise Psychology.