Content Specificity
last authored:
last reviewed:
Introduction
Content specificity is the phenomenon that performance on various types of problems differs because of the associated content, with individual variation depending on mastery of this content (Eva, Neville, and Norman, 1998). This is used to describe the correlation across problems of 0.1-0.3. It has also been termed 'case specificity' (Elstein, Shulman, and Sprafka, 1990). This content specificity is not confined to health care, research has long shown (Perkins and Salomon, 1989).
In short, problem-based learning skills are not generalizable. This has significant implications for medical education and assessment. PBL had a goal of development of problem-solving skills. This lack of generalizability does not appear to be primarily mediated by content, but rather on transfer.
Multiple sampling is required to demonstrate attainment of competence.
Difficulties with Content Specificity as an Explanation
There is abundant evidence that some factor(s) limits correlation on performance on assessment tools such as OSCE. However, the evidence that the discrepancy is due to content gaps is poor.
One ambitious study assessed clerks, residents, and staff on multiple test questions and OSCE stations using strategically designed clinical cases. Even when using the same case, presented by different actors, the correlations on performance ranged from 0.07 to 0.60 (Norman et al, 1985). Something further is happening.
Context specificity - all the variables that surround a given clinical problem - clearly has much role to play.
"a particular individual, on a particular occasion, may invoke a completely different strategy than when confronted with a similar problem on a second occasion" (Eva, Neville, and Norman, 1998). The clinical strategy chosen for a given problem may have a significant impact on performance (for more information, see clinical reasoning). Difficulties with solving a problem may therefore relate to difficulty with selecting and implementing a strategy.
It is unclear whether case-specificity particularly relates to content or topic specificity; it appears as though the type of content, or clinical processing strategy, may be more important (Dory, Gagnon, and Charlin, 2010).
An ideal number of items appears to be 2-3 per topic, as much variability comes from number of items (Norman et al, 2006).
Defining a Problem
Content can be of two main types - semantic and surface. Semantic knowledge relates to fields such as anatomy, physiology, or pathology, which surface knowledge relates to the specifics of a case "my pain is in my abdomen and is radiating to my groin". Underlying these are concepts or principles, referred to as schema or deep structure. Content and concepts are experienced within a specific context.
Well-functioning schemas are composed of relationships amongst knowledge.
Transfer
Transfer is the application of content or concepts learned in one setting to other problems. Research in the world of cognitive
analogy: recognition that principles
learned in one context are useful
in other similar contexts
psychology has examined transfer for many years (eg Gick and Holyoak, 1983), and demonstrated how fundamental the concept of analogy - the recognition that principles learned in one context are useful in other similar contexts - is. Critical factors in the use of an analogy have been identified as follows (Needham and Begg, 1993):
- encode the target problem appropriately
- retreive the encoded analogue
- select the appropriate analogue, amonst the other information in mind
- adapt the solution encoded in the analogue to the target problem
Improving the function of this transfer during training is of paramount importance.
Transfer between analogical problems is rare during early training, with learners tending to find connections between surface-level features. Accordingly, differences in surface features will make transfer very challenging. An expert, in contrast, has well-established schemas in place that permit cutting through the surface information in seeking to recognize the fundamental principles at work.
There are two main theories that explain how transfer works (Eva, Neville, and Norman, 1998). The first, abstract induction, states that exposure to multiple problems results in principles that are context-independent. The second, conservative induction, states that principles are context-bound, and that expertise results from developing the same principle in multiple contexts.
It seems likely that both surface and structural similarities go into the creation, comparison, and utilization of the fundamental analogues. Some argue that the surface features are what bring the analogues to mind, while use of solution strategies depends on structural similarity (Holyoak and Koh, 1987).
Facilitating Transfer
Transfer appears to become automatic when large numbers of base analogs are provided (Brown, Kane, and Echols, 1986). However, this is not ideal for two reasons: it is not feasible to go through every problem exhaustively, and this process does not seem to hold when context changes, or when there is a delay between training and test (Eva, Neville, and Norman, 1998).
Psychological context should be understood, and impeding factors (eg biases, faulty assumptions, etc) should be removed.
Prompting students to make explicit comparisons between analogs during training can help cement an understanding of underlying principles, and to apply them to a third. However, it appears that this needs to occur within a given time frame; waiting a week removed this effect (Catrambone and Holyoak, 1985). Instead, for longer-lasting effect, the creation of a comparison exercise that cued independent query of the important features seems helpful. Said another way, questions should lead to the identification of underlying structure.
Teachers should therefore provide prompts to guide comparisons and to encourage the reading of problems according to structure, based on analogues built from past examples (Eva, Neville, and Norman, 1998).
Other ways of improving transfer of analogues includes (Eva, Neville, and Norman, 1998, and references therein):
- working through the problem before being given the solution
- testing hypotheses
- teaching someone else
- constructing an explanation of the problem
- comparing two or more problems for similarity
The benefit of these processes is magnified through feedback (Needham and Begg, 1993).
Next, it appears that presentation of principles and examples are the most beneficial. Presenting examples in blocks, with similar items together, appears more helpful, perhaps due to improved understanding of underlying principles. One good way of presenting sequences is to begin with ideal positive examples, then ideal negative examples, and finally borderline cases (Avrahami et al, 1997).
Resources and References
Brown AL, Kane MJ, Echols CH. 1986. Young children's mental models determine analogical transfer across problems with a common goal structure. Cog Develop. 1:103-121.
Dory V, Gagnon R, Charlin B. 2010. Is case-specificity content-specificity? An analysis of data from extended-matching questions. Adv Health Sci Educ Theory Pract. 15(1):55-63.
Elstein AS, Shulman LS, Sprafka SA. 1990. Medical problem solving: a ten-year retrospective. Eval Health Prog. 13:5-36.
Eva KW, Neville AJ, Norman GR. 1998. Exploring the etiology of content specificity: factors influencing analogic transfer and problem solving. Academic Medicine. 73(10):S1-S5.
Holyoak KJ, Koh K. 1987. Surface and structural similarity in analogical transfer. Mem Cogn. 15:332-40.
Gick ML, Holyoak KJ. 1983. Schemia induction and analogical transfer. Cogn Psychol. 15:1-38.
Needham DR, Begg IM. 1993. Problem-oriented training promotes spontaneous analogical transfer: memory-oriented training promotes memory for training. Mem Cogn. 19:543-57.
Norman GR et al. 1985. Knowledge and clinical problem-solving. Med Educ. 19:344-56.
Norman G, Bordage G, Page G, Keane D. 2006. How specific is case specificity? Med Educ. 40(7):618-23.
Perkins DL, Salomon G. 1989. Are cognitive skills context-bound? Educ. Res. 18:16-25.
 |