Anthro Notes

National Museum of Natural History Bulletin for Teachers

Vol. 17 No. 3 Fall 1995

CULTURAL BIAS IN TESTING: AN ANTHROPOLOGIST'S VIEW

Anthropologists often object to “Multiculturalism” when it focuses only on the trappings of different cultures—like food or music. Real multiculturalism goes deeper. It is harder to grasp, and more threatening; it addresses our intolerance, but it holds the key to real understanding and rich cultural exchange.

As anthropologists know (but few others understand), cultures differ not just in their obvious trappings but in more basic and subtle ways: patterns of thinking; logic, perception, construction of categories, goals, and values; ideals, morals, rhythms, emotions, and probably even psychological structure. To make a simple analogy to language, cultures do not just differ in vocabulary; they also differ in sound and grammar. Cultures have varied deep structures underlying their superficial differences just as languages have grammars that underlie their vocabularies. The differences are mostly learned, not genetic. A human baby can grow up functioning in any culture its caretakers teach, just as it can grow up speaking any language it is taught.

Most of these cultural behavior rules are arbitrary. All cultures must conform to nature to ensure survival. If you don't eat, reproduce, avoid biological damage and manage illness correctly (often within broad limits), you do not survive. But much behavior—from food choices to poetry—is simply arbitrary convention, things that we agree to do a certain way for the sake of consistency and predictability, much as neighbors agree to speak a particular language although any shared language would do.

These arbitrary cultural differences run surprisingly deep. Culture, not biology, says that adult American women can cry but men should not. Culture, not biology, says that men lust after the sight of women's breasts. (In many cultures they do not.) Much of our “logic” is also culturally derived. The “socialization” of children is largely a process of teaching them the arbitrary rules. We teach them what NOT to do. (In our society, for example, boys but not girls are taught not to cry. In many cultures all children are taught this.) All cultures limit freedom of action, expression, and even thought in the name of consistency and predictability. Cultures, including our own, are blinders that keep people looking in the “right” (i.e. the agreed) direction.

Comprehending how much of our own cultural system is arbitrary is perhaps the hardest lesson. Just as learning English as a baby makes some sounds seem natural and others unnatural, so being raised in middle class American society makes it hard to comprehend other peoples’ thinking and behavior. Worse, socialization can make us blind to the very existence of alternatives, reducing tolerance of other cultures and our ability to think critically about our own assumptions.

Contemporary standardized tests demonstrate many of these points since they include many items that still demonstrate serious naivete about what people in other cultures are familiar with. One is asked, for example, to notice that a cute suburban house has an incomplete chimney. But not all youngsters have seen such a house. More important, how many have lived near (or in) such houses enough to make them objects of real interest? The mere existence of such structures somewhere in the general environment is not enough, if they have no meaning to the individuals being tested. We all learn things better when they have real meaning for us. Also note that people need not be completely ignorant of such items to be penalized on the test. They will also do badly if they are slow because their responses are less automatic.

This is only one example of the many questions posed in standardized tests in which the cultural content, the specific items, of the question, is biased. Well meaning testers try to correct these by using more culture neutral items (if such exist!). But the biases also extend beyond the content or the items selected to other arbitrary American cultural rules and assumptions that are built into the questions. It is these biases that destroy the validity of even the “fairest” test. Apparently, few people understand this.

Often the form of the questions themselves is culturally biased as for example when conventional drawing styles are used. Consider a test question showing a drawing of two cats silhouetted in front of the sun/moon. One is supposed to note that one cat has no shadow. But to get the question right, you have to know what the simple drawn figures stand for—our drawing of the sun and moon are particularly conventional, not accurate—and that a squiggly line below one cat is actually a shadow behind it. The graphic style is unknown to many cultures or even to anyone reared on television rather than on picture books.

Consider analogies; we think that analogies test simple logic. But analogy problems are questions about the categories in which we put things. The categories determine logic. An analogy exists only if pairs of items can be put in the same group. But categories are cultural conventions, not revealed “truth.” Other cultures categorize things in different ways and therefore would set up different analogies and get our test questions wrong or find them nonsensical. (Remember, we categorize objects in many, often conflicting, ways: by size, color, material, function, place of origin, or (as in a sewing basket or workshop) by things that complement each other or operate on each other in certain tasks like a needle, thread, and a torn shirt. Different classifications are useful in different situations, but no way exists to say that one is obviously “correct” in the abstract. The only way to know which categorizing scheme applies in a particular context is to be initiated into the local culture.

Cross-cultural studies in psychology (Cole and Scribner 1974; Cole, Gay, Glick, and Sharp 1971) have shown that the way people classify things depends on many variables including the situation; the question asked; the types of objects presented; the familiarity of the objects; and the amount of formal, literate, Western schooling people have had. And they show how what appears initially to be an inability to classify in a certain way can disappear once the tester learns the cultural rules of the group or asks the right question.

North American-trained doctors have often had trouble communicating with Latin American patients. The doctors are trained to work with the categories “germ” and “antibiotic.” But many Latin Americans faced with knowledge of diseases and antibiotics have a classification system that they consider more fitting: the opposition of “hot” and “cold.” Some diseases and some antibiotics are classified together as hot, some as cold. The critical principle is to oppose a hot antibiotic to a cold disease and vice versa. Their logic comes from different categories, which we have great difficulty understanding, so we are uneducated and perhaps unintelligent by their standards. If they constructed an analogy test, we would be expected to recognize immediately that the most important categorizing principle was the “temperature” of items. Perhaps our classification is “better” because it is more “scientific” (although germ theory, too, persists despite being inaccurate in many of its applications). But even if the Latin American system is less accurate, people are socialized into it.

Consider a question that involves identifying one of two famous scientists, Albert Einstein or G. W. Carver. The question involves an obvious (but too simple) gesture toward “fairness” by permitting identification of a Black scientist or a White one. But it is hardly “fair,” because the category “scientist” itself is a more significant category in White American culture than in Black culture. (Several of my Black associates consider Carver a “White man's Black hero” because he did “White” things.) Whites are therefore culturally more likely than Blacks to recognize a scientist of any color.

But this example also has at least one more bias that is much more subtle. (It may or may not apply to Black/White differences but surely applies more broadly.) There are various ways of “knowing” things or people. This question favors people who “know” visually and utilize portraits and picture books. In a culture in which parents told stories rather than reading aloud from picture books, and oral tradition was important, people might know a person like Carver or Einstein very well but be less familiar with pictures.

And, consider a question involving two sets of cartoon figures in which one is supposed to notice whether any figures from the first set are repeated in the second. The answer is “no” because, although two pseudo-human figures are very similar, the diagonals on their tunics are reversed.

The use of cartoon figures supposedly eliminates cultural bias. But what is actually tested? The key question is whether one perceives and considers it worth noting that the diagonals are reversed. Anyone from a culture in which sex was indicated by the diagonals on peoples’ clothing would get this right because their culture taught them to focus on this distinction. But most of us have been taught by our culture to tune out such distinctions. Think about earrings on males. Most of us probably pay no attention to whether a man's earring is in his right or left ear. We tune out the distinction. Yet some Americans notice because to them the distinction conveys important information about sexual preference..

In order to simplify the bewildering array of information reaching us, we all learn to tune out things that have no cultural significance. This is why unsocialized children often “notice” things their elders ignore. Different cultures teach different rules about what to tune out , just as we learn to tune out the subtle distinctions in vowel sounds that French speakers are taught to hear or just as Inuit (Eskimos) see many distinctions within what we lump together as “snow. So this question, too, is a test of cultural habits not intelligence.

The point is that even if we look beyond the obvious cultural content of tests, the questions can readily be shown to be culture bound at a multitude of additional levels like the layers of an onion. And, like an onion, peeling away the layers of bias leaves nothing. No matter how hard we try, we are testing cultural awareness, not intelligence. There is no such thing as measuring pure thinking ability because all tests (and probably all thought itself) build on cultural categories just as all language builds on conventional grammar rules. I personally have been able to dissect every test question 1 have seen in this manner and I invite you and your students to try. It is a real learning exercise. Ask yourselves what unspoken American assumptions each question makes beyond its obvious cultural content.

Awareness of the real depth of cultural differences is both frightening and exhilarating. It is frightening because it raises the specter of greater complexity in dealing with others and greater humility and flexibility about our own assumptions. The excitement comes from contemplating how our lives could be enriched. Think of how dull our food, our music or our dress would be without the enormous recent influx of foreign influence. Is that all that other cultures have to offer? Medical anthropologists and doctors are discovering important new (to us) ways to think about and treat disease processes and illness by looking at traditional healers in other cultures. And, for all of our emphasis on “family values,” we have much to learn (including new ways to think) from cultures in which families and the associated values play a far larger role. Imagine the ways that other aspects of our lives and thoughts could be enlarged if we opened ourselves to real multiculturalism.

Cole, Michael and Sylvia Scribner. Culture and Thought. New York: Wiley, 1974.

Cole Michael, John Gay, Joseph A. Glick and Donald W. Sharp. The Cultural Context of Learning and Thinking. New York: Basic Books, 1971.

Mark Cohen

Department of Anthropology

SUNY Pittsburgh