How Not to Ask About Educational Attainment

Researchers frequently need to ask about educational attainment in surveys. This question is often asked imprecisely.  The example below (adapted from a survey about computer systems) includes one of my all time top pet-peeves: a response option for post-graduate degree in surveys targeted at Americans.  It is not a term commonly used in talking about higher education in the United States.

Educ

Why does this matter?

In this example, the category “post-graduate” is used to capture all levels of education beyond a Bachelor’s degree. In my experience, most Americans don’t understand the category and guess the meaning from the context of the other options. The question likely causes needless cognitive burden on respondents and creates ambiguity in the later analysis. The example above is just one example but I see many variants of this basic error.

A quick word about grammar.  In this post, we’re primarily talking about graduate as an adjective (e.g., She went to graduate school) because this is the form that usually occurs in survey questions.  The word is also used as a noun  (e.g., She is a high school graduate) or a verb(e.g., She graduated high school).

Also, please note my critique is strictly for surveys targeted at Americans.  For an international audience, it must be even more carefully considered.

What is a “graduate” degree?

It might surprise some to know that I don’t  have a graduate degree, I have a professional degree. Other professional degrees include: J.D., M.D., M.B.A.  Some folks might think a Ph.D. is a post-graduate degree; meaning, it is beyond graduate (i.e., Master’s level) education.  But in the United States it isn’t. A Ph.D. is a graduate degree. Wikipedia says postgraduate education is known as graduate education in North America. In countries such as India and the UK, it is commonly used terminology to refer to education beyond a Bachelor’s degree.

Other issues with this particular example

This example also asks the respondent to select his/her “highest degree completed,” as this type of question often does. Completion of a high-school education does not lead to a degree but rather a diploma. (I’ll withhold an additional grammar discussion related to the word “diploma” for now. ) The response options related to finishing high school or having completed some college are not completed degrees, but progress towards high-school diplomas or other degrees. These comments might seem nit-picky, it is possible that respondents “get” the badly worded questions and precision isn’t needed for backend analysis. This issue can be easily avoided, especially when question wording could be borrowed from another source.

How should an educational attainment question be worded?

I  recommend you begin by looking at how carefully crafted surveys such as those from the Census ask about education.  The example below is from the American Household Survey Questionnaire.

ACS Ed 1

You can then edit the Census question to meet the needs of your survey/client. I used  the example below  in a screening questionnaire for a usability study.

517 Ed

The moral of the story? Don’t reinvent the wheel when using widely used questions that exist in standard forms; borrow question wording from other sources.  And ALWAYS ask yourself if your question wording is putting unnecessary cognitive stress on the respondent.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s