Current and Future Research Paper

Psychological testing: old specialty, new markets

By Jonathan Rich, Ph.D. July 1, 2007

 

Psychological testing is an important part of psychology’s history. Opinions about its future are wide ranging. Some psychologists believe it’s a nearly obsolete tool while others see it as a specialty area with tremendous growth potential. Current and Future Research Paper

Payment trends Psychological testing has followed a reimbursement course similar to that of  psychotherapy . With the incursion of managed care in the 1980s and 1990s, payment amounts were reduced and the kind and quantity of services were limited by “medical necessity” criteria. Third-party payers vigorously followed these criteria and that dramatically slashed reimbursement for testing. Typically, psychological testing is considered medically necessary if the diagnosis is still unclear after thorough diagnostic interviewing.

However, psychologists have had some good news with regard to insurance reimbursement for testing. Although some Master’s level mental health practitioners have expanded their practices to include psychological testing, insurance typically only pays when a psychologist performs the test. Despite overall declines in Medicare payments, the rate for psychological testing has actually increased recently.

In January 2006, Medicare introduced new billing codes that distinguish tests administered by psychologists from those administered by a technician or assistant. This resulted in a 26 percent to 69 percent increase in payment for outpatient testing by psychologists. Current and Future Research Paper

Aurelio Prifitera, Ph.D., psychologist and president of Harcourt Assessment International, sees mental health parity laws as another factor which will reverse the trend toward declining payment for psychological testing. Parity laws require that mental health insurance reimbursement be on equal footing as physical health payments.

Survey of testing psychologists / In a 2007 survey in 18 states, I asked 32 psychologists that use testing in their practice if they saw psychological opportunities becoming more of less plentiful, their opinions about the future of testing and the psychological testing niches.

What was most remarkable about these results was the diversity of opinions. Forty-seven percent of respondents believed that, for psychologists in general, the market for psychological testing was shrinking, while 22 percent saw the market growing. The remainder had not seen a change. But when these psychologists were asked whether they had experienced a growth in testing opportunities in their own practice, 42 percent answered affirmatively.

One recurring theme among the most pessimistic psychologists was that insurance and managed care had drastically reduced payments. A North Carolina psychologist stated, “Managed care and insurance reimbursement have been drastically cut. Testing such as the MMPI, which used to be a regular part of the intake, is not done at all.” Others lamented that testing is being less emphasized in graduate training. John L. Reeves II, Ph.D., ABPP, professor and director of behavioral medicine services at UCLA’s Orofacial Pain Clinic said, “Sadly, the majority of psychology graduate schools are doing a very poor job of teaching psychometric testing and psychodiagnostics. Few graduate departments even require a competency evaluation on such gold standard tests as the MMPI.” Current and Future Research Paper

Psychologists with specialized training such as neuropsychology had the most positive attitudes or had found niches outside traditional medical insurance arena. Frank Cushing, Ph.D., practicing in Rockford, Ill., was among the most optimistic respondents. He sees some societal trends opening new markets, including schools asking for violence risk assessments and courts asking for sex offender evaluations. He noted requests for “ ADHD  evaluations have increased with more data about children being overmedicated.”

Surveyed psychologists who perform testing reported an average minimum $850 charge for a test battery and an average maximum of $2,550. Those with a subspecialty in neuropsychology reported an average range of $1,000 to $3,260.

There appears to be no shortage of opportunities for creative and entrepreneurial psychologists. Professional school admission tests, such as the LSAT (Law School Admission Test), require specific documentation in order to accommodate learning disabilities (see lsat.org).

Other new niches include testing for citizenship waivers, pre-surgical evaluations and assessments to screen candidates for the ministry or police work. Matchmaking services such as eHarmony use psychologist expertise to design and validate tests that pair kindred spirits with each other.

Prifitera commented that testing is expanding into business settings to assess talent and into primary care medical settings to screen for mental health needs. In these contexts, psychologists may have a less direct role but are still needed to manage the assessment process.

Traditionally, testing has been done to guide mental health treatment planning. While economic pressures have decreased the demand for this application, new markets continue to make testing a viable, exciting and expanding area of practice. Current and Future Research Paper

———-

Jonathan Rich, Ph.D., is a psychologist in private practice in Irvine, Calif., specializing in psychological testing. His website is PsychologicalTesting.com. He is the author of The Couple’s Guide to Love and Money. His e-mail address is: jrich@psychologicaltesting.com.

Week 11: Current and Future Research

Introduction

“An overview of the last century shows that most fundamental principles in psychometrics and testing were available by 1930. With a few notable exceptions, the remainder of the last century of testing was devoted to applying or refining these principles.” —Dr. Susan Embretson (2004)

Although the foundation of psychological testing has remained steady for the better part of a century, as Susan Embretson notes above, there are areas in the field of psychological testing that are rapidly evolving. Many of these changes, such as IRT that you explored in a previous week, were made possible by advances in computer technology. Others, such as increased awareness of cultural influences on testing, reflect globalization and shifting awareness. Current and Future Research Paper

In this final week, you explore these and other current and future trends in psychological testing. You also complete a final examination to assess your knowledge of psychological testing and measurement. If you would like to share the Final Project you submitted last week with your colleagues, you have the option to upload it to the Doc Sharing area. Current and Future Research Paper

Reference: Embretson, S. (2004). The second century of ability testing: Some predictions and speculations. Measurement: Interdisciplinary Research and Perspectives, 2(1), 1–32.

Objectives

Students will:

· Analyze trends in psychological testing

· Apply knowledge of psychological testing and measurement

 

Readings

· Embretson, S. (2004). The second century of ability testing: Some predictions and speculations. Measurement: Interdisciplinary Research and Perspectives, 2(1), 1–32.

· Naglieri, J. A., Drasgow, F., Schmit, M., Handler, L., Prifitera, A., Margolis, A., & Velasquez, R. (2004). Psychological testing on the Internet: New problems, old issues. American Psychologist, 59(3), 150–162. Retrieved from the Walden Library databases.

· Rich, J. (2007, July 1). Psychological testing: Old specialty, new markets. The National Psychologist. Retrieved from http://nationalpsychologist.com/2007/07/psychological-testing-old-specialty-new-markets/10933.html.

 

Week 11 Discussion

Trends in Psychological Measurement

As mentioned in this week’s Introduction, computers have propelled many of the changes in psychological testing. Computers facilitate the computation needed for test construction, administration, scoring, and interpretation. Through the Internet, computers also make it much easier to collect test validation data and to administer tests at a distance. Nevertheless, these changes are accompanied by new challenges and problems. For instance, some computer programs are available that score test protocols, write narratives, and make recommendations. A computer-generated report, however, is unable to weigh clinical information in the way that a professional could and can therefore be misleading. When test validation and administration is done virtually, there is less control over the nature of the sample and the conditions of administration. Furthermore, electronic storage and transmission of test information raises concerns about security and confidentiality.

Another outcome of increasing computer technology is that item-response theory (IRT) is becoming increasingly easy to apply. New software and fast and inexpensive computers are readily accessible. This has made possible the development of item banks and computer administration that can be carefully matched to an examinee’s ability level. IRT also allows test items to be compared across languages and cultures. Current and Future Research Paper

Increased sensitivity to cultural diversity and disability is another trend. Since the 1970s, there has been a significant awareness of how culture affects and biases test performance. Early responses to this awareness were to “throw out the baby with the bathwater.” For instance, the 1979 California Larry P. v. Riles legal decision banned the use of intelligence tests for placing African-American students in special education classes. An evolving understanding of culture allows us to apply tests more appropriately, selecting test instruments that are consistent with an examinee’s background, interpreting patterns of scores rather than single summary scores, and looking at test results against the background of personal information.

To prepare for this Discussion, select one peer-reviewed journal article from the Walden Library on a current trend in psychological measurement (one mentioned above or a different one). Consider the implications of this trend for future testing, measurement, and research. Current and Future Research Paper

With these thoughts in mind:

Post by Day 4 a brief summary of the article you selected and an explanation of how the article characterizes the trend. Finally, explain two implications of this trend in future testing, measurement, and research.

Be sure to support your postings and responses with specific references to the Learning Resources.

 

Take Test: Quiz – Week 11

 

Top of Form

Test Information

Description
Instructions	Note: Please refer to the following table to complete question #27.

 

QUESTION 1

1. Which of the following would not be considered an advantage of testing over a clinical or job interview?

	a.	Test taker can be compared to a known population
	b.	Ratings will be consistent for different assessors
	c.	Assessor can get a good sense of the person as a whole
	d.	Outcomes can be predicted with known levels of accuracy

1 points   

QUESTION 2

1. The first intelligence test was developed by:

	a.	Binet
	b.	Wechsler
	c.	Wundt
	d.	Socrates

1 points   

QUESTION 3

1. Psychological tests are commonly used to assist with decisions regarding:

	a.	Academic placement
	b.	Hiring
	c.	Psychiatric diagnosis
	d.	All of the above

1 points   

QUESTION 4

1. Jill correctly answered 40 out of 50 questions on her final exam and received a score of 80%. This is an example of what type of score?

	a.	Norm-referenced
	b.	Criterion-referenced
	c.	Z-score
	d.	Percentile

1 points   

QUESTION 5

1. Jose was a straight-A student in high school and he has been accepted to a competitive college. He is having trouble deciding on a major. What type of test is most likely to help him make a decision?

	a.	An intelligence test, such as the Wechsler.
	b.	An objective test of psychopathology, such as the MMPI.
	c.	A projective test, such as the Rorschach inkblot test.
	d.	An interest inventory, such as the Self-Directed Search.

1 points   

QUESTION 6

1. A Pearson-r correlation coefficient indicates:

	a.	The degree and direction of a relationship between two variables
	b.	The formula to predict one variable from another
	c.	Whether the means of two groups are significantly different from each other
	d.	The amount of score variability

1 points   

QUESTION 7

1. On a midterm with 100 questions, the mean number correct was 80 with a standard deviation of 2. Shigeko obtained a score of 86. Based on this information, you could say that her performance was:

	a.	Slightly above average
	b.	Moderately above average
	c.	Far above average
	d.	Average

1 points   

QUESTION 8

1. Dr. Roe finds a significant positive correlation between the number of cans of diet soda consumed per year and the number of days sick per year. From this correlation, you can confidently conclude that:

	a.	Ingredients in diet soda cause illness.
	b.	People crave diet soda when they are sick.
	c.	People who drink more diet soda tend to be sick more often.
	d.	Diet soda drinkers generally have a poorer diet.

1 points   

QUESTION 9

1. Coefficient alpha measures:

	a.	Test item difficulty
	b.	The degree to which test items are consistent with each other
	c.	Test stability over time
	d.	The portion of a test score accounted for by guessing

1 points   

QUESTION 10

1. A reliability coefficient of .80 indicates that test score variance is due to:

	a.	80% true score; 20% error
	b.	20% true score; 80% error
	c.	64% true score; 36% error
	d.	36% true score; 64% error

1 points   

QUESTION 11

1. High test reliability indicates:

	a.	Low measurement error
	b.	High measurement error
	c.	Low prediction error
	d.	None of the above

1 points   

QUESTION 12

1. Professor Rogers has created a test of self-esteem and reported that he obtained a validity coefficient of .85. Before concluding that this is a good test, the most important question to ask is:

	a.	What are Rogers’ academic credentials?
	b.	How many items are on the test?
	c.	Against what criterion was the test validated?
	d.	What is the response format?

1 points   

QUESTION 13

1. It would be most appropriate to use a 1-year test–retest reliability coefficient for a test of:

	a.	Personality
	b.	Mood
	c.	Stress level
	d.	Fatigue

1 points   

QUESTION 14

1. Inter-scorer reliability would most likely be reported for a scale with what response format?

	a.	Open-ended
	b.	Multiple Choice
	c.	True/False
	d.	Rating scale

1 points   

QUESTION 15

1. Correlating high school senior SAT scores with their later freshman college GPA is an example of:

	a.	Concurrent validity
	b.	Face validity
	c.	Test-retest validity
	d.	Predictive validity

1 points   

QUESTION 16

1. Which of the following is most likely to be a useful item for a test of self-esteem with a true/false response format?

	a.	“I feel good about myself.”
	b.	“I feel good about myself and I feel confident.”
	c.	“Either I feel good about myself or I feel confident.”
	d.	“Describe how you feel about yourself.”

1 points   

QUESTION 17

1. A test item asks the respondent to choose whether he is more “Outgoing” or “Conscientious.” This is an example of what kind of item?

	a.	Ipsative
	b.	Forced choice
	c.	Likert
	d.	a or b

1 points   

QUESTION 18

1. A test requires respondents to respond to items by indicating if they “Strongly disagree,” “Disagree,” “Agree,” or “Strongly agree.” This is an example of a(n):

	a.	Open-ended response
	b.	Likert scale
	c.	Forced-choice response
	d.	Multiple-choice response

1 points   

QUESTION 19

1. Here are three questions for a measure of anxiety: Q1) I worry all the time  Q2) I feel very tense  Q3) I feel relaxed

The respondent is asked to rate each item on a 3-point scale, 1: Never, 2: Sometimes, 3: Always. Which of the following formulas would you use to transform Q3 before doing an item analysis?

	a.	4 – Q3
	b.	Q3*(-1)
	c.	3 – Q3
	d.	Q3/3

1 points   

QUESTION 20

1. Random selection from a population means that participants are chosen:

	a.	Haphazardly
	b.	So that each one has an equal likelihood of being selected
	c.	So that sample demographics closely match the population demographics
	d.	By testing participants who are easiest to contact

1 points   

QUESTION 21

1. Dr. Campbell uses an Internet form to gather standardization information for a new intelligence test. He wants data for people in an area with over a million residents. This area has few computers and little Internet access. He uses online advertising to direct residents to his test. After persistent efforts, he is able to obtain data for 1,000 residents. He is able to verify reliably that the participants are residents of his area of interest. What would you say about this dataset?

	a.	The size of the dataset is adequate so this can be considered a representative sample.
	b.	If the demographics (ethnicity, gender, age) are similar to the area’s population demographics, then this can be considered a representative sample.
	c.	If he obtained a larger sample (i.e., 10,000 people) then the sample could be considered representative.
	d.	This would not be considered a representative sample, even if it were larger or consistent with population demographics.

1 points   

QUESTION 22

1. Computerized adaptive testing (CAT) is a system in which:

	a.	Items are modified to accommodate disabilities.
	b.	Items are selected based on responses to previous items.
	c.	Items are presented which make use of computer graphics.
	d.	The respondent uses computer input devices specifically designed for the test.

1 points   

QUESTION 23

1. Six-year old Billy and his 20-year-old brother Sam both obtain valid IQ scores of 110. Their mother comments that Billy is really smarter since his score is particularly impressive “for a 6-year-old.” Which of the following would likely be correct?

	a.	Billy is not currently smarter, but he will be when he and Sam are both adults.
	b.	IQ is age adjusted, so Billy and Sam are at the same percentile, relative to their respective age groups.
	c.	Billy and Sam would be about equally matched on games requiring intelligence, such as chess.
	d.	Their mother is correct.

1 points   

QUESTION 24

1. You have developed a test of musical ability, and would like to report the scores as Stanines, with a mean of 5 and a standard deviation of 2. Raw scores on your test have a mean of 30 and a standard deviation of 5. What is the Stanine score for someone who obtains a raw score of 25?

	a.	1
	b.	3
	c.	7
	d.	9

1 points   

QUESTION 25

1. According to Item Response Theory, “theta” for a particular individual on a particular item is related to:

	a.	The probability that she will pass the item
	b.	Her ability to understand the item wording
	c.	The degree to which the item distinguishes her from other test takers
	d.	Her likelihood of correctly guessing the answer

1 points   

QUESTION 26

1. The three-parameter IRT model indicates which three item characteristics?

	a.	Discrimination, readability, difficulty
	b.	Difficulty, unique contribution to the toal score, discrimination
	c.	Difficulty, chance of guessing correctly, discrimination
	d.	Reliability, unique contribution to the total score, validity

1 points   

QUESTION 27

1. You are creating a new scale using classical test analysis. Which of these four items would you eliminate? Note: Please use the table found at the top of this page to complete this question.

	a.	Item A
	b.	Item B
	c.	Item C
	d.	Item D

1 points   

QUESTION 28

1. Which of the following provides clear evidence of unlawful adverse impact?

	a.	Men obtain lower scores on a test of management ability and so are less likely to be chosen for certain management positions.
	b.	White applicants tend to score higher on a job selection test and are hired over lower-scoring nonwhites, even though whites are already overrepresented in the job category.
	c.	Persons who are under age 40 score higher on a test of popular culture and are hired over older applicants. High scores on this test do not predict better performance on this job.
	d.	Applicants for a position as a proofreader are given a test of reading ability; an applicant with dyslexia fails the test and is not hired.

1 points   

QUESTION 29

1. Which of the following activities is a clear ethical violation?

	a.	Displaying a few MMPI items on your website for educational purposes
	b.	Releasing a psychological testing report directly to a client
	c.	Administering an IQ test and reporting the results to a private client who is curious about his IQ
	d.	Recommending specific jobs in a vocational assessment

1 points   

QUESTION 30

1. DIF is a technique that could be used to determine:

	a.	Whether an item on a translation of a test functions the same as the original item
	b.	The percentile rank of a particular client on a test
	c.	Whether an applicant will be successful on a job, based on test results
	d.	What a test score would be, adjusting for guessing

Bottom of Form