Discussion Question

Name a trait you think is elusive and thus may be difficult to measure with great validity. Discuss different ways the trait could be measured.

Elusive Traits

Our text states that ‘no test is perfectly valid’ (Patten & Newhart, 2018). It is an accepted fact  that no measurement tool or concept can perfectly account for all the factors that go into the question the research is trying to answer. This means that validity is more of a spectrum and researchers must strive to achieve as valid of a test as possible. It is fair to state that some concepts are more elusive in how they can be measured that others. One thing I consider important that presents challenges in how to gauge is critical thinking skills. Critical thinking is a broad concept that is difficult to establish quantifiable criteria with.

Personal Experiences

When generating a test meant to evaluate participants critical thinking skills, a researcher must decide what metrics to use and what scenarios to place before them.  This will present difficulties considering the differences in past experience each participant has. For example, a study asking participants to assemble some sort of mechanical device will provide better results amongst people who grew up fixing cars than among individuals who played instruments. The same can be said if the situation were reversed. There is little to no way to account for personal experiences. Proper sampling techniques can dilute the discrepancies. Ultimately, any type of test is going to fall more in line with some peoples personal experiences than other which can construe data.

Revaluation Difficulties

A hallmark of research and experimentation lies within testing the participates for improvement. This will be difficult when testing critical thinking skills. Only a sample of questions can be provided at at time. Once reevaluation time comes the participants will hold a practice effect because they have already completed the task. This will have an effect on the validity of the study.  A treatment may or may not have been introduced in between the test but the practice effect will still remain.

Scope of Critical Thinking

Critical thinking is a large concept that is difficult to establish a definition for. A quick google search can prove how many different stances exist on what it entitles. Despite its importance to business and educators, critical thinking as a skill seems to be a trait that is difficult to establish criteria for. This transmits into a difficulty to quantify critical thinking. A researcher can identify traits and skills associated with critical thinking and assign values to them. Additionally, they can rank these skills. However, no hard measurement system exist making the weight of each trait opinionated.

Best way to Measure Critical Thinking

Despite the general consensus that critical thinking is important educators agree that few good assessments exist (Bissell & Lemons, 2006).  One thing that must be avoided when measuring critical thinking skills is to avoid closed questions such as multiple choice testing.  This is because no critical thinking is necessarily needed to come to the correct answer. In most test, a participant has a 25-20 percent chance of getting the right answer without thinking. The most recent and effective methods of studying critical thinking is the Halpern Critical thinking assessment (Dwyer, 2018). In this test, participants are asked to make a choice and then, later justify their answers. The questions are answered in an open manner allowing for explanation and interpretation. This allows for personal views to be accounted for but makes larger sample evaluation difficult.
 
 

References

Bissell, A. N., & Lemons, P. P. (2006, January). A New Method for Assessing Critical Thinking in the Classroom . Biosciencemag, 56(1), 66–72.