The science of psychological assessment

View previous topic View next topic Go down

Re: The science of psychological assessment

Post by Prajnaparamita on Wed Apr 01, 2015 1:49 pm

kleenestar wrote:(That said, these tests are generally more useful for fostering reflection and self-knowledge than they are for accurately telling you about yourself. I haven't read the research on the enneagram test, but MBTI, for example, is, er, well, not the most valid test ever created. This is not to say I don't love taking personality tests! But with my psychology researcher hat on, I just feel the need to point out that there are issues with most of them.)

Hey Kleenestar, I imagine this would be a totally different thread (but only if you would be interested) but I mentioned earlier in the thread having been forced to take some of the "scientifically validated" official personality tests in various times in my life (for instance Beck Depression Inventory/Anxiety Inventory, MMPI-2, Forer Structured Sentence Completion Test and the Rorschach inkblot test) and I was wondering if maybe you could talk some about how the real diagnostic personality tests work and what we can learn from them?

_________________
Foster! It saves lives! And kittens! People will just give you kittens! For free!
avatar
Prajnaparamita

Posts : 375
Reputation : 209
Join date : 2015-02-05

View user profile

Back to top Go down

Re: The science of psychological assessment

Post by kleenestar on Wed Apr 01, 2015 3:49 pm

I wish I could, but these days I'm mostly typing one-handed (baby!) so I can't commit to doing this well. Doubly so because I'm only familiar with the research on some tests (e.g. MBTI) which would mean I'd have to do a lot of bringing myself up to speed. Thanks for asking, though - I'm super flattered.
avatar
kleenestar

Posts : 289
Reputation : 204
Join date : 2014-10-01

View user profile

Back to top Go down

Re: The science of psychological assessment

Post by Hirundo Bos on Wed Apr 01, 2015 6:20 pm

Prajnaparamita wrote:
Hey Kleenestar, I imagine this would be a totally different thread (but only if you would be interested) but I mentioned earlier in the thread having been forced to take some of the "scientifically validated" official personality tests in various times in my life (for instance Beck Depression Inventory/Anxiety Inventory, MMPI-2, Forer Structured Sentence Completion Test and the Rorschach inkblot test) and I was wondering if maybe you could talk some about how the real diagnostic personality tests work and what we can learn from them?

kleenestar wrote:I wish I could, but these days I'm mostly typing one-handed (baby!) so I can't commit to doing this well. Doubly so because I'm only familiar with the research on some tests (e.g. MBTI) which would mean I'd have to do a lot of bringing myself up to speed. Thanks for asking, though - I'm super flattered.

MA in psychology here. I don't know the research on particular tests, and have some weaknesses around research comprehension, but I do know some of the reasoning behind such tests, if Prajna or anyone else are interested.
avatar
Hirundo Bos

Posts : 532
Reputation : 285
Join date : 2014-10-01

View user profile http://abouthirundo.blogspot.com

Back to top Go down

The science of psychological assessment

Post by jcorozza on Wed Apr 01, 2015 6:54 pm

Hirundo Bos wrote:
Prajnaparamita wrote:
Hey Kleenestar, I imagine this would be a totally different thread (but only if you would be interested) but I mentioned earlier in the thread having been forced to take some of the "scientifically validated" official personality tests in various times in my life (for instance Beck Depression Inventory/Anxiety Inventory, MMPI-2, Forer Structured Sentence Completion Test and the Rorschach inkblot test) and I was wondering if maybe you could talk some about how the real diagnostic personality tests work and what we can learn from them?

kleenestar wrote:I wish I could, but these days I'm mostly typing one-handed (baby!) so I can't commit to doing this well. Doubly so because I'm only familiar with the research on some tests (e.g. MBTI) which would mean I'd have to do a lot of bringing myself up to speed. Thanks for asking, though - I'm super flattered.

MA in psychology here. I don't know the research on particular tests, and have some weaknesses around research comprehension, but I do know some of the reasoning behind such tests, if Prajna or anyone else are interested.

MA in school counseling here - I'd be interested in this type of discussion as well - I'm familiar with many of the assessments, though not as familiar with their validity/other problems, but it's something I'd be interested enough to look into it (and it could come in handy in the future!)
avatar
jcorozza

Posts : 418
Reputation : 167
Join date : 2015-03-08

View user profile

Back to top Go down

Re: The science of psychological assessment

Post by Prajnaparamita on Wed Apr 01, 2015 9:14 pm

Okay guys, seeing as I requested this I'm going to start things off. I've gone through three neuro/psych batteries, one when I was 11, one when I was 16 and one six months ago. I've only seen the report for the most recent one, which I have open in front of me right now. Out of everything, I was given, the MMPI probably bothered me the most (okay getting the Rorschach bothered me the most, when I was 11 because I was convinced it meant they thought I was crazy and if I didn't answer right I was going to be dragged off to the crazy house, and ever since then it's been like "seriously, this crap again? It looks like two people doing the tango--what do you think about that? Wake me up again when it's time to play Tower of London." Yeah, I've been through this a lot...)

Having done some research on it on my own after I had to go through this again most recently there's some things that have really, really bothered me (beyond wondering about the scientific validity of these tests in general) and that's the really retrograde (to say the least) stuff about gender. For instance, Scale 5--Masculinity/Femininity (Mf) which is supposed to scale how well you fit into the gender binary--why does it even exist? For one the questions are pretty shitty--I obviously cannot know for sure what question is for what scale, but you can usually guess. I found the complete question list (posted on a forum for people trying to bypass personality testing, lol!) and here are some of the (many) ones I think are relevant:

I have often wished I was a member of the opposite sex

I like to read mechanics magazines/I would like to be a solider/I think I would like the work of a building contractor/There was never a time in my life when I liked to play with dolls/I like adventure stories better than romantic stories

I would like to be a florist/I would like to be a nurse/I like collecting flowers or growing houseplants/If I were an artist, I would like to draw flowers/I liked playing house when I was a child (There's also one about enjoy arranging flowers that I remember but wasn't able to find again--it's amazing that my ovaries let me think about anything other than flowers!)

So umm, yeah, that's kinda shitty, but also I assume the psychologist is grading these questions for abnormalities from the gender of the patient that the psychologist has come to assume. And even beyond these sexist stereotypes, what purpose does Scale 5 serve, really? Suppose hypothetical we had an actual test to be able to determine gender dysphoria? What would be the point? "We have detected a high level of gender dysphoria, and have concluded that despite how the patient currently identifies, she is actually a man." That seems so much more damaging than helpful, as far as I can see at least. Anyway, that's the mindset I have coming into this--seriously questioning the scientific validity of something that already seemed to fraught with misinformation.

Also, does anyone have any insight on how honest people generally are with the questions? So many are so damn obvious, and while I've heard people are more likely to answer honestly when asked by a computer than a real person, the questions just seem so obvious in their intent ("depression, mania, psychopath, hypochondriac, mania, depression") that I imagine trolling it would be fairly effortless if you wished to do so. I mean maybe it's like my experience with the sentence completion test--eventually the sheer tedium wore me down to the point of answering far more personally than I had intended, but given how wacky most of the questions are, I don't feel like I spaced out at all!

Anyway, here's what was written up about my performance on the MMPI-2 on the most recent assessment:


“The MMPI-2 validity configuration indicates that Prajna [not using my real name, Razz ] approached the personality inventory in a valid manner. The scores show a moderate elevation on Scale F within the validity scales, which suggests and acknowledgement of more unusual symptoms and more psychological distress than is typical for most individuals.

Wait, major elevation on Scale F means I'm trolling you, minor elevation means I'm under psychological stress? Geez, wish I'd been under more and totally invalidated all of this for you! But seriously, I haven't heard of Scale F also being used to show psychological distress anywhere else--is there some kind of cutoff between showing distress (and I'm not exactly sure how it does) and just answering randomly?


This is a pattern usually associated with feelings of being easily overwhelmed and potentially unable to cope with the stress of everyday life. Low self-esteem is seen in the low score on Scale K (T score = 41).

This is interesting--I thought the Scale K was used to determine whether the patient evaluated was being defensive and evasive, which could also invalidate the test. But low levels of defensiveness shows low self-esteem? It's almost like there's "off-label" diagnostic uses for the validity scales; has anyone else heard of this before? Is it common?


In terms of the clinical scale configuration, Prajna’s profile has seven significant elevations. In order of elevation, Scale 2 (T score =96),

Depression


Scale 7 (T score = 83),

OCD, and similar tendencies, which is interesting that it would be my second highest, because even with all the diagnoses I have collected over the years, that has never been one. But it's also for guilt and self-doubt (big depressive symptoms for me) which don't seem so clearly linked? Shouldn't that be a separate scale?


Scale 6 (T score = 78),

Paranoia, of both the schizophrenic and overly socially anxious kind. Likewise, why aren't those two things separated out?


Scale 3 (T score = 73),

Hysteria--actually surprised this isn't higher for me.


Scale 1 (T scale = 73)

Hypochondria. Yo, have you ever considered that some of those health ailments I keep on describing are real and come from all that psychological distress and tension you keep on saying I'm experiencing? Nope? Never mind then.


Scale 8 (T score = 72),

Schizophrenia. Wut?


and Scale 0 (T score = 72)

Social introversion.


are considered clinically significant. This combination of scores is often seen in sensitive individuals who are depressed, tense, worried, ruminative, and generally distressed. This number of clinically significant elevations is associated with marked psychological distress and symptoms, and these are generally at such a level that they will impair concentration and attentional functioning.

NO SHIT SHERLOCK! Seriously, if you'd taken five minutes to listen to me, I could have told you all of that. Well that's an hour of my life I'm never getting back. The stuff on the Beck Anxiety/Depression Index comes next, which just says all that was said above, really.

Anyway, enough rambling for me. I can talk about the Forer sentence completion test, but I think I've asked enough questions/ranted enough as is.

_________________
Foster! It saves lives! And kittens! People will just give you kittens! For free!
avatar
Prajnaparamita

Posts : 375
Reputation : 209
Join date : 2015-02-05

View user profile

Back to top Go down

Re: The science of psychological assessment

Post by PintsizeBro on Thu Apr 02, 2015 3:03 pm

I'm a psych degree-haver as well. I love personality tests, they're a lot of fun, but there's a real lack of rigor in a lot of them. Prajna's example of using old, tired stereotypes to measure gender dysphoria is just the beginning.

There is some merit to a lot of these tests, but at best they need to be taken with a grain of salt and some less strict interpretation of the information.

PintsizeBro

Posts : 307
Reputation : 233
Join date : 2015-02-13

View user profile

Back to top Go down

Re: The science of psychological assessment

Post by Prajnaparamita on Thu Apr 02, 2015 6:07 pm

PintsizeBro wrote:Prajna's example of using old, tired stereotypes to measure gender dysphoria is just the beginning.

Oh wow, thank you so much for validating my experience in regards to this. Every single person I've talked to about my concerns about the MMPI has been like "there there dear, these things are complicated and the professionals know what they're doing and they know best." There was also this weird undercurrent of sexism in how they analyzed the results too. For instance in the sentence completion test one of my answers was:

When she thought about marriage... "She wasn't interested."

This sentence was cited as evidence for me experiencing social isolation and disconnect from others. I mean really, a woman can't feel like marriage isn't for her without it being seen as a symptom of a greater pathology?

_________________
Foster! It saves lives! And kittens! People will just give you kittens! For free!
avatar
Prajnaparamita

Posts : 375
Reputation : 209
Join date : 2015-02-05

View user profile

Back to top Go down

Re: The science of psychological assessment

Post by Hirundo Bos on Tue Apr 07, 2015 3:34 pm

Hmm. I've been slow in saying something about this, partly because of holidays, partly because I don't seem to remember as much about the subject as I thought I would... but I think I mostly agree with PintsizeBro. I'm fascinated with multiple choice-based personality tests, especially with the way they are made: By comparing scores across a large number of people, then use the statistical results, rather than the actual content of the questions, to see what information can be guessed from the replies. But with statistical results, there is always a degree of uncertainty, and with many of the tests the uncertainty is quite high. So a test can't really tell you who you are... and shouldn't, on its own, be used to make a diagnosis. At best, it can give some pointers in the right direction.

Something I also remember – when we learned about the MMPI, we were warned that the test were calibrated in a way that almost anyone would get significant scores on one or more scales. This was supposed to make it more sensitive to individual differences between people already assumed to have psychological issues, but it would also make the results look more worrying than they actually were.
avatar
Hirundo Bos

Posts : 532
Reputation : 285
Join date : 2014-10-01

View user profile http://abouthirundo.blogspot.com

Back to top Go down

Re: The science of psychological assessment

Post by jcorozza on Tue Apr 07, 2015 10:25 pm

Prajnaparamita wrote:
Having done some research on it on my own after I had to go through this again most recently there's some things that have really, really bothered me (beyond wondering about the scientific validity of these tests in general) and that's the really retrograde (to say the least) stuff about gender. For instance, Scale 5--Masculinity/Femininity (Mf) which is supposed to scale how well you fit into the gender binary--why does it even exist? For one the questions are pretty shitty--I obviously cannot know for sure what question is for what scale, but you can usually guess. I found the complete question list (posted on a forum for people trying to bypass personality testing, lol!) and here are some of the (many) ones I think are relevant:

I have often wished I was a member of the opposite sex

I like to read mechanics magazines/I would like to be a solider/I think I would like the work of a building contractor/There was never a time in my life when I liked to play with dolls/I like adventure stories better than romantic stories

I would like to be a florist/I would like to be a nurse/I like collecting flowers or growing houseplants/If I were an artist, I would like to draw flowers/I liked playing house when I was a child (There's also one about enjoy arranging flowers that I remember but wasn't able to find again--it's amazing that my ovaries let me think about anything other than flowers!)

I'll admit, the MMPI is not one I am terribly in the know about, but I remember talking about biases in my assessment class, and just looking at some of the questions now, I see a ton of class bias, too, with questions like:

I am satisfied with the amount of money I make (T/F)
or
I worry a great deal over money. (T/F)

If you don't have a lot of money, these are perfectly reasonable questions to answer "true" to.

Then there's stuff that deals with what could be actual physical problems, like:

My hands and feet are usually warm enough.
or
There seems to be a lump in my throat much of the time.
(A friend of mine had this problem - that feeling along with a fairly persistent cough. Turns out she has Cystic Fibrosis)

I think that those are some of the ones that lean towards the hypochondria scale...but what if you actually experience these physical symptoms?

And then there are some that just make me wonder what's wrong with the people making this test, like:

At times I feel like swearing.
- is this abnormal? Are there people who NEVER feel like swearing? Pretty sure they're in the minority
or
I do not always tell the truth
-hmm. If you say "true" to this...how can we take any other answer at face value? And if you say "false", and you actually are a liar, well, yeah.

And then, of course, is the problem with "true/false" answers - I had thought they used a likert scale of some sort, which would make more sense, since some of the questions would be more useful without absolutes.

Soooo many of these questions are poorly worded, or have a variety of possible implications besides what they're trying to get from them.

I think one of the problems lies in the idea that - and I saw this on a blog that posted the questions - it feels like there are right and wrong answers, and there is a heck of a lot of implied value judgment in them.


avatar
jcorozza

Posts : 418
Reputation : 167
Join date : 2015-03-08

View user profile

Back to top Go down

Re: The science of psychological assessment

Post by Prajnaparamita on Tue Apr 07, 2015 10:41 pm

jcorozza wrote:
I'll admit, the MMPI is not one I am terribly in the know about, but I remember talking about biases in my assessment class, and just looking at some of the questions now, I see a ton of class bias, too, with questions like:

I am satisfied with the amount of money I make (T/F)
or
I worry a great deal over money. (T/F)

If you don't have a lot of money, these are perfectly reasonable questions to answer "true" to.

Oh my god that's so true I never thought about that! Living in poverty and struggling to make ends meet (or having been raised in an environment like that) doesn't make you hysterical, it makes you grounded in the reality of your situation.

jcorozza wrote:
I think that those are some of the ones that lean towards the hypochondria scale...but what if you actually experience these physical symptoms?

Yeah, my thoughts too. I dunno, maybe those constant stomachaches I reported could have been caused my the colitis I developed at a freakishly young age from all the constant stress?

jcorozza wrote:
And then there are some that just make me wonder what's wrong with the people making this test, like:

At times I feel like swearing.
- is this abnormal?  Are there people who NEVER feel like swearing?  Pretty sure they're in the minority
or
I do not always tell the truth
-hmm. If you say "true" to this...how can we take any other answer at face value?  And if you say "false", and you actually are a liar, well, yeah.

Okay, I think these two are part of the validity scales, namely the one meant to measure whether or not the person is lying on the test. This is done by seeing if the person answers questions about themselves that portrays them in an impossibly flattering light, like that they never swear or never lie. idk how well that actually works, but there you go.





Hirundo Bos wrote:
Something I also remember – when we learned about the MMPI, we were warned that the test were calibrated in a way that almost anyone would get significant scores on one or more scales. This was supposed to make it more sensitive to individual differences between people already assumed to have psychological issues, but it would also make the results look more worrying than they actually were.

Oh wow I didn't even think of this before, but it makes sense. If it's assumed that everyone who is given the MMPI has some kind of psychological problem, then the MMPI is not meant to determine whether or not the person has a psychological issue, but what kind and how severe. Just wondering, is there some kind of "control" response that someone who has absolutely no kind of pathology/psychological issues would be expected to give?

_________________
Foster! It saves lives! And kittens! People will just give you kittens! For free!
avatar
Prajnaparamita

Posts : 375
Reputation : 209
Join date : 2015-02-05

View user profile

Back to top Go down

Re: The science of psychological assessment

Post by jcorozza on Tue Apr 07, 2015 10:57 pm

I'm not going to lie - in looking at the questions, I of course had to answer them all - my highest (69) was subtle paranoia and naivete (65) - these seem pretty contradictory to me. On the other hand, my anxiety was below 50, and my depression was dead center (since anxiety is my most prevalent issue, with depression raising it's ugly head less often), I was like...uh, what? Then again, if I were to take it WHILE DEPRESSED (which for me is predominantly situational) my answers would be drastically different. Which is why I dislike the T/F dichotomy.
avatar
jcorozza

Posts : 418
Reputation : 167
Join date : 2015-03-08

View user profile

Back to top Go down

Re: The science of psychological assessment

Post by Hirundo Bos on Wed Apr 08, 2015 10:36 am

Prajnaparamita wrote:
jcorozza wrote:
And then there are some that just make me wonder what's wrong with the people making this test, like:

At times I feel like swearing.
- is this abnormal?  Are there people who NEVER feel like swearing?  Pretty sure they're in the minority
or
I do not always tell the truth
-hmm. If you say "true" to this...how can we take any other answer at face value?  And if you say "false", and you actually are a liar, well, yeah.

Okay, I think these two are part of the validity scales, namely the one meant to measure whether or not the person is lying on the test. This is done by seeing if the person answers questions about themselves that portrays them in an impossibly flattering light, like that they never swear or never lie. idk how well that actually works, but there you go.

Yes. Questions like that are used in a lot of questionnaires, not only for personality tests, and the reasoning behind is what Prajna says. If someone answers "I never lie", that's very likely to be a lie. (And in the cases where it isn't, it would be unusual enough to hint at some pathology, something that when I think of it would affect the results when the test is specifically about pathology.)

Prajnaparamita wrote:
Hirundo Bos wrote:
Something I also remember – when we learned about the MMPI, we were warned that the test were calibrated in a way that almost anyone would get significant scores on one or more scales. This was supposed to make it more sensitive to individual differences between people already assumed to have psychological issues, but it would also make the results look more worrying than they actually were.

Oh wow I didn't even think of this before, but it makes sense. If it's assumed that everyone who is given the MMPI has some kind of psychological problem, then the MMPI is not meant to determine whether or not the person has a psychological issue, but what kind and how severe. Just wondering, is there some kind of "control" response that someone who has absolutely no kind of pathology/psychological issues would be expected to give?

I'm not sure, I can't remember having heard of such controls. (An underlying question here is where to draw the line between pathological and high-but-non-pathological levels of an issue. That question is conceptual more empirical, so people who rely on questionnaires alone won't really be able to adress it. It is also a question with practical and ethical implications.)

jcorozza wrote:I'm not going to lie - in looking at the questions, I of course had to answer them all - my highest (69) was subtle paranoia and naivete (65) - these seem pretty contradictory to me.  On the other hand, my anxiety was below 50, and my depression was dead center (since anxiety is my most prevalent issue, with depression raising it's ugly head less often), I was like...uh, what?  Then again, if I were to take it WHILE DEPRESSED (which for me is predominantly situational) my answers would be drastically different. Which is why I dislike the T/F dichotomy.

I'm not really sure what's the case with the MMPI, but in general, both situational and long term values can be measured with questionnaires... referred to as states and traits, respectively... Although I think it's most common to speak about long term traits. Something that, if it is the case, can make traits appear more static than they are. Which again might feed stereotypes, bringing us back to Prajna's post.
avatar
Hirundo Bos

Posts : 532
Reputation : 285
Join date : 2014-10-01

View user profile http://abouthirundo.blogspot.com

Back to top Go down

Re: The science of psychological assessment

Post by jcorozza on Wed Apr 08, 2015 11:04 am

Hirundo Bos wrote:

I'm not really sure what's the case with the MMPI, but in general, both situational and long term values can be measured with questionnaires... referred to as states and traits, respectively... Although I think it's most common to speak about long term traits. Something that, if it is the case, can make traits appear more static than they are. Which again might feed stereotypes, bringing us back to Prajna's post.

I think some can, but based on some of the questions...this one does a poor job. The problem is that so much of it is set up as absolutes, and since you can't answer a "sometimes", you often have to decide what answer is the least false, if that makes any sense.
avatar
jcorozza

Posts : 418
Reputation : 167
Join date : 2015-03-08

View user profile

Back to top Go down

View previous topic View next topic Back to top


 
Permissions in this forum:
You cannot reply to topics in this forum