The test quality project - part 1: Norwegian psychologists' test attitudes and test use, 2021, Ryder

[Midnattsol]

Copied from the thread "Fear conditioning as a pathogenic mechanism in the postural tachycardia syndrome, Norcliffe-Kaufmann et al, 2022"

I don’t understand how people don’t kick these charlatans out of the profession. My guess is that the broader medical establishment feels these conditions don’t warrant legitimate research and should be de-medicalized. Thus, there is absolutely no scrutiny or oversight.

I think another problem is that it is easy to think that whatever test was used in a published paper is a validated one. After all, it's been peer reviewed! The number of times I've explained that certain tests for anxiety or depression doesn't work for pwIllness due to symptom overlap I'm left with blank stares and/or that the authors must have thought of that. A few months (or maybe two years? can't quite remember) a psychologist wrote an opinion piece on how many scales used in clinical practice should not be used like that because they were tools developed for research and had not been validated for a clinical setting. She argued that unfortunately we use them because we lack something else (and the knowledge that they are not validated).

Edit: The opinion piece was from last year, and I might have misrememberd some of her arguments. Then again there were multiple pieces written since she got replies etc.. but this seems to be the first one:

Imagine that your GP on your own initiative had made a test at home on the kitchen counter and used it to check if you were ill. Or that your doctor left it to the health secretary to interpret the test result. What would it have done with confidence in the test result, in the doctor, yes, in the health care system?

For most people, the answer is self-explanatory. Transferred to mental health practice, the answer is not as obvious. A large number of psychological tests and surveys are performed daily in the health service. There are key tools in the work to investigate and diagnose patients with mental disorders.

...

But today psychological tests are used such as:

• is not sufficiently quality assured,
• does not follow international standards and guidelines for testing
• and which are imported from outside, but are not authorized for and translated into Norwegian conditions.

Google translate: No one will take responsibility for quality assurance of psychological tests

She refers to a study on test use, and it is a very nice read (although it is Norwegian centered and touches upon translation and localisation issues that will not be applicable in all settings) written about in the article Test field without control and the actual study can be found here: The test quality project - part 1: Norwegian psychologists' test attitudes and test use

They are all worth reading

 

[Midnattsol]

On the study:
The Norwegian Psychological Association (NPF) has long been concerned about the test field and believes that more actors must contribute to raising the quality of a landscape that is characterized by a lack of overview of tools, and lack of compliance with international standards and guidelines for testing. Based on previous studies that showed quality problems in the test field, in the autumn of 2017 NPF entered into a collaboration with the Norwegian Directorate of Health and the National Institute of Public Health (NIPH) on a recent survey of the test field, where 1400 Norwegian psychologists reported their attitudes to test quality and test use. The purpose was to provide an updated overview of challenges in the test field and to identify which tests were widely used and at the same time had problems related to copyright, translation, Norwegian standard data, reliability or validity, to be able to assess possible corrective measures. This article describes the first part of the project: the design of the survey, the results about psychologists 'attitudes to test quality, which tests they use, and information revealed about the tests' quality status at the Health Library, PsykTestBarn and FHI. The results of the most frequently used tests with quality questions are presented in Test Quality Project Part 2.

Part 2 of the study can be found here: The test quality project - part 2: Tests with a need for quality measures

While it is mainly concerned with how the tests/questionnaires are used in a Norwegian context, it brings up important topics such as what population a test is valid for (here cultural norms between countries, but similar issues arise between healthy and ill populations).