The idea that there’s a single scalar value that measures anything like “general intelligence” (“g”), commonly known as “IQ” or “intelligence quotient”, has been pretty much blown out of the water by this comprehensive study by the University of Western Ontario’s Brain and Mind Institute.
Our attempt to answer [the question of how to quantify relative intelligence] dates back more than five years, when Roger [Highfield] encountered work that I had conducted with Adrian [Hampshire] at the Medical Research Council in Cambridge on a reliable way to carry out cognitive tests online so we could monitor rehabilitation after brain injury, the effect of smart drug trials and so on.
Roger wondered if we could use this test to carry out a mass intelligence test. Drawing on earlier data from brain scans, Adrian and I came up with a series of tests which we knew would trigger activity in as much of the brain’s anatomy as possible, combining the fewest tasks to cover the broadest range of cognitive skills.
We expected a few hundred responses. But thanks to articles in The Daily Telegraph, Discovery and New Scientist, 110,000 people took part from every corner of the world. Once I had used statistical methods to analyse more than a million data points on a representative group of around 45,000, I found that when a wide range of cognitive abilities are probed, the variations in performance can only be explained with at least three distinct components: short-term memory; reasoning; and finally, a verbal component.
No one component, or ‘IQ’, explained all the variations revealed by the tests.
To bolster our results, Adrian and I used a $5 million brain scanner, which relies on a technique known as functional magnetic resonance imaging (fMRI), to study 16 participants as they carried out all 12 tests. We found that each of the three different factors identified by the analysis did indeed correspond to a different brain network: these differences in cognitive ability map onto three distinct circuits in the brain.
The results disprove once and for all the idea that a single measure of intelligence, such as ‘IQ’, is enough to capture all of the differences in cognitive ability that we see between people. Instead, several different brain circuits contribute to intelligence, each with its own unique capacity. A person may well be good in one of these areas, but they are just as likely to be bad in the other two.
I look forward to further verification of the results from the fMRIs, but the ginormous sample size for the actual study (110,000?! That’s absurd!) is pretty much exactly what we need to prove that intelligence is not so simple as it has been abstracted in popular culture.
I further look forward to future ‘race realists’ and other such science-abusing motivated reasoners being forced to hang their prejudices on SEVERAL scalar values, instead of just one.
27 thoughts on “IQ test exposes the myth that is 'g'”
“I further look forward to future ‘race realists’ and other such science-abusing motivated reasoners being forced to hang their prejudices on SEVERAL scalar values, instead of just one.”
I’m not getting my hopes up here.
These people wouldn’t need to look any further than gender essentialists and copy there rhethoric. The smart ones manage to things about suitability for math and sciences vs. linguistic or social skills and then insinuate one set is superior instead of making blanket statements about women being generally “dumber”, only “different”. I’m sure they could pick that trick up pretty soon.
What pisses me off about such race claims (or any claims about group differences in general) has little to do with there being a common factor – and a lot to do with essentialism and dishonest arguments about heritability. It’s not so much in assuming a common factor, but in making assupmtions about what drives it and what drives the differences. As though finding a high heritability estimate within a sample would tell us anything whatsoever about systematic differences between groups. (Btw, Eric Turkheimer has found very low (nonsignificant?) heritability for intelligence among children from marginalized groups. )
The genetic determinst view on intelligence has been a pathetic excuse for accepting a status quo marginalization and discrimination for a long, long time.
No study needed to establish that.
That being said, it doesn’t look as sensational as it is presented, since multiple layer models of intelligence and specific abilities are nothing new. It’s absolutely clear that “g” can’t account for all, or even most of the variance. In order to ultimately disprove “g” (as an additional layer) you’d need to show that these three factors are statistically independent.
I wish people would refrain from using language like this.
I hope that in the original article they give a shout out to S. J. Gould, who made this argument about factor analysis in depth a few decades ago.
I once heard IQ tests described as “knowing enough to live in the culture that wrote the test”, and not necessarily about being smart. The writers have an ingrained bias in their tests.
I doubt anyone reading this could live the way African Bushmen tribes do, knowing where and how to find food and water or how to store it the way they do. If they wrote the IQ tests, most of us would be rated as morons while they would be geniuses.
And keep in mind that these claims involve the combination of the nonsense of “race” with the nonsense of “g.”
Yes, cultural bias most likely plays a role, and not only when the content is verbal.
Luckily, someone fixed it (well, actually just exposed it) a while ago:
Kind of chuckled at this article. My psychology professor from way back in the 80s commented on IQ tests as basically being good at what they measure, but no one knows what the tests are really measuring. Even back then, they knew that IQ tests didn’t really measure intelligence.
Article: Wikipedia – g factor (psychometrics)
So a single number would still work. It’d just be attributed to a rough model lumping together the influences from several bits of anatomy (unless a localized smartness module was expected, this isn’t news). And the people only looking at test scores to play good brain / bad brain with a color wheel will be unaffected.
Well, they’re not too bad when used for the very simple original purposes of determining whether, and by how much, people might fall below the population average. Good if you want a crude cut-off point or dividing line for accepting/rejecting army conscripts or for allocating special needs education services. Not much good for determining which out of a dozen clever people is “cleverest”.
Even then, learning needs and learning delays are now well understood as being particular problems which are best dealt with in a targeted way. The crude measures of the past were really only useful for crude applications like which children/adults were admitted, or even forced, into certain institutions.
Very interesting link.
I just skimmed the cell/neuron paper and it shows some very promising methods and yes they cite Gould(1981) TMM. The press coverage and the linked blog article are however quite misleading.
First this is not even close to being the biggest study of mental capabilities. At least in the US massive intelligence tests for military recruits, immigrants and students applying for college were common for decades. The SAT is taken by more than a million students each year, and is very similar to an IQ test for practical purposes.
Second it is not a cohort study. This is a self selected sample of people who are interested in science and have the verbal capabilities to enjoy science articles in blogs and journals. The main findings of the survey are summarized in Fig 4 in the original paper. 4a) shows that verbal is the only score that is almost flat, even though normally verbal scores increase with age across a fairly large range (this supports the self-selection hypothesis or a non-conventional verbal score that is not related to vocabulary). Moreover Fig 4c ) shows that all sub-scores are highly correlated with educational achievement- this favours the use of g for admission and education purposes and does not debunk it.
The paper shows that measuring a g- like score of somebody you are most likely to meet here, e.g. educated skeptics interested in reading science articles will not lead to new information unless you look at partial scores at a finer resolution. This is a limited, but important finding. There were also no differences in central tendencies between male and female participants, which is consistent with prior expectation.
The most important practical implication of the paper is stop smoking, it is bad for short term memory (alcohol and coffee seem to be ok).
If we are interested in debunking the concept of g or at least reducing its influence on college admission and military careers the methodology should be applied to a more representative sample. The methods in the paper are very interesting and as an online test are easily adapted to test for example a cohort of high school students in a district, senior citizens or patients in a hospital. I hope the authors will find interest and funding to follow up on their goal to debunk g with data and don’t stop at preliminary results.
I wonder what MENSA thinks of this?
“Intelligence” is one of those words that can mean whatever you want it to mean. Even Mensa will acknowledge that they test for a particular set of intelligence criteria — reasoning ability, pattern recognition and memory — and not for everything that can be classified as intelligence.
It sounds like these researchers came up with a novel definition of intelligence, devised a test for it and, not surprisingly, found that it ranked people differently when compared to a different test using a different definition of intelligence. It’s like comparing results of people who took both a calculus exam and a test on inductive proofs, and trying to conclude that there is no such thing as math aptitude.
Video: David Mitchell’s Soapbox – Mensa
What does brain area, or regions, or networks mean? Are they saying that executive function and long term memory are not part of intelligence? Or that they are if sufficient areas of the brain are encompassed?
And nobody that I’ve heard of thinks of IQ scores as not being limited, and subject to variable ability within selective ranges, or scores. Motivation is as large a predictor of performance as mental acuity. Anxiety can play a huge role in the results of a single test, IQ or otherwise.. Learning disability can skew results badly.
Of course, no rating, or g, covers everything. IQ is a predictor of a probability of success, all things being equal, and that includes social, or societal environments.
Is any of this even news to anyone?(rising inflection on ‘anyone’).
I(obviously), don’t understand the point of this research:
No fucking kidding.
Sorry, but you here show as much knowledge about this topic as a creationist touting the latest study that supposedly “once and for all” disproves evolution.
There is a single scalar value (“g”) which accounts for a very large portion of the variance in performance on various cognitive tests. This is due to the positive inter-correlation in test scores (the “positive manifold”). There are copious amounts of data supporting this. There is no debate whatsoever about this in the intelligence research community. “g” of course does not account for all of the variance. But yes it definitely measures something like “general intelligence” when “general intelligence” is defined as “something which predicts performance on cognitive tests” (and how else would it sensibly be defined)?
The only debate is, since one can orthogonally rotate the principal components, one can have performance on cognitive tests predicted by multiple factors, and not just a single factor (which is the debate over whether intelligence is unitary or composed of multiple factors). Even so, there will still be a single scalar, which is the appropriate linear combination of the multiple factors, and which predicts test performance.
“g” captures a large portion of the differences. Not all of them. Everyone in the field knows this. Really, this “objection” is on par with “if man came from apes, why are there still apes?”, showing elementary ignorance of the field (in the case of evolution, that it acts on population, not on individuals; in the case of intelligence, that all of the variance is captured by a single factor).
No kidding. Is anything ever as simple as it is in pop culture?
Again, imagining this paper will have any impact on “race realism” shows you don’t understand the field.
@9: You realize that the “blog post” that I linked to is reproducing a press release written by one of the paper’s primary authors, right?
@14: Please be sure to tell the paper’s authors that they show as much knowledge of their field as creationists of evolution. Please also tell them — scientists at UWO — that everyone in their field knows the things they just proved, and that there is no debate whatsoever about the thing that they’re debating about.
@3 – When I was in college one professor borrowed the class I was in and gave us an “IQ” test. I scored as barely educable, and that was better than most of the class.
It was a pictoral school readiness test for Japanese children. 🙁
It’s interesting in this context that IQ tests, which supposedly measure “general intelligence” have had to be repeatedly restandardised due to rising scores* (the “Flynn Effect”), and that the gains are greater in so-called “fluid intelligence” than in the supposedly more culture-bound “crystallised intelligence”.
*This rise may now have halted in some populations.
Their research is somewhat interesting but it does not, in any way, discredit the existing literature on IQ or on “g”. About 1960 it was demonstrated to my satisfaction that IQ is actually visible to a trained observer during ordinary conversation (among the students at my college).
I will not miss IQ – it made as much sense as trying to describe an engine, transmission, drive train, and tires with a single number. The number “horsepower” tells you so little about these 4 systems – for really effective acceleration in a straight line you would need to know about the engine power curve, torque output, the number of shifts available, 2 wheel or 4 wheel drive, and the suitability of the tires to the road, among others. Trying to encapsulate all in a single number is a recipe to fool yourself into thinking you know more than you really do, and to not only compare apples or oranges, but to try to grow orange trees from apple seeds.
But less controversial psychological constructs like the big five personality traits [ http://en.wikipedia.org/wiki/Big_Five_personality_traits ] have comparable statistical validity as IQ. Will brain imagining leave these in bad shape too?
I’m actually going to lean towards reinforcing what #7, Sky Captain said.
At least in my understanding of current psychometrics teaching, (g) isn’t a single thing, but a model for the high correlations between realms of understanding, which are separate. The WAIS (most common test of adult intelligence) currently has a final score, the measure of (g) and a separate composite score that separates out the influence of working memory and processing speed, as well as subtest scores.
w/r/t #19, yes, the Big Five Personality test is statistically very very sound. However, it isn’t actually measuring the same construct as an IQ test, so I’d argue that they aren’t comparable. Delay of Gratifications tests, however, may have better predictive power in the same realm as IQ, and could be compared, though arguably it’s not measuring intelligence at all.
One of my sons was given a Wechsler test and declared to have average intelligence. He actually scored in the top one percentile in one sub-test and in the bottom one percentile in another. He continues to amaze.
I think it is important to remember that the old “IQ” tests were originally designed to test sub-normal intelligence, and were co-opted to prove the undesirability of southern Europeans and Asians, in aid of limiting their immigration. The tests showed that the English were much smarter than Italians (that is, the English did better on the test). The fact that the English understood the language and shared a fairly large part of their culture with the test-writers was not considered, because they were getting the answers they wanted*.
I agree with whoever upthread mentioned Gould’s The Mis-Measure of Man. Some of it is probably pretty out of date, but the historic stuff is really interesting and tends to make you highly suspicious of “IQ” testing. Besides, it’s Gould.
*Why, yes, the descendants of those who were discriminated against during that period are very willing to treat other people the way their great grandparents were treated.
I and my husband became quite adept at predicting IQ scores for school students whose parents brought along a psychological assessment that we could check ourselves against. Of course, when it came to dealing with learning problems, the identification of individual skills and problems is far more important when designing targeted tuition programs. But knowing that a (very) few students had a significant general learning problem was also very helpful in assessing progress and designing programs with a lot more repetition of much more finely ‘divided’ tasks.
And the bloke who trained us? He could guess a prospective student’s IQ in a screening interview of 5-10 minutes. When it came to adults it took him a bit longer – but he rarely made a mistake.
Would the results have been more impressive if the fMRI had cost 10 million. Just a thought. Whatever the cost, the images primarily are looking at changes in blood flow which is already an indirect measure of brain function. Interaction with individuals seems to still be more meaningful in determining a level of function.
alanuk @21: WAIS and (child version) WISC test total scores are not valid when subtest scores vary that much (unless the outlier is an optional extra). If someone looked at results like that and came up with “average intelligence” (and not something like “at least average”), they didn’t know how to use the test.
Jason @15: Are you saying you’ve never heard of researchers exaggerating the new, unique and amazing nature of their research in a press release? The only thing new here is the imaging, at least I haven’t seen that before. As Kate Donovan points out, the WAIS-IV gives you a total score (g), but also scores for verbal skills, nonverbal reasoning skills, and short term memory and processing speed. Sound familiar?
Anne @25: I’m not saying that at all. It could be exaggerated all to hell compared to the actual study’s results, which people have interpreted differently from the primary authors. What I was saying at 15 was that people trying to undercut it as “a blog post” or exhibiting as much knowledge of the field as creationists of evolution or talking about stuff that are totally not controversial at all in the field, are wrong.
Anne, I almost couldn’t believe that processing speed was a part of the evaluation. I work with computers, where processor speed is also a measure of ‘ability'( I know, no surprise to most people).
Jason, these guys may, or may not, be on to something, but my clock speed on the WISC-III was lower, by 15 points, than my other scores: FSIQ = x, VIQ = x+3, PIQ = x-23. Frontoparietal cortical atrophy was shown in a CT scan.
This has an impact on overall IQ and general performance as my executive function is impaired, and thus a lower overall score than expected.
– – –
You know what? These guys, at the University of Western Ontario’s Brain and Mind Institute, may, or may not, be on to something:
– – – –
They seem to be dead wrong:They aren’t on to something:
One location, one network that includes most areas of the brain under control from a primary node.
– – –
Here is more:
– – –
This one has CT and fMRI pictures!
One localized area of the brain, where our executive function is seated, that encompasses control – directly or indirectly – over the whole brain network.
I figure that artists, athletes that excel , scientists, janitors(me!), lol, etc., are marked by high executive functioning and creativity, which uses executive tasks.
There, anyone can write a paper that sounds good!
What a fucking long comment
Comments are closed.