IQ test exposes the myth that is 'g'

The idea that there’s a single scalar value that measures anything like “general intelligence” (“g”), commonly known as “IQ” or “intelligence quotient”, has been pretty much blown out of the water by this comprehensive study by the University of Western Ontario’s Brain and Mind Institute.

Our attempt to answer [the question of how to quantify relative intelligence] dates back more than five years, when Roger [Highfield] encountered work that I had conducted with Adrian [Hampshire] at the Medical Research Council in Cambridge on a reliable way to carry out cognitive tests online so we could monitor rehabilitation after brain injury, the effect of smart drug trials and so on.

Roger wondered if we could use this test to carry out a mass intelligence test. Drawing on earlier data from brain scans, Adrian and I came up with a series of tests which we knew would trigger activity in as much of the brain’s anatomy as possible, combining the fewest tasks to cover the broadest range of cognitive skills.

[…]
We expected a few hundred responses. But thanks to articles in The Daily Telegraph, Discovery and New Scientist, 110,000 people took part from every corner of the world. Once I had used statistical methods to analyse more than a million data points on a representative group of around 45,000, I found that when a wide range of cognitive abilities are probed, the variations in performance can only be explained with at least three distinct components: short-term memory; reasoning; and finally, a verbal component.

No one component, or ‘IQ’, explained all the variations revealed by the tests.

To bolster our results, Adrian and I used a $5 million brain scanner, which relies on a technique known as functional magnetic resonance imaging (fMRI), to study 16 participants as they carried out all 12 tests. We found that each of the three different factors identified by the analysis did indeed correspond to a different brain network: these differences in cognitive ability map onto three distinct circuits in the brain.

The results disprove once and for all the idea that a single measure of intelligence, such as ‘IQ’, is enough to capture all of the differences in cognitive ability that we see between people. Instead, several different brain circuits contribute to intelligence, each with its own unique capacity. A person may well be good in one of these areas, but they are just as likely to be bad in the other two.

Emphasis mine.

I look forward to further verification of the results from the fMRIs, but the ginormous sample size for the actual study (110,000?! That’s absurd!) is pretty much exactly what we need to prove that intelligence is not so simple as it has been abstracted in popular culture.

I further look forward to future ‘race realists’ and other such science-abusing motivated reasoners being forced to hang their prejudices on SEVERAL scalar values, instead of just one.

{advertisement}
IQ test exposes the myth that is 'g'
{advertisement}

27 thoughts on “IQ test exposes the myth that is 'g'

  1. 1

    “I further look forward to future ‘race realists’ and other such science-abusing motivated reasoners being forced to hang their prejudices on SEVERAL scalar values, instead of just one.”

    I’m not getting my hopes up here.
    These people wouldn’t need to look any further than gender essentialists and copy there rhethoric. The smart ones manage to things about suitability for math and sciences vs. linguistic or social skills and then insinuate one set is superior instead of making blanket statements about women being generally “dumber”, only “different”. I’m sure they could pick that trick up pretty soon.

    What pisses me off about such race claims (or any claims about group differences in general) has little to do with there being a common factor – and a lot to do with essentialism and dishonest arguments about heritability. It’s not so much in assuming a common factor, but in making assupmtions about what drives it and what drives the differences. As though finding a high heritability estimate within a sample would tell us anything whatsoever about systematic differences between groups. (Btw, Eric Turkheimer has found very low (nonsignificant?) heritability for intelligence among children from marginalized groups. )
    The genetic determinst view on intelligence has been a pathetic excuse for accepting a status quo marginalization and discrimination for a long, long time.
    No study needed to establish that.

    That being said, it doesn’t look as sensational as it is presented, since multiple layer models of intelligence and specific abilities are nothing new. It’s absolutely clear that “g” can’t account for all, or even most of the variance. In order to ultimately disprove “g” (as an additional layer) you’d need to show that these three factors are statistically independent.

  2. 2

    The results disprove once and for all…

    I wish people would refrain from using language like this.

    the idea that a single measure of intelligence, such as ‘IQ’, is enough to capture all of the differences in cognitive ability that we see between people. Instead, several different brain circuits contribute to intelligence, each with its own unique capacity. A person may well be good in one of these areas, but they are just as likely to be bad in the other two.

    I hope that in the original article they give a shout out to S. J. Gould, who made this argument about factor analysis in depth a few decades ago.

  3. 3

    I once heard IQ tests described as “knowing enough to live in the culture that wrote the test”, and not necessarily about being smart. The writers have an ingrained bias in their tests.

    I doubt anyone reading this could live the way African Bushmen tribes do, knowing where and how to find food and water or how to store it the way they do. If they wrote the IQ tests, most of us would be rated as morons while they would be geniuses.

  4. 6

    Kind of chuckled at this article. My psychology professor from way back in the 80s commented on IQ tests as basically being good at what they measure, but no one knows what the tests are really measuring. Even back then, they knew that IQ tests didn’t really measure intelligence.

  5. 7

    @OP:

    “But does one number really represent an individual’s ability to remember, to reason and to think? The answer is an emphatic no, according to the results
    […]
    No one component, or ‘IQ’, explained all the variations revealed by the tests.”

    Article: Wikipedia – g factor (psychometrics)

    “It is a variable that summarizes the consistent finding of positive correlations among different cognitive tasks, reflecting the fact that an individual’s performance at one type of cognitive task tends to be comparable to his or her performance at other kinds of cognitive tasks. The g factor typically accounts for 40 to 50 percent of the variance in IQ test performance
    […]
    While the existence of g as a statistical regularity is well-established and uncontroversial, there is no consensus as to what causes the positive correlations between tests.

    So a single number would still work. It’d just be attributed to a rough model lumping together the influences from several bits of anatomy (unless a localized smartness module was expected, this isn’t news). And the people only looking at test scores to play good brain / bad brain with a color wheel will be unaffected.

  6. 8

    Even back then, they knew that IQ tests didn’t really measure intelligence.

    Well, they’re not too bad when used for the very simple original purposes of determining whether, and by how much, people might fall below the population average. Good if you want a crude cut-off point or dividing line for accepting/rejecting army conscripts or for allocating special needs education services. Not much good for determining which out of a dozen clever people is “cleverest”.

    Even then, learning needs and learning delays are now well understood as being particular problems which are best dealt with in a targeted way. The crude measures of the past were really only useful for crude applications like which children/adults were admitted, or even forced, into certain institutions.

  7. 9

    Very interesting link.
    I just skimmed the cell/neuron paper and it shows some very promising methods and yes they cite Gould(1981) TMM. The press coverage and the linked blog article are however quite misleading.

    First this is not even close to being the biggest study of mental capabilities. At least in the US massive intelligence tests for military recruits, immigrants and students applying for college were common for decades. The SAT is taken by more than a million students each year, and is very similar to an IQ test for practical purposes.

    Second it is not a cohort study. This is a self selected sample of people who are interested in science and have the verbal capabilities to enjoy science articles in blogs and journals. The main findings of the survey are summarized in Fig 4 in the original paper. 4a) shows that verbal is the only score that is almost flat, even though normally verbal scores increase with age across a fairly large range (this supports the self-selection hypothesis or a non-conventional verbal score that is not related to vocabulary). Moreover Fig 4c ) shows that all sub-scores are highly correlated with educational achievement- this favours the use of g for admission and education purposes and does not debunk it.
    The paper shows that measuring a g- like score of somebody you are most likely to meet here, e.g. educated skeptics interested in reading science articles will not lead to new information unless you look at partial scores at a finer resolution. This is a limited, but important finding. There were also no differences in central tendencies between male and female participants, which is consistent with prior expectation.

    The most important practical implication of the paper is stop smoking, it is bad for short term memory (alcohol and coffee seem to be ok).
    If we are interested in debunking the concept of g or at least reducing its influence on college admission and military careers the methodology should be applied to a more representative sample. The methods in the paper are very interesting and as an online test are easily adapted to test for example a cohort of high school students in a district, senior citizens or patients in a hospital. I hope the authors will find interest and funding to follow up on their goal to debunk g with data and don’t stop at preliminary results.

  8. 11

    “Intelligence” is one of those words that can mean whatever you want it to mean. Even Mensa will acknowledge that they test for a particular set of intelligence criteria — reasoning ability, pattern recognition and memory — and not for everything that can be classified as intelligence.

    It sounds like these researchers came up with a novel definition of intelligence, devised a test for it and, not surprisingly, found that it ranked people differently when compared to a different test using a different definition of intelligence. It’s like comparing results of people who took both a calculus exam and a test on inductive proofs, and trying to conclude that there is no such thing as math aptitude.

  9. 13

    What does brain area, or regions, or networks mean? Are they saying that executive function and long term memory are not part of intelligence? Or that they are if sufficient areas of the brain are encompassed?
    And nobody that I’ve heard of thinks of IQ scores as not being limited, and subject to variable ability within selective ranges, or scores. Motivation is as large a predictor of performance as mental acuity. Anxiety can play a huge role in the results of a single test, IQ or otherwise.. Learning disability can skew results badly.
    Of course, no rating, or g, covers everything. IQ is a predictor of a probability of success, all things being equal, and that includes social, or societal environments.

    Is any of this even news to anyone?(rising inflection on ‘anyone’).

    I(obviously), don’t understand the point of this research:

    The results disprove once and for all the idea that a single measure of intelligence, such as ‘IQ’, is enough to capture all of the differences in cognitive ability that we see between people. Instead, several different brain circuits contribute to intelligence, each with its own unique capacity. A person may well be good in one of these areas, but they are just as likely to be bad in the other two.

    No fucking kidding.

  10. 14

    Sorry, but you here show as much knowledge about this topic as a creationist touting the latest study that supposedly “once and for all” disproves evolution.

    The idea that there’s a single scalar value that measures anything like “general intelligence” (“g”), commonly known as “IQ” or “intelligence quotient”, has been pretty much blown out of the water

    There is a single scalar value (“g”) which accounts for a very large portion of the variance in performance on various cognitive tests. This is due to the positive inter-correlation in test scores (the “positive manifold”). There are copious amounts of data supporting this. There is no debate whatsoever about this in the intelligence research community. “g” of course does not account for all of the variance. But yes it definitely measures something like “general intelligence” when “general intelligence” is defined as “something which predicts performance on cognitive tests” (and how else would it sensibly be defined)?

    The only debate is, since one can orthogonally rotate the principal components, one can have performance on cognitive tests predicted by multiple factors, and not just a single factor (which is the debate over whether intelligence is unitary or composed of multiple factors). Even so, there will still be a single scalar, which is the appropriate linear combination of the multiple factors, and which predicts test performance.

    The results disprove once and for all the idea that a single measure of intelligence, such as ‘IQ’, is enough to capture all of the differences in cognitive ability that we see between people.

    “g” captures a large portion of the differences. Not all of them. Everyone in the field knows this. Really, this “objection” is on par with “if man came from apes, why are there still apes?”, showing elementary ignorance of the field (in the case of evolution, that it acts on population, not on individuals; in the case of intelligence, that all of the variance is captured by a single factor).

    …intelligence is not so simple as it has been abstracted in popular culture.

    No kidding. Is anything ever as simple as it is in pop culture?

    I further look forward to future ‘race realists’ and other such science-abusing motivated reasoners being forced to hang their prejudices on SEVERAL scalar values, instead of just one.

    Again, imagining this paper will have any impact on “race realism” shows you don’t understand the field.

  11. 15

    @9: You realize that the “blog post” that I linked to is reproducing a press release written by one of the paper’s primary authors, right?

    @14: Please be sure to tell the paper’s authors that they show as much knowledge of their field as creationists of evolution. Please also tell them — scientists at UWO — that everyone in their field knows the things they just proved, and that there is no debate whatsoever about the thing that they’re debating about.

  12. 16

    @3 – When I was in college one professor borrowed the class I was in and gave us an “IQ” test. I scored as barely educable, and that was better than most of the class.

    It was a pictoral school readiness test for Japanese children. 🙁

  13. 18

    Their research is somewhat interesting but it does not, in any way, discredit the existing literature on IQ or on “g”. About 1960 it was demonstrated to my satisfaction that IQ is actually visible to a trained observer during ordinary conversation (among the students at my college).

  14. 19

    I will not miss IQ – it made as much sense as trying to describe an engine, transmission, drive train, and tires with a single number. The number “horsepower” tells you so little about these 4 systems – for really effective acceleration in a straight line you would need to know about the engine power curve, torque output, the number of shifts available, 2 wheel or 4 wheel drive, and the suitability of the tires to the road, among others. Trying to encapsulate all in a single number is a recipe to fool yourself into thinking you know more than you really do, and to not only compare apples or oranges, but to try to grow orange trees from apple seeds.

    But less controversial psychological constructs like the big five personality traits [ http://en.wikipedia.org/wiki/Big_Five_personality_traits ] have comparable statistical validity as IQ. Will brain imagining leave these in bad shape too?

  15. 20

    I’m actually going to lean towards reinforcing what #7, Sky Captain said.

    At least in my understanding of current psychometrics teaching, (g) isn’t a single thing, but a model for the high correlations between realms of understanding, which are separate. The WAIS (most common test of adult intelligence) currently has a final score, the measure of (g) and a separate composite score that separates out the influence of working memory and processing speed, as well as subtest scores.

    w/r/t #19, yes, the Big Five Personality test is statistically very very sound. However, it isn’t actually measuring the same construct as an IQ test, so I’d argue that they aren’t comparable. Delay of Gratifications tests, however, may have better predictive power in the same realm as IQ, and could be compared, though arguably it’s not measuring intelligence at all.

  16. 21

    One of my sons was given a Wechsler test and declared to have average intelligence. He actually scored in the top one percentile in one sub-test and in the bottom one percentile in another. He continues to amaze.

  17. 22

    I think it is important to remember that the old “IQ” tests were originally designed to test sub-normal intelligence, and were co-opted to prove the undesirability of southern Europeans and Asians, in aid of limiting their immigration. The tests showed that the English were much smarter than Italians (that is, the English did better on the test). The fact that the English understood the language and shared a fairly large part of their culture with the test-writers was not considered, because they were getting the answers they wanted*.

    I agree with whoever upthread mentioned Gould’s The Mis-Measure of Man. Some of it is probably pretty out of date, but the historic stuff is really interesting and tends to make you highly suspicious of “IQ” testing. Besides, it’s Gould.
    .
    .

    *Why, yes, the descendants of those who were discriminated against during that period are very willing to treat other people the way their great grandparents were treated.

  18. 23

    About 1960 it was demonstrated to my satisfaction that IQ is actually visible to a trained observer during ordinary conversation (among the students at my college).

    I and my husband became quite adept at predicting IQ scores for school students whose parents brought along a psychological assessment that we could check ourselves against. Of course, when it came to dealing with learning problems, the identification of individual skills and problems is far more important when designing targeted tuition programs. But knowing that a (very) few students had a significant general learning problem was also very helpful in assessing progress and designing programs with a lot more repetition of much more finely ‘divided’ tasks.

    And the bloke who trained us? He could guess a prospective student’s IQ in a screening interview of 5-10 minutes. When it came to adults it took him a bit longer – but he rarely made a mistake.

  19. 24

    Would the results have been more impressive if the fMRI had cost 10 million. Just a thought. Whatever the cost, the images primarily are looking at changes in blood flow which is already an indirect measure of brain function. Interaction with individuals seems to still be more meaningful in determining a level of function.

  20. 25

    alanuk @21: WAIS and (child version) WISC test total scores are not valid when subtest scores vary that much (unless the outlier is an optional extra). If someone looked at results like that and came up with “average intelligence” (and not something like “at least average”), they didn’t know how to use the test.
    Jason @15: Are you saying you’ve never heard of researchers exaggerating the new, unique and amazing nature of their research in a press release? The only thing new here is the imaging, at least I haven’t seen that before. As Kate Donovan points out, the WAIS-IV gives you a total score (g), but also scores for verbal skills, nonverbal reasoning skills, and short term memory and processing speed. Sound familiar?

  21. 26

    Anne @25: I’m not saying that at all. It could be exaggerated all to hell compared to the actual study’s results, which people have interpreted differently from the primary authors. What I was saying at 15 was that people trying to undercut it as “a blog post” or exhibiting as much knowledge of the field as creationists of evolution or talking about stuff that are totally not controversial at all in the field, are wrong.

  22. 27

    Anne, I almost couldn’t believe that processing speed was a part of the evaluation. I work with computers, where processor speed is also a measure of ‘ability'( I know, no surprise to most people).
    Jason, these guys may, or may not, be on to something, but my clock speed on the WISC-III was lower, by 15 points, than my other scores: FSIQ = x, VIQ = x+3, PIQ = x-23. Frontoparietal cortical atrophy was shown in a CT scan.
    This has an impact on overall IQ and general performance as my executive function is impaired, and thus a lower overall score than expected.
    – – –

    You know what? These guys, at the University of Western Ontario’s Brain and Mind Institute, may, or may not, be on to something:

    To what extent are some people smarter than others? For a century, psychologists have believed that we can boil differences in cognitive skill down to a single number known as an Intelligence Quotient (IQ). But does one number really represent an individual’s ability to remember, to reason and to think? The answer is an emphatic no, according to the results of a landmark experiment conducted on many tens of thousands of people with the help of Roger Highfield of the Science Museum Group.

    – – – –
    They seem to be dead wrong:They aren’t on to something:

    Executive Control and the Prefrontal Cortex

    Evidence for a Frontoparietal Control System Revealed by Intrinsic Functional Connectivity:
    Our current research further examines the functional networks underlying executive control, providing lesion evidence for a distributed fronto-parietal circuit that principally involves dorsolateral PFC and superior parietal cortex (Barbey et al., in press c ; Koenigs et al., 2009). We have investigated the neural substrates of the general factor of intelligence (g) and executive function in 182 patients with focal brain lesions using voxel-based lesion-symptom mapping (Barbey et al., in press c).We observed a significant effect on g and executive function with lesions in left hemispheric white matter sectors including the superior longitudinal/arcuate fasciculus that connect frontal and parietal cortices. Despite its distributed nature, the neural substrates of g and executive function were remarkably circumscribed, concentrated in the core of white matter, and comprising a narrow subset of regions associated with performance on individual Wechsler Adult Intelligence Scale (WAIS) and Delis-Kaplan Executive Function System (D-KEFS) subtests. The largest overlap between WAIS subtests and g was found for Verbal Comprehension and Working Memory, and for executive function measures of the D-KEFS (i.e., Trail Making, Verbal Fluency, Card Sorting, and Twenty Questions). Collectively, these subtests assess verbal knowledge about the world, verbal reasoning, working memory capacity, as well as cognitive flexibility and executive control, and are associated with a distributed fronto-parietal network. This suggests that g and executive function draw on the combination of conceptual knowledge and executive processes, and that the communication between areas associated with these capacities is of critical importance.

    The observed findings contribute to a growing body of neuropsychological patient evidence indicating that damage to a distributed network of frontal and parietal regions is associated with impaired performance on tests of general intelligence (Gläscher et al., 2010, 2009; Chiang et al., 2009; Colom & Thompson, 2011; Colom et al., 2009; Jung & Haier, 2007). A recent study by Gläscher and colleagues (2010) applied voxel-based lesion-symptom mapping to elucidate the neural substrates of psychometric g, reporting a left lateralized fronto-parietal network that converges with the observed pattern of findings and further supports the role of this network in components of general intelligence that draw upon conceptual knowledge and working memory. Our findings advance this line of research by elucidating the relationship between general intelligence and executive function – demonstrating that these domains recruit a highly overlapping and broadly distributed network of frontal and parietal regions.

    The fronto-parietal network identified by the present analysis includes lateral frontopolar cortex, anterior PFC, dorsolateral PFC, anterior cingulate/medial PFC, and the inferior and superior parietal lobe. This constellation of regions is commonly engaged by tasks that require executive control processes (for reviews, see Botvinick et al. 2004; Dosenbach et al. 2007; Gruber & Goshke 2004; Ramnani & Owen 2004). The fronto-parietal network is recruited by paradigms that elicit controlled processing related to the simultaneous consideration of multiple interdependent contingencies (Kroger et al. 2002), conflicting stimulus-response mappings (Crone et al. 2006), and integrating working memory with attentional resource allocation (Koechlin et al. 1999). In addition, many of the regions in the fronto-parietal network show sustained activity over the duration of a task block (Dosenbach et al. 2006; Velanova et al. 2003; Yarkoni et al. 2005), supporting the maintenance and integration of items for goal-directed behavior.

    The observed pattern of findings supports the proposal that the fronto-parietal network provides a unified architecture for the integration and control of cognitive representations. According to this framework, processes for integration and control are critical for the optimal recruitment of internal resources to exhibit goal-directed behavior – supporting conceptual representations and executive processes that provide the basis for high-level cognition. We propose that mechanisms for integration and control are carried out by a central system that has extensive access to sensory and motor representations (cf., Miller & Cohen, 2001) and that the fronto-parietal network is at an ideal site in the brain to support these functions.

    One location, one network that includes most areas of the brain under control from a primary node.

    – – –

    Here is more:

    Abstract

    Two functionally distinct, and potentially competing, brain networks have been recently identified that can be broadly distinguished by their contrasting roles in attention to the external world versus internally directed mentation involving long-term memory. At the core of these two networks are the dorsal attention system and the hippocampal-cortical memory system, a component of the brain’s default network. Here spontaneous blood-oxygenation-level-dependent (BOLD) signal correlations were used in three separate functional magnetic resonance imaging data sets (n = 105) to define a third system, the frontoparietal control system, which is spatially interposed between these two previously defined systems. The frontoparietal control system includes many regions identified as supporting cognitive control and decision-making processes including lateral prefrontal cortex, anterior cingulate cortex, and inferior parietal lobule. Detailed analysis of frontal and parietal cortex, including use of high-resolution data, revealed clear evidence for contiguous but distinct regions: in general, the regions associated with the frontoparietal control system are situated between components of the dorsal attention and hippocampal-cortical memory systems. The frontoparietal control system is therefore anatomically positioned to integrate information from these two opposing brain systems.

    The second system—the hippocampal-cortical memory system—is part of a network of regions that are active during passive mental states linked to internally directed cognition including recollection of the past and thinking about the future (often labeled the default network) (Buckner et al. 2008; Raichle et al. 2001).1 This brain system includes regions in ventral medial prefrontal cortex, posterior inferior parietal lobule, retrosplenial cortex, posterior cingulate, and the lateral temporal lobe. Analysis of the hippocampal-cortical memory system using fMRI** measures of spontaneous activity reveals a tightly coupled set of regions (Buckner et al. 2008; Greicius et al. 2004; Kahn et al. 2008; Vincent et al. 2006) that overlap regions active during episodic memory retrieval (Maguire 2001; Svoboda et al. 2006; Vilberg and Rugg 2008; Vincent et al. 2006; Wagner et al. 2005).
    ** Wonder how much it cost. Maybe > $5m?

    – – –

    This one has CT and fMRI pictures!

    Frontoparietal cortical networks for directing attention and the eye to visual locations: Identical, independent, or overlapping neural systems
    Abstract

    Functional anatomical and single-unit recording studies indicate that a set of neural signals in parietal and frontal cortex mediates the covert allocation of attention to visual locations, as originally proposed by psychological studies. This frontoparietal network is the source of a location bias that interacts with extrastriate regions of the ventral visual system during object analysis to enhance visual processing. The frontoparietal network is not exclusively related to visual attention, but may coincide or overlap with regions involved in oculomotor processing. The relationship between attention and eye movement processes is discussed at the psychological, functional anatomical, and cellular level of analysis.

    One localized area of the brain, where our executive function is seated, that encompasses control – directly or indirectly – over the whole brain network.
    I figure that artists, athletes that excel , scientists, janitors(me!), lol, etc., are marked by high executive functioning and creativity, which uses executive tasks.

    There, anyone can write a paper that sounds good!

    What a fucking long comment

Comments are closed.