Jump to content

Issues in Interdisciplinarity 2020-21/Evidence in Intelligence Testing

Add links
From Wikibooks, open books for an open world

Introduction

[edit | edit source]

The means by which intelligence is defined and tested has significant implications for individuals livelihoods in modern society so the subject has been the source of controversy[1][2][3].

Intelligence is an abstract concept formulated from variable aspects, and different disciplines have conflicting views on what counts as evidence in measuring intelligence. Some even argue that intelligence testing should cease. However, it can be a valuable theoretical and practical tool in studying mental development and assessing cognitive abilities[4]. Resolving interdisciplinary conflicts through integration could develop more holistic approaches to intelligence testing which can be used to benefit individuals and society[5][6]. Tests need to be universal across machines, animals, humans, and therefore disciplines, and results standardised against humans.[7] Reaching a common understanding of evidence of intelligence means we'll have certainty of when a machine has achieved intelligence.

This Wikibook will look at evidence in intelligence testing in psychology, computer science and an anthropological/sociological perspective with the goal of highlighting conflicts.

Evidence in Intelligence Testing in Psychology

[edit | edit source]
Normal IQ distribution with mean 100 and standard deviation 15

In Psychometrics, the most common measure of intelligence is Intelligence Quotient (IQ). There are multiple modern IQ tests, the most common in the English-speaking world being the Wechsler Intelligence Scale, with different versions for adults and children. David Wechsler believed intelligence to be made up of interconnecting elements of cognitive ability that could be isolated and measured.[8] The current version of the test, WAIS-IV, is composed of indexes: verbal comprehension, perceptual reasoning, working memory, and processing speed.[9]

The IQ score is calculated based on standard deviation comparing individual performance against the average. These scores are normally distributed where average score is the most common (mean of 100 and standard deviation of 15 IQ points).[10] IQ scores are therefore estimates of intelligence rather than being a direct measure. IQ tests have been shown to have high statistical reliability with a confidence interval of approximately 10 points and a standard error of 3 points.[11] Although performance in IQ tests can vary on an individual basis due to external factors such as motivation and anxiety, casting doubt on validity, IQ scores correlate with performance in jobs and schools.[12][13] Some psychologists regard this as sufficient evidence that IQ tests are viable for practical use in education and jobs.[14]

Other psychologists have been critical of IQ tests with Wayne Weiten stating "IQ tests are valid measures of the kind of intelligence necessary to do well in academic work. But if the purpose is to assess intelligence in a broader sense, the validity of IQ tests is questionable." IQ tests can measure forms of intelligence, but broader aspects including creativity and emotional intelligence are unaccounted.[15] There are alternative tests for measuring aptitude in response to the criticisms. An example is the Mayer-Salovey-Caruso Emotional Intelligence Test.[16]

An Anthropological and Sociological Perspective

[edit | edit source]

Anthropological and sociological understandings focus on an individuals interactions with their wider culture and society. Since intelligence is a contested concept for which definitions vary both across and within cultures[17][18], those within anthropology argue that tests measuring intelligence must be emic – i.e., derived from within – the culture of the individual being tested[19]. Due to qualitative differences between cultures, some argue that quantitative comparisons made across cultures will be unhelpful [20]. Taking an anthropological perspective, Sternberg and Kaufman have argued that cultures "designate as “intelligent” the cognitive, social and behavioural attributes that they value as adaptive to the requirements of living in those cultures"[21]. Similarly, some sociologists argue that intelligence is a social construct, and its evidence is the product of a particular sociohistorical context concerned with issues surrounding social stratification and inequality[22].

In relation to measuring intelligence, those taking anthropological and sociological perspectives have highlighted the modifiability of intelligence[23], along with other external factors influencing performance in particular testing methods. Berry and Irvine have emphasised the need to appreciate the different levels of context that influence the performance of intelligence, including ecological and experimental contexts[24]. Their work has shown how cognitive styles develop in accordance with environmental demands[25]. The requirement is then for the experimental context (context in which intelligence is being tested) to align with the individuals own learning and everyday context. One example comes from Brazilian street children, who, despite being reliant upon their mathematics skills to run their own business to survive, performed poorly on solving the same mathematical problems when tested in school due to the abstract nature of the problems presented which were removed from their real-world context[26]. Conventional IQ and psychometric based tests, are thus considered problematic for their basis in assumptions of the acontextuality of intelligence and cognitive performance, especially when used to compare individuals from diverse cultural and economic backgrounds.

Evidence in Intelligence Testing in Computer Science

[edit | edit source]

Language as Evidence

The imitation game follows from Descartes’ ideas of language versatility in new and challenging situations being the first test of intelligence.[27] An interrogator, separated from a machine and human, asks questions. Through answering, the machine tries to trick the interrogator that it's the human, and the human tries to help the interrogator guess. If the machine is successful, it provides evidence that it's intelligent according to Turing.[28] The machine must mimic a human, pretending it's unable to perform complex equations and faking natural-looking human spelling mistakes. Evidence for intelligence here is adaptability, not the g-factor view of handling complexity.

The HAL project measures conversational ability as evidence of intelligence and an estimate of human maturity is assigned to the machine. The machine’s speech is examined for evidence looking at vocabulary size, response types and mean length of utterance.[29]

Cybernetics

Goal-focused systems (both animal and mechanical) where communication, or conversation, between system and environment is a prerequisite for activity.[30] The system’s response/activity as it interacts with the changing environment is taken as evidence of intelligence.[31] Evidence of intelligence is focused on activity and purposeful behaviour rather than reason.

Universal Intelligence Test

A machine receives rewards for interacting with an environment, it must learn the environment structure and what actions receive rewards to maximise amount received. Extra reward is given for applying Occam’s razor which is a intuitive, yet intelligent, method.[32]

IQ

It's rarely considered evidence of intelligence when a machine completes a classical IQ test, possible from 1963 when an AI program passed geometric analogy tasks from WAIS.

Often results are comparable to humans but, like in the case of number completion, the way machines solve problems can be different meaning results have different error distributions.

Related is psychometric AI, the field of building machines that can solve a range (using a single test is useless to evaluate a machine since it can be specialised to solve it) of established tests.[33]

Conclusion

[edit | edit source]

There is still much debate over what is considered evidence in intelligence testing. In machines, testing is focused on evidence for real-time flexibility, adaptability, and creativity; especially in terms of language and conversation. Like anthropological and sociological approaches, computer science tends to focus on interactions with the external environment. This goes against the universal, static, approach of IQ tests which, in its emphasis on evidence for intrinsic characteristics of intelligence measured in isolation, tends to ignore context. Engagement between these disciplines, alongside unexplored disciplines, could help move away from this narrow understanding of intelligence and towards a more practical understanding.

References

[edit | edit source]
  1. ↑ Phelps R. Measuring Intelligence: Facts and Fallacies. JDBP. 2006;27(4): 356
  2. ↑ Beeghley L, Butler EW. The Consequences of Intelligence Testing in the Public Schools before and after Desegregation. SP. 1974;21(5): 740-754.
  3. ↑ Bartholemew D. Measuring Intelligence: Facts and Fallacies. United States of America: Cambridge University Press; 2004.
  4. ↑ Chapter II: Applications of Intelligence Testing. RER. 1932;5(3):199.
  5. ↑ Benson ES. Intelligent intelligence testing. MoP. 2003;34(2): 48.
  6. ↑ Sternberg RJ, Grigorenko EL. Intelligence and culture: how culture shapes what intelligence means, and the implications for a science of well-being. PT: BS. 2004;359(1449).
  7. ↑ Hernandez-Orallo J, Martinez-Plumed F, Schmid U, Siebers M, Dowe DL. Computer models solving intelligence test problems: Progress and implications. AI. 2016;230: 74-107.
  8. ↑ Kaplan RM, Saccuzzo DP. Psychological Testing: Principles, Applications, & Issues. 8th ed. Belmont, CA: Wadsworth, Cengage learning; 2010.
  9. ↑ Pearson Clinical. Wechsler Adult Intelligence Scale® – Fourth UK Edition [Internet]. 2020. [Accessed 2 December 2020]; Available from: https://www.pearsonclinical.co.uk/Sitedownloads/Productpdfs/wais-4ukclinical8ppfinalweb.pdf
  10. ↑ Gottfredson LS. Chapter 1: Logical Fallacies Used to Dismiss the Evidence on Intelligence Testing. In: Phelps RF, editor. Correcting Fallacies about Educational and Psychological Testing. Washington, DC: American Psychological Association; 2009.
  11. ↑ Pearson Clinical. WISC-V Interpretive Considerations for Laurie Jones (6/1/2015) [Internet]. 2020. [Accessed 2 December 2020]; Available from: https://images.pearsonclinical.com/images/assets/wisc-v/WISC-VInterpretiveReportSample-1.pdf.
  12. ↑ Carlton SG, Gutierrez L. Psychological variables and Wechsler Adult Intelligence Scale-IV performance. AN: Adult. 2017;24(4): 357-363.
  13. ↑ Weiten W. Psychology: Themes and Variations. 10th ed. Belmont Calif.: Wadsworth Cengage Learning; 2016.
  14. ↑ Kaufman AS. IQ Testing 101. New York: Springer Publishing; 2009.
  15. ↑ Weiten W. Psychology: Themes and Variations. 10th ed. Belmont Calif.: Wadsworth Cengage Learning; 2016.
  16. ↑ Mayer, JD, Salovey P, Caruso DR, Sitarenios G. Measuring emotional intelligence with the MSCEIT V2.0. E. 2003;3: 97-105.
  17. ↑ Sternberg RJ. Culture and Intelligence. AP. 2004;59(5): 325–338.
  18. ↑ Sternberg RJ, Grigorenko EL. Intelligence and culture: how culture shapes what intelligence means, and the implications for a science of well-being. PT: BS. 2004;359(1449).
  19. ↑ Sternberg RJ. Metaphors of Mind: Conceptions of the Nature of Intelligence. Cambridge: Cambridge University Press; 1990.
  20. ↑ Sternberg RJ. Metaphors of Mind: Conceptions of the Nature of Intelligence. Cambridge: Cambridge University Press; 1990.
  21. ↑ Sternberg RJ, Kaufman JC. Human Abilities. ARP. 1998;49: 497.
  22. ↑ Squibb PG. The Concept of Intelligence – A Sociological Perspective. TSR. 1973;21(1): 57-75.
  23. ↑ Sternberg RJ, Kaufman JC. Human Abilities. ARP. 1998;49: 497.
  24. ↑ Sternberg RJ. Metaphors of Mind: Conceptions of the Nature of Intelligence. Cambridge: Cambridge University Press; 1990.
  25. ↑ Berry JW, Irvine SH. Human Abilities in Cultural Context. Cambridge University Press; 1988.
  26. ↑ Ceci SJ, Hembrooke HA. A bioecological model of intellectual development. APA. 1995:303-345
  27. ↑ Descartes R. Discourse on the Methods of Rightly Conducting One’s Reason and of Seeking Truth in the Sciences. Leiden: René Descartes; 1637.
  28. ↑ Turing, A., 1950, “Computing Machinery and Intelligence,” Mind, 59 (236): 433–60.
  29. ↑ Legg S, Hutter M. Tests of Machine Intelligence. 50 Years of Artificial Intelligence: Essays Dedicated to the 50th Anniversary of Artificial Intelligence. Berlin, Heidelberg: Springer-Verlag Berlin Heidelberg; 2007. p. 232-242.
  30. ↑ Pangaro P. Cybernetics – A Definition [Internet]. 2013. [Accessed 3 December 2020]; Available from: https://pangaro.com/definition-cybernetics.html
  31. ↑ Study with The Open University: OpenLearn. Machines, minds and computers: 3.2 Cybernetics and Symbolic AI [Internet]. 6th December 2017. [Accessed 3 December 2020]; Available from: https://www.open.edu/openlearn/science-maths-technology/computing-and-ict/computing/machines-minds-and-computers/content-section-3.2
  32. ↑ Legg S, Hutter M. Tests of Machine Intelligence. 50 Years of Artificial Intelligence: Essays Dedicated to the 50th Anniversary of Artificial Intelligence. Berlin, Heidelberg: Springer-Verlag Berlin Heidelberg; 2007. p. 232-242.
  33. ↑ Hernandez-Orallo J, Martinez-Plumed F, Schmid U, Siebers M, Dowe DL. Computer models solving intelligence test problems: Progress and implications. AI. 2016;230: 74-107.
Retrieved from "https://en.wikibooks.org/w/index.php?title=Issues_in_Interdisciplinarity_2020-21/Evidence_in_Intelligence_Testing&oldid=3793400"