I will always promote Jose Hernandez-Orallo's The Measure of All Minds [1]
It attempts to codify how we should go about measuring and evaluating the somewhat fuzzy concept of "intelligence." He proposes an extension of his "Anytime Intelligence Test" which could be used to test animal and machine intelligence on a level playing field.
Measurement of task capability against a baseline is the most overlooked problem in AI and as far as I am aware Hernandez-Orallo is the only one focusing on it.
Notice that all of the major "breakthrough" moments in AI over the last half century had a human baseline that an AI was competing against. Those baselines were ones that had been already developed over years (sometimes a century) and were part of competitive games already. Go, Chess, DOTA etc... had leaderboards or international rankings.
For fuzzier things like driving, translation, strategy, trading etc... there is no generally accepted and measurable baseline test for what is considered human level, only proxies and unit specific tests. So we continue to not know when an AI system is measurably at or exceeding human level. Without this we can't definitively know how much progress we're making on Human Level Intelligence.
It attempts to codify how we should go about measuring and evaluating the somewhat fuzzy concept of "intelligence." He proposes an extension of his "Anytime Intelligence Test" which could be used to test animal and machine intelligence on a level playing field.
Measurement of task capability against a baseline is the most overlooked problem in AI and as far as I am aware Hernandez-Orallo is the only one focusing on it.
Notice that all of the major "breakthrough" moments in AI over the last half century had a human baseline that an AI was competing against. Those baselines were ones that had been already developed over years (sometimes a century) and were part of competitive games already. Go, Chess, DOTA etc... had leaderboards or international rankings.
For fuzzier things like driving, translation, strategy, trading etc... there is no generally accepted and measurable baseline test for what is considered human level, only proxies and unit specific tests. So we continue to not know when an AI system is measurably at or exceeding human level. Without this we can't definitively know how much progress we're making on Human Level Intelligence.
[1]https://www.cambridge.org/core/books/measure-of-all-minds/DC...