I have had the chance to write and submit many, many research papers for peer-review in the past 20 years... more than 350 actually! Of course, not all of them could pass the reviewers' keen and thoughtful attention and many have been rejected. Common reasons for rejection used to be lack of data, poor quality of the data, or insufficient data analysis. All valid reasons, helping to improve the papers, the work, and my own research skills.
However, recently, I have had a paper rejected for a novel reason: not using AI/ML techniques. Although AI/ML techniques are useful in many domains to solve many problems, I claim that rejecting a paper because of *not* using AI/ML techniques is harmful for two reasons. First, it doesn’t help me improve the paper, the work, and my own research skills. Applying any black-box techniques, such as current AI/ML techniques, doesn’t improve our understanding of the problem and its possible solutions. Moreover, it doesn’t help me improve my research skills.
On the contrary, second, rejecting a paper because of *not* using these techniques shows two fundamental problems. One: the reviewer didn’t try to understand the problem, the data, and our solution and, because of the hype or sheer laziness, just recommended the “technique du jour”. Two: even if AI/ML techniques could have been appropriate, they would have produced a black box that does not further our understand of the problem and its solution.
Hype
In our submission, we collected and analysed manually a small but rich dataset of pieces of texts in free-form format. According to the reviewer, we should have used machine-learning techniques rather than “simple” (but effective and precise) manual analyses. At first, I rejected this nonsensical, baseless comment. Then, I asked myself: has this reviewer been blinded by the promises of AI/ML or has the scientific method changed in the current era of big data?
I claim that many people are currently blind to the true benefits and limitations of AI/ML techniques. Unfortunately, “The term "AI" became a real buzzword through the last decade, with companies of all shapes and sizes latching onto the term, often for marketing purposes.” [13], including in research where it is touted as the solution to every problem, not matter the problem. Yet, research is about improving our understanding of problems and their solutions while “AI is nowhere near delivering human-like understanding to machines” [1].
When doing research, I study a problem because I want, first, to understand that problem better and, second, propose a solution if possible or necessary. Consequently, AI/ML technique do not help research. For example, “deep learning models do not have any understanding of their input, at least not in any human sense” [1].
Indeed, “[f]rom the outset, there were two schools of thought regarding how understandable, or explainable, AI ought to be. Many thought it made the most sense to build machines that reasoned according to rules and logic, making their inner workings transparent to anyone who cared to examine some code. Others felt that intelligence would more easily emerge if machines took inspiration from biology, and learned by observing and experiencing” [11].
Current AI/ML techniques are inspired from biology and work “given large amounts of human-annotated data” [1]. So, when little data is available and when understanding the problem is the goal, they are not useful. “Current work in machine learning […] doesn’t provide explanatory models and theories that fit observations with high precision” [6].
“The exaggerated claims made in both papers, and the resulting hype surrounding these, are symptoms of a tendency among science journalists—and sometimes scientists themselves—to overstate the significance of new advances in AI and machine learning” [10].
“As always, when one sees large claims made for an AI system, the first question to ask is, “What does the system actually do?”” [10] and, more importantly in my opinion, what does it explain? And I’m not the only one concerned with the understanding (or lack thereof!) brought by AI/ML techniques: “the Department of Defense has identified explainability as a key stumbling block” [11]. I echo this statement: “Like many AI researchers, Nord is also concerned about the impenetrability of results produced by neural networks; often, a system delivers an answer without offering a clear picture of how that result was obtained” [2]. “And you can’t ask it: there is no obvious way to design such a system so that it could always explain why it did what it did” [11].
The Scientific Method
The fundamental problem with current AI/ML techniques and the hype surrounding them is that they give the false impression that any scientific problems can now be solved using these techniques. Yet, “Creativity is different from mechanical calculus, and it is also different from seeing things in a conventional way” [4].
Indeed, “[w]ith minimal human input, AI systems such as artificial neural networks — computer-simulated networks of neurons that mimic the function of brains — can plow through mountains of data, highlighting anomalies and detecting patterns that humans could never have spotted” [2], which led to “some scientists are arguing that the latest techniques in machine learning and AI represent a fundamentally new way of doing science” [2].
But, NO, generative modeling is not fundamentally different from Kepler studying Brahe’s data or from Microsoft Azure finding pattern in millions of images. Although some researchers claim that “generative modeling [is] novel enough to be considered a potential “third way” of learning about the universe.”, it does nothing more than identify patterns in large amount of data. It does not understand or explain the data! “The next step, Schawinski says, has not yet been automated: “I have to come in as a human, and say, ‘OK, what kind of physics could explain this effect?’”” Fortunately, Hogg says “[b]ut in my view, my work is still squarely in the observational mode.” and Polsterer that “the algorithms can only do what they’ve been trained to do.”, which save partly the article. (As a side note, the article [2] reports so many unfounded, grand claims by Schawinski that it made me cringe!)
Again, NO, the scientific method cannot be replaced by large amount of data and some AI/ML techniques. “Hype surrounding AI has peaked and troughed over the years as the abilities of the technology get overestimated and then re-evaluated” [13]. The main reason being, of course, that “[p]resent a machine with data that varies even slightly from the training data with which they were taught, and they cannot compute” [1]. The most famous example being the case in which “[the algorithm] hasn’t realized that “sheep” means the actual animal, not just a sort of treeless grassiness” [14] and that “if goats climb trees, they become birds. Or possibly giraffes. (It turns out that Microsoft Azure is somewhat notorious for seeing giraffes everywhere due to a rumored overabundance of giraffes in the original dataset)” [14].
Thus, the sadness of a reviewer rejecting a paper because I didn’t use AI/ML techniques is “a philosophical split between science fundamentalists and pragmatists, with the latter being willing to accept correlation and adequately accurate predictive power, saying “Who cares about the theory? This works!”” [6]. But, I care! “In the past few years, our tolerance of sloppy thinking has led us to repeat many mistakes over and over. If we are to retain any credibility, this has to stop” [9]. Besides, “preconceived notions influence, of course, the way the discovered patterns are interpreted too. Thus, when supporters of a purely data-driven approach claim that “numbers speak for themselves,” or that they are not a priori committed to any theoretical view, they are not doing science, but rather metaphysics” [4]. (Satell [5] misses entirely the point of the difference between discovering patterns and explaining them in an article that reads like an “infomercial” for Boeing (!)…)
It is true that “[w]hile the hypothesis-based scientific method has been very successful, its exclusive reliance on deductive reasoning is dangerous because according to the so-called Duhem–Quine thesis, hypothesis testing always involves an unknown number of explicit or implicit assumptions, some of which may steer the researcher away from hypotheses that seem implausible, although they are, in fact, true [21]. According to Kuhn, this bias can obstruct the recognition of paradigm shifts [22]” [3]. Yet, “[t]he traditional scientific method is often presented in discrete steps, but it should really be seen as a form of critical thinking, subject to review and independent validation [8].” Thus, by definition, the scientific method requires transparency for review and external validation; transparency which is lacking currently with most AI/ML techniques.
Moreover, Succi and Coveney [8] “contrasted the assumptions underlying these theories [of machine learning], such as the law of large numbers, with the mathematical reality of complex biological systems. Specifically, they carefully identified genuine features of these systems, such as nonlinearities, nonlocality of effects, fractal aspects, and high dimensionality, and argued that they fundamentally violate some of the statistical assumptions implicitly underlying big-data analysis, like independence of events.”
Besides, as pointed out by Leonelli, “Big Data that is made available through databases for future analysis turns out to represent highly selected phenomena, materials and contributions, to the exclusion of the majority of biological work. What is worse, this selection is not the result of scientific choices, which can therefore be taken into account when analysing the data. Rather, it is the serendipitous result of social, political, economic and technical factors, which determines which data get to travel in ways that are non-transparent and hard to reconstruct by biologists at the receiving end” [4].
“[I]n most cases, understanding the why is crucial for reaching a level of knowledge that can be used with confidence for practical applications and for making reliable predictions” [4] or, as my dad used to as, “why don’t rabbits wear glasses?”.
Conclusion
“The widespread misimpression that data + neural networks is a universal formula has real consequences: in what scientists choose to research, in what companies and governments choose to fund, in what journals and conferences choose to accept for publication, in what students choose to study, and in what universities choose to teach. The reality is that neural networks, in anything like their current form, cannot replace the sophisticated tools of scientific analysis, developed over centuries, and have not thus far duplicated the great scientific accomplishments of the past—let alone improved upon them. They should be seen as tools that supplement existing techniques, not foundational revisions to how we do science” [10].
When Anderson writes that “Google conquered the advertising world with nothing more than applied mathematics. It didn't pretend to know anything about the culture and conventions of advertising — it just assumed that better data, with better analytical tools, would win the day.” [7], he is so wrong on so many levels. He confuses hypotheses with models when he writes that “The scientific method is built around testable hypotheses. These models, for the most part, are systems visualized in the minds of scientists.” [7]. He also seems to misunderstand the objectives and roles models: to simplify and explain some phenomena. His use of the word “caricature” when talking about models confirms this confusion: a model is a model and, by definition, does not (and should not) try to explain everything! It is not a caricature but an explanatory tool. And, no, please!, correlation is NOT enough, unless you agree to be the black swan in a world of white swans...
The article [2] concludes that “the struggle to describe what goes on inside the “mind” of a machine is mirrored by the difficulty we have in probing our own thought processes.”, which is quite the strawman! I may not know what’s going on in my mind or my colleagues’, but I sure want to know, test, and audit how a computer program is reaching the decision to change lane or not while driving a car…
Life evolved over (at least) 3.5 billion years… The important word, here, being “evolved”. It seems to me foolish to believe that we could in turn evolve an artificial (general) intelligence in only a few years!
“At the start of the 2010s, one of the world leaders in AI, DeepMind, often referred to something called AGI, or "artificial general intelligence" being developed at some point in the future. […] But those conversations were taken less and less seriously as the decade went on. At the end of 2019, the smartest computers could still only excel at a "narrow" selection of tasks.” [13]
“There’s already an argument that being able to interrogate an AI system about how it reached its conclusions is a fundamental legal right. Starting in the summer of 2018, the European Union may require that companies be able to give users an explanation for decisions that automated systems reach.” [11]
““We have machines that learn in a very narrow way,” Bengio said. “They need much more data to learn a task than human examples of intelligence, and they still make stupid mistakes.”” [12]
“Facebook’s director of AI said recently that his company and others should not expect to keep making progress in AI just by making bigger deep learning systems with more computing power and data. “At some point we're going to hit the wall,” he said. “In many ways we already have.”” [12]
“Clune was due to present Friday on the idea of making smarter AI by turning the technology in on itself. He’s part of an emerging field called metalearning concerned with crafting learning algorithms that can devise their own learning algorithms.” [12] and meta-metalearners… and…
“As thousands of AI experts shuffled away from Bengio’s packed talk Wednesday, Irina Rish, an associate professor at the University of Montreal also affiliated with Mila, was hopeful his words would help create space and support for new ideas at a conference that has become dominated by the success of deep learning. “Deep learning is great but we need a toolbox of different algorithms,” she says.” [12]
(Another interesting concept: that of “allochthonous” knowledge acquisition [3]… future blog!)
(Linus Torvald validated my claim of AI as a hype [15] 😊)
Sources
[1] Why Deep Learning Won't Replace Its Human Counterparts Anytime Soon (https://www.techrepublic.com/article/why-deep-learning-wont-replace-its-human-counterparts-anytime-soon/)
[2] How Artificial Intelligence Is Changing Science (https://www.quantamagazine.org/how-artificial-intelligence-is-changing-science-20190311/#)
[3] Perspective: Dimensions of the Scientific Method (https://journals.plos.org/ploscompbiol/article/file?id=10.1371/journal.pcbi.1007279&type=printable)
[4] Could Big Data Be the End of Theory in Science? (https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4766450/pdf/EMBR-16-1250.pdf)
[5] Is It Time to Rethink the Scientific Method? (https://www.inc.com/greg-satell/heres-how-big-data-and-machine-learning-will-help-.html)
[6] Will data science eventually replace the scientific method? (https://www.quora.com/Will-data-science-eventually-replace-the-scientific-method)
[7] The End of Theory: The Data Deluge Makes the Scientific Method Obsolete (https://www.wired.com/2008/06/pb-theory/)
[8] Big Data: The End of the Scientific Method? (https://royalsocietypublishing.org/doi/pdf/10.1098/rsta.2018.0145)
[9] Artificial Intelligence Meets Natural Stupidity (http://www.cs.yorku.ca/~jarek/courses/ai/F11/naturalstupidity.pdf)
[10] Are Neural Networks About to Reinvent Physics? (http://nautil.us/issue/78/atmospheres/are-neural-networks-about-to-reinvent-physics)
[11] The Dark Secret at the Heart of AI (https://www.technologyreview.com/s/604087/the-dark-secret-at-the-heart-of-ai/)
[12] A sobering message about the future at AI’s biggest party (https://arstechnica.com/information-technology/2019/12/a-sobering-message-about-the-future-at-ais-biggest-party/)
[13] Researchers: Are we on the Cusp of an ‘AI Winter’? (https://www.bbc.com/news/technology-51064369)
[14] This Neural Net Hallucinates Sheep (http://nautil.us/blog/this-neural-net-hallucinates-sheep)
[15] Linus Torvalds slams AI as ‘90% marketing and 10% reality’ (https://www.techradar.com/pro/linus-torvalds-slams-ai-as-90-percent-marketing-and-10-percent-reality)
