Since the release of ChatGPT by OpenAI and subsequently numerous other AI-based tools, we as scientists have been both fascinated by the possibilities but also see the risks associated with the use of these tools in research and teaching. In this context, Scholar AI is offered as a tool that helps us to summarise extensive literature on a subject area. Here is the result of an experiment I did this morning.
In the courses on my book “Collecting, Processing and Presenting Geoscientific Information” (Trauth, Sillmann, 2018) for both Bachelor’s and Master’s students, the participants have to find the five most relevant papers on their group topic in the first exercise. Before that, we try to find possible criteria for relevance. To do this, we first look at reviews (and the possibility of manipulation) of books on Amazon, where you can write book reviews without them being checked for their veracity; there is only a check to see if they are offensive. When using Google Scholar, we quickly find that relevant articles are cited very frequently, but Google Scholar does not provide a critical selection of cited and citing journals. This is the difference to the Web of Science, where only journals that carry out a stringent peer review process are taken into account.
Yesterday I used the Web of Science to search for literature on the topic and found more or less the expected papers that I already have in my literature collection, and a few papers that I had not seen before. Many of these papers are not open access, therefore I had to get them via my university account or ask the authors to send me the PDF file. According to the operators of the new Scholar AI by OpenAI, this is one of the weaknesses of many AI-based tools: many of them do not have access to literature behind a paywall. Scholar AI, however, seems to have access to Elsevier journals, while Elsevier recently launched its own Scopus AI.
When asking Scholar AI the question “Write a review on the impact of climate change on the termination of the old Egyptian empire”, a popular science article by two (former) Master’s students (Dufton & Branton, 2009) makes it into the synthesis, probably due to the very succinct and highly relevant title “Climate Change in Early Egypt”. This is by no means a bad article, on the contrary, but it is an article in a journal that is not in the list of journals according to the accepted criteria of the Web of Science.
When I look at the result, I also notice that it is a rather loose list of articles, with the first article being presented as “One of the key pieces” and the others as “Another important aspect …”, “Further research …” or “… is further evaborated …”. The individual aspect never cite more than one article, nor is any connection made between the aspects. The summary at the end is another list of the aspects mentioned above, but at least an “interplay” is recognised, even if I suspect that it wasn’t the AI’s conclusion; instead it was probably said in one of the sources used. This is of the biggest problems, and this is a weakness of an AI-based tool like Scholar AI, is that things are said without sources being given. An interesting observation, you get very similar formulations when you ask questions about other topics. The links to the literature are also often incorrect; below, the article by Welc & Marks (2014), correctly linked in the second paragraph, once again appears in the reference to Mueller-Wollermann (2014).
In addition to the use of information from sources that are not cited, you always have to reckon with the fact that more than just three coherent words are taken over without citation – and that is plagiarism according to our rules. This can of course be checked with the usual tools; the ones we also use to plag-check essays written by our students. But overall, if used creatively, I see a potential application of a Scholar AI tool in education: students could analyse an AI-generated answer, detect hallucinations and fill in missing sources to create a better text.
References
Dufton, D., Branton, T., 2009. Climate Change in Early Egypt. Egyptian Archeology, 36, 36–37.
Question
Write a review on the impact of climate change on the termination of the old Egyptian empire.
Answer api.scholarai.io ( (16 June 2024)
The termination of the Old Egyptian Empire, which marked the end of the Old Kingdom period around 4200 years ago, has been linked to various factors, including climate change. Recent geoarchaeological evidence suggests that climate change played a significant role in this historical transition.
One of the key pieces of evidence comes from a study by Welc and Marks (2014), which indicates that a climatic shift around 4200 BP led to significant environmental changes that affected the Nile’s flooding patterns. These changes resulted in reduced agricultural productivity, contributing to social and economic instability (Welc & Marks, 2014).
Another important aspect is the broader environmental context of Egypt during this period. Dufton and Branton (2009) discuss how the Predynastic and Early Dynastic periods were characterized by a ‘landscape conflict’ between the expanding desert and the fertile Nile valley. This environmental tension likely exacerbated the challenges faced by the Old Kingdom as it approached its end (Dufton & Branton, 2009).
Further research by Shawky El-Khalifa et al. (2022) highlights the long-term impact of increasing CO2 levels and rising temperatures on Egypt’s agricultural sector. These climatic factors would have reduced agricultural GDP and strained the resources of the Old Kingdom, making it more vulnerable to collapse (Shawky El-Khalifa et al., 2022).
Mueller-Wollermann (2014) points out that climate change, alongside internal struggles and potential invasions, contributed to the end of the Old Kingdom. This multi-faceted approach underscores the complexity of factors leading to the collapse (Mueller-Wollermann, 2014).
The impact of climate change on ancient Egyptian agriculture is further elaborated by Taha et al. (2022), who note that fluctuations in climate negatively affected the productivity of crucial cereal crops, leading to agricultural decline (Taha et al., 2022).
Moreover, the study by Kaniewski et al. (2013) suggests a broader regional impact of climate change, linking the environmental shifts to the Late Bronze Age crisis and subsequent invasions, which also had repercussions for Egypt (Kaniewski et al., 2013).
In summary, the termination of the Old Egyptian Empire can be attributed to a combination of climate change-induced environmental stress, agricultural decline, internal social struggles, and external pressures. This multifactorial perspective highlights the intricate interplay between natural and human factors in the collapse of one of history’s earliest centralized states.