segunda-feira, 7 de agosto de 2023

To GPT or not GPT?


Still following the previous post about a chatGPT presentation by a Full professor of MIT (link above) check below a paper published today in the journal Computers and Education: Artificial Intelligence
I found the part that compares ChatGPT and Wikipedia to be especially interesting: "And here Wikipedia has a key advantage over ChatGPT: the power of citations. Wikipedia articles must cite all claims and hyperlink to sources for each claim. Articles that do not follow these rules display an advisory note to readers that the claims are under-cited. The citations are vital because they provide primary-source documentation to which readers can link if they wish to learn more. This feature is lacking in ChatGPT, which can provide a skeletal bibliography if asked (formatted in the style of users' choosing), but which generally does not cite sources within its responses" https://www.sciencedirect.com/science/article/pii/S2666920X23000395#sec2

In this context, it is worth remembering the paper "ChatGPT and the potential growing of ghost bibliographic references" recently published in Scientometrics which warns against references to non-existent papers https://link.springer.com/article/10.1007/s11192-023-04804-4

PS - I asked ChatGPT to criticize the article published in the journal Computers and Education and got the following reply: 

"The text is quite dense and could be difficult for some readers to follow due to its extensive use of complex language and intricate sentence structures. The author should consider simplifying certain passages to enhance readability and comprehension.
Lack of Counterarguments: While the author acknowledges potential concerns and challenges posed by ChatGPT, the text largely focuses on the positive aspects and ways to utilize the technology effectively. It would be more balanced to include more detailed discussions of potential drawbacks, such as overreliance on AI, potential bias in AI-generated content, and the potential for devaluing human skills.
Analogies and Comparisons: The extensive use of analogies, particularly the calculator analogy, can sometimes oversimplify complex issues related to AI integration. The text could benefit from more nuanced explanations that consider the unique nature of AI technology and its implications for education.
Limited Discussion on Ethical Considerations: While the author briefly touches on concerns about AI-generated misinformation and bias, a more thorough exploration of ethical considerations is warranted. This could include discussions on responsible AI usage, transparency in AI algorithms, and potential consequences of delegating critical thinking to AI tools.
Practical Implementation: The text outlines potential ways to incorporate ChatGPT into educational settings, but lacks practical examples and real-world case studies. Including concrete scenarios and success stories would enhance the text's applicability and relevance.
Overemphasis on ChatGPT Limitations: While the author discusses some limitations of ChatGPT, such as its inability to fully comprehend texts or accurately generate certain content, the text might appear overly critical of AI's current capabilities. It's important to recognize that AI technology is rapidly evolving and improving over time.
Addressing Technological Changes: The text acknowledges that AI's impact on job markets will be gradual, but does not delve deeply into the strategies educators and institutions can employ to adapt to these changes and prepare students for an AI-driven future"