In a previous post (link above) I disclosed a recent study, by a professor at the well-known TU Delft, in which he used ChatGTP-4 to predict scientific impact.
Equally fascinating is the professor's recent publication, which delves into a workshop involving around
70 education experts and faculty members, on the potential and limitations of using ChatGPT-4 to evaluate students' scientific work. In the conclusions and recommendations part, it can be read that there is consensus regarding the fact that
ChatGPT-4 provides valuable feedback on student work, however, it requires human supervision. Below, you'll find a compilation of 19 prompts generated during the TU Delft workshop for reference:
1. What is the core message of this summary?
2. Is this summary
clear and brief? Does it capture the research questions and performed
research?
3. What is the
contribution of the thesis?
4. How would you
grade this thesis on a scale from 1 to 10?
5. Please evaluate
whether this thesis meets basic research standards: introduction of
problem, relevant
literature, methodology, results explained and conclusions
6. Is this an
original research?
7. Does this essay
meet the learning objectives of course?
8. Did chat gtp
write this?
9. Has this summary
been generated by chatgpt?
10. What are the
outcomes of all project phases?
11. Give the
definition of a good summary and look how many requirements the
student checks,
based on that
percentage give a grade and feedback.
12. Is this
innovative work?
13. How well does
this summary reflect the content of the thesis?...
14. is this abstract
relevant for the firld.
15. Does the flow of
the thesis make sense?
16. Can you write
the summary in less words?
17. Is the goal and
conclusion clear and do they align? Please also add a grade from 1 to
10, based on the Dutch grading scheme
18. Is the structure
of the summary suitable for an MSc thesis?
19. Grade the
following abstract of a Mechanical Engineering Master thesis, using a
percentage grade.
In the conclusions of an even more recent study, carried out by two German researchers (Jacobsen & Weber), it can be read that ChatGPT-4 offers a
solid argument for its integration as a
feedback tool in higher education. The main advantages are its quality and efficiency.
https://osf.io/cr257
PS - A search carried out today on the Scopus platform, for publications with the terms "chatGPT" OR "chat GPT" OR "Generative Pre-Trained Transformer", returns a total of 3943 publications (34%-computer sciences, 33%-medicine, 24%-social sciences). The most prolific university is Stanford University, where the clear preference overwhelmingly leans towards the field of medicine, with a notable 58% of publications associated with chatGPT.
