Building a chatbot to learn about and with AI

Students work collaboratively to build practical AI competencies through the iterative design and deployment of specialised Socratic chatbots for developing and testing knowledge on topics in an introductory Islamic Studies subject.

Snapshot

Educator: Dr Rachel Woodlock
Subject: ISLM10002 Islam in the Modern World
Discipline: Islamic Studies
Study Level: Undergraduate, Level One
Class Size: 30 students
Delivery Mode: Combination of on-campus lectures and tutorials

Design

Why?

This design aims to develop students’ AI literacy and awareness of academic integrity through hands-on, self-directed engagement with generative AI (genAI) supported by scaffolded project-based and inquiry-based learning (Stefanou et al. 2013; Şenyiğit et al. 2021). Students work either individually or in small groups to build a Socratic chatbot on a topic within the subject matter domain. Students are invited to brainstorm their topics through reflective prompts, such as ‘what do Muslims debate about today?’ and come up with a range of key questions to consider in an Islamic context, including dating, crypto and digital currencies, censorship and animal rights in farming, among others.

Once built, students use the chatbot as an educational tool to get peers to explore and learn about their topic. Purposeful engagement with genAI tools can develop self-regulated learning (SRL) in terms of students’ ‘forethought, performance, and reflection’ during the learning activity (Lan & Zhou, 2025). Providing support for students’ metacognition towards learning tasks involving genAI is seen as important for enhancing SRL in terms of active task strategy and evaluation.

How?

Preparation and resources

With the assistance of staff from the Faculty of Engineering and IT, a paid LLM API account and hosting were set up. The learning outcomes and assessments were redesigned to shift to an experiential learning approach with both individual and group work and incorporate opportunities for reflection. Instead of a take-home essay, the main assessment became a portfolio of work that accompanied class activities. Time over a five-week period that had previously been allotted to delivering content via lectures was given to guided activities with associated worksheets.

Activity Sequence

Week One: Choose a topic and establish a baseline
Explore the Socratic chatbot activity and application interface. Choose a topic and complete a brief pre-test of students' topic knowledge using three open-ended questions. Begin developing the chatbot prompt.
Week Two: Source literature for RAG
Learn about Retrieval-Augmented Generation (RAG). Source literature to upload for RAG and discuss copyright compliance. Continue refining the prompt.
Week Three: Evaluate AI responses critically
Discuss how to exercise evaluative judgement when engaging with generative AI, including identifying and dealing with hallucinations and bias. Continue refining the prompt.
Week Four: Finalise the prompt and reflect
Finalise the chatbot prompt while reflecting on the ethics and pedagogical value of engaging with AI.
Week Five: Deploy, assess and analyse
Deploy the chatbots with peers. Repeat the topic-knowledge test to assess learning gains. Complete a portfolio analysing how peers interacted with the chatbot.

Reflection

The multi-stage design gave students opportunities to reflect on their learning at various steps of the process. Most significantly, the portfolio assignment required students to assess their chatbots and their effectiveness as learning resources. This provided students with a rich opportunity for meta-learning.

Insights

Dr Rachel Woodlock had these observations to share:

Students came to the activity with varying levels of familiarity and comfort with using GenAI. Most students were excited to throw themselves into the activity, and appreciated the novelty of it. A key takeaway for me as an educator, has been to clearly communicate how the activity is not just about “learning AI” but is a way of facilitating conversation about AI use whilst applying a specific AI tool to learning the subject content.

Resources

If you are interested in learning more about the activity and its implementation, please see the resources linked below or contact Dr Rachel Woodlock for further details. Please consult AI at Melbourne and Using AI for student assessment and feedback before implementing the use of AI tools in the classroom.

Contribute

Tell us if you have tried this design or something similar.

Click here to share your design

References

Lan, M., & Zhou, X., (2025). A qualitative systematic review on AI empowered self-regulated learning in higher education. NPJ Science of Learning. 10, 21.

Xu, X., Qiao, L., Cheng, N., Liu, H., & Zhao, W. (2025). Enhancing self-regulated learning and learning experience in generative AI environments: The critical role of metacognitive support. British Journal of Educational Technology, 56, 1842–1863.