Developing and Assessing University Level Exams with Artificial Intelligence (AI) in Selected Universities of Baluchistan

By

Syed Muhammad Ali Shah

Syed Muhammad Kamil

Nadia

Hasnain Mehdi

Wajeeha khan

Muzammil

Supervised by

Ms. Shahtaj Akram

Table of contents

Chapter 1............................................................4

1.1 Introduction............................................................4

1.2 Background...............................................................6

1.3 Statement of problem.........................................................8

1.4 Research Objectives:.........................................................8

1.5 Research Questions:.........................................................8

1.6 Research Significance:...................................................8

1.7 Limitations:.................................................................9

Chapter 2: Review of the literature:................................................9

2.1 Introduction:.................................................................9

2.2 Integration of AI into traditional methods of learning:................................11

2.3 Various AI tools and ChatGPT:.........................................................11

2.4 AI aims to bridge gaps through innovative techniques:................................12

2.5 Machine Learning for Students Performance Prediction:................................12

2.6 Use of AI in pedagogy:...........................................................13

2.7 Artificial intelligence for student assessment:........................................14

2.8 Conclusion:....................................................................14

Chapter 3 Research Methodology:....................................................15

3.1 Introduction:.................................................................15

3.2 Research Design:............................................................16

3.3 Participants:.................................................................16

3.4 Participant Criteria:........................................................16

3.5 Sample Size:.................................................................17

3.6 Data Collection Methods:..................................................17

3.7 Data Analysis:...............................................................18

3.8 Research Ethics:...........................................................19

3.9 Conclusion:.................................................................20

References.................................................................21

Page | 2

Chapter 1

1.1 Introduction

The advent of Artificial Intelligence (AI) has heralded a transformative era (Kherdekar, n.d.) in the field of education (Gocon & Aydemir, 2020a; Kherdekar, n.d.; Neha, n.d.-b; N. D. Nguyen, 2023), offering unprecedented opportunities to enhance pedagogical methodologies

(Al-Matari, n.d.-b; Georgieva-Hristozova, 2023a; Matthew et al., 2024a; T. Nguyen & Nguyen, 2023; Ugochukwu Okwdili Matthew et al., 2023a; Vazquez-Cano, 2021) and assessment

practices (International Association for Development of the Information Society., 2014). This

research endeavors to delve into the realm of academia, specifically within the context of

Baluchistan's higher education landscape (Gao, 2022a; Pisca et al., 2023a; Xia & Li, 2022a), where the implementation of AI in the development and evaluation of university-level examinations remains a subject of nascent exploration. As we embark on this intellectual

journey, it is essential to recognize the dynamic potential of AI technologies, including

language models like ChatGPT, (Introducing ChatGPT, n.d.-b) in shaping the discourse on

educational innovation.

With an acute awareness of the imperative role education plays in fostering intellectual

growth and societal progress (Schwartzman et al., 2017), this study seeks to introduce a novel

paradigm in examination practices through the infusion of advanced AI technologies. ChatGPT,

(Introducing ChatGPT, n.d.-b) as a representative example of AI language models, serves as

an intellectual collaborator, contributing to the discourse surrounding the synthesis of cutting-edge technologies and traditional academic rigor in the development of university-level exams (Joshi et al., 2021a).

Baluchistan, a region endowed with rich cultural diversity and historical significance,

grapples with unique challenges in its pursuit of academic excellence (Khan et al., 2023a).

Page | 3

Traditional examination systems (Rind & Malik, 2019a), albeit indispensable, may be

insufficient in capturing the dynamic intricacies of contemporary education (Tatlah et al., n.d.).

According to (Haniya et al., 2020) recognizing the pressing need for innovation in assessment methodologies (Owan et al., 2023), this research aspires to propose a groundbreaking approach that harnesses the capabilities of AI, including natural language processing algorithms employed by ChatGPT (Introducing ChatGPT, n.d.-a), to create and appraise university-level exams. In doing so, it aims to address the multifaceted dimensions of higher education (Rind & Malik, 2019b; Tatlah et al., n.d.) in Baluchistan, aligning them with global standards while considering the region's distinctive cultural and socio-economic milieu.

The focal point of this investigation lies in the synthesis of cutting-edge AI technologies, such as those employed by ChatGPT (Haque, 2023), with the traditional academic rigor that defines university-level examinations.

One notable application is the creation of comprehensive question papers, facilitated by the linguistic and contextual understanding capabilities of AI.

(Ungerer & Slade, 2022; Zaman, 2023a) by scrutinizing the potential advantages, challenges, and ethical considerations associated with the integration of AI in this context, this research endeavours to provide a comprehensive understanding of the implications and feasibility of such an innovative approach.

Through meticulous examination, critical analysis, and empirical study, the aim is to elucidate the transformative impact that AI-driven examination systems (Luckin, 2017a), with contributions from models like ChatGPT (Introducing ChatGPT, n.d.-a), can exert on the educational landscape of Baluchistan, fostering an environment conducive to academic excellence and intellectual progression.

As we engage in this exploration, the collaboration with AI models like ChatGPT symbolises the intersection of human intellect and machine intelligence (Haque, 2023), ushering in a new era of academic inquiry and innovation.

The ensuing chapters of this thesis will unfold the intricacies of AI-mediated examination development and evaluation,

Page | 4

scrutinizing its potential to revolutionize the educational paradigm in Baluchistan (Gao, 2022b;

Georgieva-Hristozova, 2023b; Matthew et al., 2024b; Xia & Li, 2022b; Zaman, 2023a), thereby

contributing to the broader discourse on the intersection of technology and higher education.

1.2 Background

The higher education sector in Baluchistan stands at a crossroads (Khan et al., 2023b),

facing a confluence of challenges and opportunities that necessitate a reevaluation of

conventional pedagogical practices (Tatlah et al., n.d.). Historically characterized by a

commitment to academic excellence and the cultivation of intellectual acumen, the region's

universities now confront the imperative to adapt to the demands of a rapidly evolving global

educational landscape (Joshi et al., 2021b). Baluchistan's unique socio-economic fabric,

characterized by diverse linguistic and cultural nuances, adds a layer of complexity to the

pursuit of educational advancement.

In this context, researchers (Rubab & Imran, 2023) states the conventional examination

systems employed in the region's universities exhibit inherent limitations in gauging the

multifaceted dimensions of student knowledge and skills (Juma, 2021a). The traditional

summative assessment methods (French et al., 2023), while venerable in their legacy, may

not be fully aligned with the contemporary imperatives of fostering critical thinking, problem-

solving abilities, and adaptability – skills paramount for success in the 21st-century knowledge

economy (Juma, 2021a). The advent of Artificial Intelligence (Pisca et al., 2023b), with ChatGPT exemplifying

the capabilities of language models (Haque, 2023; Herrmann-Werner et al., 2024; Introducing

ChatGPT, n.d.-a), presents an unprecedented opportunity to reimagine and revolutionize the

assessment landscape, transcending the constraints of traditional examination paradigms

(Rubab & Imran, 2023). As global academia increasingly acknowledges the potential of AI in

Page | 5

enhancing the efficiency and efficacy of educational processes (Al-Matari, n.d.-a; Gocen & Aydemir, 2020b; Ugochukwu Okwudili Matthew et al., 2023b; Xia & Li, 2022b), the exploration of its application in examination development and evaluation emerges as a critical avenue for investigation.

Baluchistan's higher education institutions, with their commitment to academic advancement (Khan et al., 2023b), stand to benefit from embracing innovative approaches that harness the power of AI. The integration of machine learning algorithms, natural language processing (Francisci, 2023a), and data analytics into the fabric of examination systems holds the promise of not only providing a more nuanced understanding of student performance but also of streamlining the assessment process (Herrmann-Werner et al., 2023, 2024), thereby enhancing the overall quality of education.

However, this transition to AI-mediated examination systems necessitates careful consideration of contextual factors, ethical implications (Ungerer & Slade, 2022), and the imperative to strike a harmonious balance between tradition and innovation. This research,

with insights drawn from collaborative efforts with AI models like ChatGPT (Introducing ChatGPT, n.d.-a), seeks to bridge the gap between theoretical discourse and practical implementation. It offers a comprehensive exploration of the viability and impact of AI-driven examinations in the specific context of Baluchistan's higher education milieu, including the creation of question papers.

By situating this investigation within the broader framework of global trends in educational technology and drawing on the rich tapestry of Baluchistan's academic heritage, this study aspires to contribute to the ongoing dialogue on the integration of AI in higher education (Gocen & Aydemir, 2020b; Pisica et al., 2023b), offering insights that are both contextually relevant and globally resonant. In doing so, it aims to chart a course toward a

Page | 6

future where Baluchistan's universities, in collaboration with advanced AI technologies (Neha, n.d.-a), stand as beacons of innovation, equipping their students with the skills and knowledge requisite for success in an ever-evolving world (Juma, 2021a).

1.3 Statement of problem

This scholarly inquiry delves into the intricacies of localized AI integration, specifically leveraging (Franciscu, 2023b) ChatGPT,(Introducing ChatGPT, n.d.-b) within the esteemed confines of higher educational institutions in Baluchistan(Juma, 2021b; Khan et al., 2023a). The crux of this investigation lies in furnishing educators with an avant-garde tool(Zaman, 2023b), empowering them to meticulously curate question papers. The overarching objective is to endow pedagogues with an instrumentality for the cultivation of critical thinking and creativity among students. The fundamental query that underscores this research is: (Luckin, 2017b)In what manner can this bespoke AI solution augment the efficacy of educators in the realm of question paper development and assessment?

1.4 Research Objectives:

• To Assess the Efficacy of ChatGPT in Question Paper Development for the university level exams.

1.5 Research Questions:

• How effective is ChatGPT in generating question papers for the university level exams?

1.6 Research Significance:

1. Advancement of Educational Technology

2. Empowering Educators

Contributes to the growing body of knowledge on the integration of advanced AI technologies in education, particularly in question paper development and assessment?

Page | 7

Provides educators in Baluchistan with a cutting-edge tool to enhance their question

paper development, fostering a more comprehensive evaluation of students' cognitive abilities.

3. Cultivation of Critical Thinking

Addresses the imperative need to cultivate critical thinking and creativity among

students, aligning with global educational objectives.

4. Ethical Guidelines in AI Integration

Establishes ethical considerations and guidelines for the integration of AI in educational

practices, ensuring fairness, transparency, and unbiased assessment.

5. Global Relevance and Local Context

Offers insights that bridge the gap between global trends in educational technology and

the specific cultural and socio-economic context of Baluchistan, contributing to a nuanced

understanding of AI's impact on education.

1.7 Limitations:

- The study is focused on selected higher education institutions of Baluchistan may limit

generalizability to other contexts.

- Availability of educators and examination committee members for interviews.

- Reliance on ChatGPT's performance in generating exam questions, which may vary.

Chapter 2: Review of the literature:

2.1 Introduction:

The literature review provides a comprehensive overview of the integration of

artificial intelligence (AI) in education, highlighting its potential to revolutionize teaching and

learning processes. Various studies explore the evolution of AI in education, its applications in

in

student assessment, and the challenges and opportunities associated with its implementation.

From enhancing educational assessment through AI-powered tools to predicting student

performance using machine learning algorithms, the review delves into the multifaceted ways

AI is reshaping the educational landscape. Additionally, it discusses the ethical considerations

raised by the rapid advancement of AI technology and examines students' perceptions of AI in

education. By synthesizing findings from diverse research endeavors, the review offers

valuable insights into the current state and future directions of AI integration in education.

Potential of artificial intelligence tools in education

Artificial intelligence (AI) is revolutionizing industries, including education. To

improve teaching and learning experiences, educators and professionals working in educational

evaluation must keep up with the rapid innovations in AI technology (Owan et al.2023).

According to Owan et al (2023), AI-powered educational assessment technologies have many

advantages, such as increasing test efficiency and accuracy, generating student-specific

feedback, and letting teachers modify their lesson plans to suit each student's individual needs.

As a result, AI has the power to completely change how education is given, which will

eventually improve student learning results (Owan et al.2023). The different applications of

artificial intelligence for measuring and evaluating education are examined in this research (Owan et al. 2023). It focuses on the integration of large language AI models in classroom

assessment, covering topics such as test purpose determination and specification, developing,

test blueprint, test item generation/development, test instruction preparation, item

assembly/selection, test administration, test scoring, test result interpretation, test

analysis/appraisal, and reporting (Owan et al. 2023). It examines the difficulties in utilizing AI-

powered tools for educational evaluation as well as the role that teachers play in AI-based

assessment. Finally, the paper discusses solutions for addressing these problems and increasing

the usefulness of AI in educational assessment. To summarize, utilizing artificial intelligence

in educational assessments has both advantages and disadvantages (Owan et al. 2023).

Therefore, it is imperative that educators, legislators, and other relevant parties collaborate on

creating plans that enhance the advantages of artificial intelligence in educational evaluation

while reducing the risks involved (Owan et al. 2023). In the end, adding AI into assessment in

education has the potential to revolutionize the field, enhance student learning, and provide

students the tools they need to thrive in the twenty-first century (Owan et al. 2023).

2.2 Integration of AI into traditional methods of learning:

Aasen (2012), states that many studies on the use of AI in education did not talk much

about the teaching methods they are using but, most of the time, AI is used for checking how

well students are doing continuously, and a key thing it does is automatically grade students.

Some studies compare how good AI is compared to traditional methods and suggests that

teachers need more training on how to use AI. (Aasen, 2012), also points out that we still need

more research to figure out the best ways to use AI in assessing students, especially in schools

and other education levels. The researchers stress that it's not just about the technical side of

AI but also understanding how it affects education.

Artificial intelligence, a recent technology changing how we live, is not well-known

among many university professors. In this situation, it's important to bring in information

technology and improve communication in classroom (Kalogiannakis, 2020).

2.3 Various AI tools and ChatGPT:

Students have been using artificial intelligence tools for their studies, like learning

management systems, online discussion boards, exam integrity systems, lecture transcriptions,

etc (Kalogiannakis, 2020). A recent addition to these tools is ChatGPT, which understands

regular language and gives responses that sound like a person talking, acting as a helpful tool

for conversations, it supports students in their learning by providing help and introducing

Page | 10

challenges. This allows students to get assistance without spending too much time on

assignments, while also developing important skills for their future jobs (Kalogianakis, 2020).

2.4 AI aims to bridge gaps through innovative techniques:

Artificial Intelligence (AI) endeavors to equip computers with human-like intelligence, enabling them to think and respond similarly to humans (Lesinski et al., 2016). Unlike humans who learn from experience, computers rely on algorithms to perform tasks, but AI aims to bridge this gap by developing innovative techniques to imbue computers with intelligence (Došilović et al., 2018). AI applications are expanding across various industries, with future advancements expected to enable more natural interactions between humans and machines (Nilsson, 2014).

2.5 Machine Learning for Students Performance Prediction:

Machine Learning, a subset of AI, allows systems to learn and improve from experience without explicit programming (Mitchell et al., 2013). The primary goal is to enable computers to learn automatically and make decisions for future tasks (Nilsson, 2014). Machine Learning algorithms, such as supervised and unsupervised learning, facilitate this process.

Supervised learning algorithms, including Artificial Neural Networks (ANN), Naive Bayes, k-Nearest Neighbours (k-NN), Support Vector Machines, and Decision Trees, learn from labeled data to classify or predict outcomes (Qazdar et al., 2019). On the other hand, unsupervised learning algorithms, like clustering, discover patterns in unlabelled data (Kassambara, 2017).

In educational environments, Machine Learning classification models are employed to predict student performance based on academic features (Khan et al., 2021). These models assist in identifying students at risk of unsatisfactory outcomes, allowing for timely

Page | 11

intervention (Kausar et al, 2020). Various algorithms, including Random Forest, Stacking

Classifiers, Genetic Algorithm (GA), and Decision Trees, have been used to develop predictive

models (Orong et al., 2020; Chen et al., 2018; Saa, 2016).

2.6 Use of AI in pedagogy:

With the rise of new technology, like artificial intelligence (AI), educators, researchers,

and administrators are paying attention to its potential in education (Chen et al., 2020). The

Horizon 2018 report notes that AI and adaptive learning are becoming important for tailoring

educational materials to individual needs (Educause, 2018). AI is being used in various ways,

from grading and predicting student performance to offering personalized learning experiences

and creating intelligent tutoring systems (Ahmad et al, 2020). Recent reviews show many

studies are looking into how AI can be used for teaching and learning (Zawacki-Richter et al.,

2019; Zhai et al., 2021).

Some studies have investigated how students feel about AI in education, especially in

fields like medicine (Santomartino & Yi, 2022; Pinto dos Santos et al., 2019). Others focus on

students in K–12 and higher education. For example, a study in China found that primary

school students' motivation to learn AI depended on their confidence, comfort, and belief in its

positive impact (Chai et al., 2021). Another study surveyed university students in Germany,

China, and the UK about their attitudes towards AI, covering aspects like trust, societal impact,

and job prospects (Sindermann et al., 2021). Despite this research, there's still a gap in

understanding how students perceive AI in education.

The use of AI-based systems in education is gaining attention among researchers due

to the rapid advancement of technology and ongoing discussions about the ethical

considerations surrounding AI usage (Smith, 2020). Over the past decade, there has been a rise

in initiatives exploring various applications of AI to support teaching and learning.

2.7 Artificial intelligence for student assessment:

Artificial intelligence (AI) is being used in more and more industries, including education. AI is mostly used in education for tutoring and assessment purposes (Gonzalez et al. 2021). Using data from a systematic review, this paper examines the application of AI to student assessment. Two databases were searched for this purpose: Web of Science and Scopus.

After 454 papers were located, 22 papers were chosen for further analysis in accordance with the PRISMA Statement. The studies that have been examined make it evident that the majority of them do not reflect the pedagogy that underpins the educational action (Paz et al. 2021).

Similar to this, formative assessment appears to be AI's primary use. Another important feature of AI in assessment is the ability to grade students automatically (Rosabel et al. 2021).

Numerous research examines the distinctions between using AI and not using it. We go over the findings and draw the conclusion that, in order to fully grasp the potential of artificial intelligence (AI) in educational evaluation, particularly at lower education levels, more research and teacher preparation are necessary. Furthermore, there has to be a greater abundance of study on educational elements of AI rather than just technical advancements, (gonzález et al. 2021).

2.8 Conclusion:

The use of artificial intelligence in education has raised many debates and challenges.

According to Owan et al (2023), AI-powered educational assessment technologies have many advantages, such as increasing test efficiency and accuracy, generating student-specific feedback, and letting teachers modify their lesson plans to suit each student's individual needs.

Artificial intelligence (AI) are technical systems that can do things like learning, adapting, correcting themselves, and using data for complex tasks, like humans (Popenici & Kerr, 2017).

Page | 13

There is a growing interest in using AI for student assessment, which is a crucial aspect

of education (Smith & Johnson, 2019). This includes automating the grading process for exams

and other student assignments to reduce teachers' workload and ensure fairness in grading, for

instance, AI-based tools can automatically correct multiple-choice exams and analyse other

types of student work (Adams et al., 2020).

A recent addition to these tools is ChatGPT, which understands regular language and

gives responses that sound like a person talking, acting as a helpful tool for conversations, it

supports students in their learning by providing help and introducing challenges

(Kalogiannakis, 2020).

AI is mostly used in education for tutoring and assessment purposes (Gonzalez et al. 2021). formative assessment appears to be AI's primary use and another important feature of AI in assessment is the ability to grade students automatically (Rosabel et al. 2021). The research study indicates how artificial intelligence might shape the future of higher education and discusses the current education system, and the issues faced by both teachers and students on changes in administration rules and regulation concerning the growing trend of AI tools in education (Owan et al. 2023). Thematic analysis method would be utilized

Page | 14

to uncover patterns and themes within the data, (Braun & Clarke, 2006) allowing for a

comprehensive exploration of the research questions and objectives.

3.2 Research Design:

The research design for this study will be intended to be qualitative, (Bailey, 2014) as

it aims to delve into the perspectives, experiences, and practices of educators regarding the

integration of ChatGPT(Introducing ChatGPT, n.d.-b) in question paper development.

Qualitative research is chosen for its ability to provide rich, detailed insights into the

phenomena under investigation, aligning with the exploratory nature of this study.

3.3 Participants:

Sampling Strategy

A purposive sampling strategy (Kalleberg, 2007) would be employed to select

participants who would possess relevant experience and expertise in higher education,

(Population vs. Sample | Definitions, Differences & Examples, n.d.) examination development,

and AI technologies. Educators, examination committee members, and academic

administrators from selected universities are intended to be invited to participate in the study.

3.4 Participant Criteria:

- Educators with a minimum of 2 years of experience in conventional question paper

development.

- Examination committee members involved in setting university-level exams.

- Academic administrators responsible for overseeing assessment practices.

3.5 Sample Size:

A sample size of 10 participants will be deemed to achieve data saturation, ensuring a

comprehensive exploration of the research questions and objectives. (Guest et al., 2006)

Page | 15

However, flexibility is maintained to include additional participants if necessary. Certainly!

Here's a more concise version. The statement aligns with (Guest et al., 2006) who discussed

how around 10 participants can achieve data saturation. Flexibility allows for more if needed.

3.6 Data Collection Methods

Semi-Structured Interviews

Semi-structured interviews will be conducted with the participants to gather in-depth

insights into their experiences, perceptions, and practices regarding the use of

ChatGPT(Introducing ChatGPT, n.d.-b) in question paper development. The decision to use a

semi-structured interview pattern for data collection in the study was made for several reasons,

drawing on the benefits outlined by various researchers. One of these researchers is (Smith,

1995) who discusses the advantages of semi-structured interviews.

1. Flexibility: Semi-structured interviews offer a guide of key topics while allowing for new

themes to emerge.

2. Depth of Responses: Participants can elaborate, providing richer data, crucial for complex

topics like AI in exams.

3. Participant Comfort: Allows participants to express thoughts at their own pace.

4. Exploring Diverse Perspectives: Facilitates exploration of varied viewpoints from

educators, students, and administrators.

5. Validity and Reliability: Enhances validity and reliability by covering key topics yet

allowing organic conversation flow (Smith, 1995).

The interview guide includes open-ended questions focusing on:

- Familiarity with AI technologies and ChatGPT.

- Experiences in traditional question paper development.

- Perceived advantages and challenges of using ChatGPT.

3.7 Data Analysis

Thematic Analysis

Thematic analysis is considered to be used to systematically identify, analyze, and report patterns (themes) within the qualitative data gathered from interviews(Braun & Clarke, 2006). thematic analysis was chosen as the method for several reasons:

1. Flexibility Thematic analysis is flexible, crucial for exploring complex topics like AI in exams.

2. Identifying Patterns Excels at finding common themes across diverse perspectives (educators, students, administrators).

3. Relevance to Study Unveils key themes: attitudes toward AI, implementation challenges, benefits, and ethical considerations.

4. Depth of Analysis Provides nuanced insights beyond simple categorization into AI's role in university exams.

The following steps are followed:

1. Data Familiarization, Researchers immerse themselves in the data, gaining familiarity with interview transcripts and exam papers.

2. Generating Initial Codes, Initial codes are created to label interesting features of the data related to ChatGPT's efficacy, advantages, challenges, and impact on question paper development.

Page | 17

3. Searching for Themes, Codes are collated into potential themes, grouping similar codes together.

4. Reviewing Themes, Themes are reviewed and refined to ensure they accurately represent the data.

5. Defining and Naming Themes, Final themes are defined, named, and organized into a coherent narrative.

3.8 Research Ethics:

Informed Consent

Prior to data collection, participants are provided with informed consent forms outlining the purpose of the study, confidentiality measures, and their rights as participants. Only those who provide written consent are included in the research.

Anonymity and Confidentiality

Participant identities and specific university affiliations are anonymized to ensure confidentiality. Data is stored securely and accessible only to the research team.

Ethical Considerations

- Respect for participants' autonomy and rights.

- Transparency in research procedures and intentions.

- Minimization of harm and mitigation of risks.

- Ensuring data accuracy and reliability.

Trustworthiness and Rigor

Credibility

Page | 18

Credibility is established through prolonged engagement with the data, member

checking with participants to validate findings, and triangulation of data sources (interviews

and document analysis).

Transferability

Detailed descriptions of the research context, participants, and data collection methods

enhance the transferability of findings to similar contexts.

Dependability

Research procedures and decision-making processes are clearly documented, allowing

for the replication of the study.

Confirmability

To ensure objectivity, researchers maintain reflexivity and document their biases and

assumptions throughout the research process.

3.9 Conclusion:

This chapter delineates the qualitative research methodology employed to investigate the efficacy of ChatGPT in university-level exam development within the selected universities of Baluchistan's higher education landscape. Through semi-structured interviews and the study aims to uncover valuable insights into the integration of AI technologies in assessment practices. Thematic analysis will elucidate patterns and themes, providing a rich understanding of educators' perspectives and experiences.

References

Al-Matari, A. S. (n.d.-a). Artificial Intelligence and the Future of Teaching and Learning. https://doi.org/10.13140/RG.2.2.28132.76160

Bailey, L. F. (2014). The origin and success of qualitative research. https://doi.org/10.2501/IJMR-2014-013, 56(2), 167–184.

https://doi.org/10.2501/IJMR-2014-013

Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2), 77–101. https://doi.org/10.1191/1478088706qp0630a

Franciscu, S. (2023b). ChatGPT: A Natural Language Generation Model for Chatbots. January. https://doi.org/10.13140/RG.2.2.24777.83044

French, S., Dickerson, A., & Mulder, R. A. (2023). A review of the benefits and drawbacks of high-stakes final examinations in higher education. Higher Education. https://doi.org/10.1007/S10734-023-01148-Z

Gao, B. (2022b). Research and Implementation of Intelligent Evaluation System of Teaching Quality in Universities Based on Artificial Intelligence Neural Network Model. In Mathematical Problems in Engineering (Vol. 2022). Hindawi Limited. https://doi.org/10.1155/2022/8224184

Georgieva-Hristozova, V. (2023b). Artificial Intelligence in Education – Pedagogical Reflections. Педагогически Форум, 11(4), 25–32. https://doi.org/10.15547/pf.2023.024

Haniya, S., Tzirdes, A. O., Georgiadou, K., Montebello, M., Kalantzis, M., & Cope, B. (2020). Assessment Innovation in Higher Education by Integrating Learning Analytics. International Journal of Learning Analytics. 6(1), 53–57. https://doi.org/10.18178/IJLT.6.1.53-57

Guest, G., Bunce, A., & Johnson, L. (2006). How Many Interviews Are Enough? An Experiment with Data Saturation and Variability. Field Methods, 18(1), 59–82. https://doi.org/10.1177/1525822X05279903

Page | 20

Herrmann-Werner, A., Festl-Wietek, T., Holderried, F., Herschbach, L., Griewatz, J., Masters, K., Zipfel, S., & Mahling, M. (2023). Assessing ChatGPT’s Mastery of Bloom’s Taxonomy using psychosomatic medicine exam questions. MedRxiv, August, 2023.08.18.23294159.

https://doi.org/10.1101/2023.08.18.23294159

Herrmann-Werner, A., Festl-Wietek, T., Holderried, F., Herschbach, L., Griewatz, J., Masters, K., Zipfel, S., & Mahling, M. (2024). Assessing ChatGPT’s Mastery of Bloom’s Taxonomy Using Psychosomatic Medicine Exam Questions: Mixed-Methods Study. Journal of Medical Internet Research, 26, e52113. https://doi.org/10.2196/52113

International Association for Development of the Information Society. (2014). MCCSIS 2014. IADIS Press.

Introducing ChatGPT. (n.d.-a). Retrieved February 5, 2024, from https://openai.com/blog/chatgpt

Joshi, S., Rambola, R. K., & Churi, P. (2021b). Evaluating artificial intelligence in education for next generation. Journal of Physics: Conference Series, 1714(1). https://doi.org/10.1088/1742-6596/1714/1/012039

Kalleberg, R. (2007). Robert K. Merton: A Modern Sociological Classic. Journal of Classical Sociology, 7(2), 131–136. https://doi.org/10.1177/1468795X07078032

Khan, A., Bashir, S., Bazai, P., & Rehman, M. U. (2023b). Higher Education in Balochistan: Status and Way Forward. Journal of Social Sciences Review, 3(1), 68–85. https://doi.org/10.54183/jssr.v3i1.116

Kherdekar, R. (n.d.). Advances in Artificial Intelligence. https://www.researchgate.net/publication/359051296

Luckin, R. (2017b). Towards artificial intelligence-based assessment systems. Nature Human Behaviour, 1(3). https://doi.org/10.1038/s41562-016-0028

Matthew, U. O., Kazaure, J. S., Ndukwu, C. C., Ebong, G. N., Nwanakwugwu, A. C., & Nwamoh, U. C. (2024b). Artificial Intelligence Educational Pedagogy Development (pp. 65–93). https://doi.org/10.4018/979-8-3693-2314-4.ch003

Page | 21

Neha, K. (n.d.-b). Role of Artificial Intelligence in Education.

https://www.researchgate.net/publication/351082272

Nguyen, N. D. (2023). Exploring the role of AI in education. London Journal of Social Sciences, 6, 84–95. https://doi.org/10.31039/liss.2023.6.108

Nguyen, T., & Nguyen, M. T. (2023). Empowering Education: Exploring the Potential of Artificial Intelligence; Chapter 9-Artificial Intelligence (AI) in Teaching and Learning: A Comprehensive Review.

https://www.researchgate.net/publication/374508985

Owan, V. J., Abang, K. B., Idika, D. O., Etta, E. O., & Bassey, B. A. (2023). Exploring the potential of artificial intelligence tools in educational measurement and assessment. Eurasia Journal of Mathematics, Science and Technology Education, 19(8). https://doi.org/10.29333/ejmste/13428

Pisca, A. I., Edu, T., Zaharia, R. M., & Zaharia, R. (2023b). Implementing Artificial Intelligence in Higher Education: Pros and Cons from the Perspectives of Academics. Societies, 13(5). https://doi.org/10.3390/soc13050118

Population vs. Sample | Definitions, Differences & Examples. (n.d.). Retrieved April 27, 2024, from https://www.scribbr.com/methodology/population-vs-sample/

Rind, I. A., & Malik, A. (2019a). The examination trends at the secondary and higher secondary level in Pakistan. Soc Sci Hum Open, 1(1). https://doi.org/10.1016/j.ssaho.2019.100002

Rubab, I., & Imran, A. (2023). A Comparative Analysis of Traditional and Online Assessments: Perceptions and Performance of Undergraduate Students. Pakistan Journal of Humanities and Social Sciences, 11(2). https://doi.org/10.52131/pjhss.2023.1102.0439

Schwartzman, S., Busemeyer, M. R., Busemeyer, M., Cloete, N., Drori, G., Lassnigg, L., Schober, B., Schweisfurth, M., & Verma, S. (2017). The contribution of education to social progress SEE PROFILE Coordinating Lead Authors: Christiane Spiel, Simon Schwartzman. https://www.researchgate.net/publication/330579447

Smith, J. A. (1995). Semi structured interviewing and qualitative analysis. Rethinking Methods in Psychology, 9–26. https://uk.sagepub.com/en-gb/eur/rethinking-methods-in-psychology/book204294

Page | 22

Tatlah, I. A., Mukhtar, S., Tatlah, A., & Saeed, M. (n.d.). AN ANALYTICAL STUDY OF HIGHER EDUCATION SYSTEM OF PAKISTAN.

Ugochukwu Okwudili Matthew, Kafayat Motomori Bakare, Godwin Nse Ebong, Charles Chukwuebuka Ndukwu, & Andrew Chinonso Nwanakwugwu. (2023b). Generative Artificial Intelligence (AI) Educational Pedagogy Development: Conversational AI with User-Centric ChatGPT4. Journal of Trends in Computer Science and Smart Technology, 5(4), 401–418. https://doi.org/10.36548/itcsst.2023.4.003

Ungerer, L., & Slade, S. (2022). Ethical Considerations of Artificial Intelligence in Learning Analytics in Distance Education Contexts. In SpringerBriefs in Open and Distance Education (Issue May). Springer Nature Singapore. https://doi.org/10.1007/978-981-19-0786-9_8

Vázquez-Cano, E. (2021). Artificial intelligence and education: A pedagogical challenge for the 21st century. In Educational Process: International Journal (Vol. 10, Issue 3, pp. 7–12). Universitepark. https://doi.org/10.22521/EDUPIJ.2021.103.1

Xia, X., & Li, X. (2022a). Artificial Intelligence for Higher Education Development and Teaching Skills. Wireless Communications and Mobile Computing, 2022. https://doi.org/10.1155/2022/7614337

Owan, V. J., Abang, K. B., Idika, D. O., Etta, E. O., & Bassey, B. A. (2023). Exploring the potential of artificial intelligence tools in educational measurement and assessment. Eurasia Journal of Mathematics, Science and Technology Education, 19(8), em2307.

González-Calatayud, Víctor, Paz Prendes-Espinosa, and Rosabel Roig-Vila. 2021. "Artificial Intelligence for Student Assessment: A Systematic Review" Applied Sciences 11, no. 12: 5467.

UNESCO. Elaboration of a Recommendation on the Ethics of Artificial Intelligence. (accessed on 22 May 2021)

Chai, C.S.; Wang, X.; Xu, C. An extended theory of planned behavior for the modelling of Chinese secondary school students’ intention to learn artificial intelligence. Mathematics 2020, 8, 2089.

Xiao, M.; Yi, H. Building an efficient artificial intelligence model for personalized training in colleges and universities. Computer. Appl. Eng. Educ. 2021, 29, 350–358.

Sohn, K.; Kwon, O. Technology acceptance theories and factors influencing artificial intelligence-based intelligent products. Telemat. Inform. 2020, 47, 101324.

Page | 23

Chai, C.S.; Lin, P.Y.; Jong, M.S.Y.; Dai, Y.; Chiu, T.K.F.; Qin, J.J. Primary school students’ perceptions and behavioral intentions of learning artificial intelligence. Educ. Technol. Soc. 2020, in press.

Ince, M.; Yiğit, T.; Hakan Işık, A. A Novel Hybrid Fuzzy AHP-GA Method for Test Sheet Question Selection. Int. J. Inf. Technol. Decis. Mak. 2020, 19, 629–647.

Ouguengay, Y.A.; El Faddouli, N.-E.; Bennani, S. A neuro-fuzzy inference system for the evaluation of reading/writing competencies acquisition in an e-learning environment. J. Theor. Appl. Inf. Technol. 2015, 81, 600–608

Samarakou, M.; Fylladitakis, E.D.; Karolidis, D.; Früh, W.-G.; Hatziapostolou, A.; Athinaios, S.S.; Grigoriadou, M. Evaluation of an intelligent open learning system for engineering education. Knowl. Manag. E-Learning Int. J. 2016, 8, 496–513.

Deo, R.C.; Yaseen, Z.M.; Al-Ansari, N.; Nguyen-Huy, T.; Langlands, T.A.M.; Galligan, L. Modern Artificial Intelligence Model Development for Undergraduate Student Performance Prediction: An Investigation on Engineering Mathematics Courses. IEEE Access 2020, 8, 136697–136724.

Adiguzel, T., Kaya, M. H., & Cansu, F. K. (2023). Revolutionizing education with AI: Exploring the transformative potential of ChatGPT. Contemporary Educational Technology, 15(3), ep429.

Cukurova, M., & Luckin, R. (2018). Measuring the impact of emerging technologies in education: A pragmatic approach. Springer.

Bassey, B. A., Ubi, I. O., Anagbogu, G. E., & Owan, V. J. (2020). Permutation of UTME multiple-choice test items on performance in use of English and mathematics among prospective higher education students. The Journal of Social Sciences Research, 6(4), 483-493.

Arslan, K. (2012). Eğitimde yapay zeka ve uygulamaları. Batı Anadolu Eğitim Bilimleri Dergisi, 11(1), 71-88.

4. Hurley, R. (2019). *Data Science: A Comprehensive Guide to Data Science, Data Analytics, Data Mining, Artificial Intelligence, Machine Learning, and Big Data.* Independently Published. ISBN 978-1704636030

5. Balán, J. (2020). Expanding Access and Improving Equity in Higher Education: The National Systems Perspective. In S. Schwartzman (Ed.), *Higher Education in Latin America and the Challenges of the 21st Century* (pp. 59–75). Springer.

6. Hulman, A., Dollerup, O.L., Mortensen, J.F., Fenech, M., Norman, K., Stoevring, H., & Hansen, T.K. (2023). ChatGPT-versus Human-Generated Answers to Frequently Asked Questions About Diabetes: A Turing Test-Inspired Survey among Employees of a Danish Diabetes Center. *medRxiv.* Doi:10.1101/2023.02.13.23285745.

7. Nov, O., Singh, N., & Mann, D.M. (2023). Putting ChatGPT's Medical Advice to the (Turing) Test. *medRxiv,* 2023.01.23.23284735.

8. Hadi, M.U., Qureshi, R., Shah, A., Irfan, M., Zafar, A., Shaikh, M.B., Akhtar, N., Wu, J., & Mirjalili, S. (2023). A Survey on Large Language Models: Applications, Challenges, Limitations, and Practical Usage. *TechRxiv.* doi:https://doi.org/[insert DOI if available]

9. Liang, J.-C., Hwang, G.-J., Chen, M.-R.A., & Darmawansah, D. (2021). Roles and Research Foci of Artificial Intelligence in Language Education: An Integrated Bibliographic Analysis and Systematic Review Approach. *Interactive Learning Environments.* doi:https://doi.org/[insert DOI if available]

10. Almaiah, M.A. (2018). Acceptance and Usage of a Mobile Information System Services in University of Jordan. *Educational Information Technology, 23,* 1873–1895. doi:[insert DOI if available]

11. Chatterjee, S., & Bhattacharjee, K.K. (2020). Adoption of Artificial Intelligence in Higher Education: A Quantitative Analysis Using Structural Equation Modelling. *Educational Information Technology, 25,* 3443–3463. doi:https://doi.org/[insert DOI if available]

13. Almaiah, M.A., Jalil, M.M.A., & Man, M. (2016). Empirical Investigation to Explore Factors that Achieve High Quality of Mobile Learning System Based on Students’ Perspectives. *Engineering Science and Technology International Journal, 19,* 1314–1320. doi:https://doi.org/[insert DOI if available]

14. García-Peñalvo, F.J., & Corell, A. (2020). The COVID-19: The Enzyme of the Digital Transformation of Teaching or the Reflection of a Methodological and Competence Crisis in Higher Education? *Campus Virtuales, 9,* 83–98. doi:[insert DOI if available]

16. Creswell, A., White, T., Dumoulin, V., Arulkumaran, K., Sengupta, B., & Bharath, A. A. (2018). Generative adversarial networks: An overview. *IEEE Signal Processing Magazine, 35*(1), 53–65.

Vusumuzi Maphosa & Mfowabo Maphosa(2023) Artificial intelligence in higher education: a bibliometric analysis and topic modeling approach, Applied Artificial Intelligence, 37:1, DOI: 10.1080/08839514.2023.2261730

Page | 25

Adams, T., Brown, J. S., & Smith, K. (2020). Automating the grading process: Exploring the potential of AI-based tools in student assessment. Journal of Educational Technology, 42(3), 317-329.

Brown, E., & Chen, S. (2018). Analyzing human behavior in educational settings using artificial intelligence: A review of recent advancements. Educational Psychology Review, 30(4), 911-934.

Davis, L., & Taylor, R. (2021). Exploring the use of social robots in educational settings: A literature review. Computers & Education, 168, 104162.

Garcia, M., & Chen, J. (2020). Leveraging AI for didactic purposes: Creating virtual learning environments and virtual assistants. Journal of Artificial Intelligence in Education, 30(3), 377-389.

Jones, A., & Lee, C. (2019). Predicting student academic performance using AI: A review of current research and future directions. Computers in Human Behavior, 101, 287-296.

Smith, T. (2020). The ethical implications of using AI in education: A critical review. Educational Technology & Society, 23(1), 45-56.

White, H., & Miller, P. (2021). Factors influencing the adoption of AI-based tools in various fields: Lessons for education. Journal of Technology in Education, 14(2), 87-102.