AI in Higher Education

Donald Johnson knows technology and he knows teaching. Now, he's learning how to use AI in higher education.

As soon as he heard about ChatGPT in November of 2022, Johnson signed up for the free version and started playing around with it. He uses it in his classes now. He says his knowledge and thinking are evolving, just like the AI programs and offerings themselves. His key takeaway so far: Use it to make yourself better than you are alone.

But he has questions, and not just the one that is top of mind for most faculty members: potential cheating by students using AI. Johnson also wonders:

• How can AI be used productively in the teaching and learning process?
• What about faculty using the generative AI large language model and other programs like it to write letters of recommendation for students?
• What about using it to help turn a phrase while writing an academic research paper?

Johnson, who holds the rank of University Professor in the Department of Agricultural Education, Communications and Technology in the Dale Bumpers College of Agricultural, Food and Life Sciences, is in his 31st year at the U of A. He teaches courses in agricultural systems technology, which he described as kind of a bridge between engineers who design the technology and users of the technology. He teaches students from freshmen to doctoral students, on campus primarily, with one online graduate class in quantitative statistics each fall.

"I was aware of AI before that because it has been used for years in ag systems," Johnson said about the unveiling of ChatGPT, "but the use of AI at that point was a fairly complex technological undertaking. You had to learn specialized languages and skills and so forth. It's not like the chatbot that we have now that you just ask a plain language question and that gives you a response."

Benefit for Students

Johnson's research centers on effective classroom teaching. So, naturally, his goal with AI is to help students understand how to use the technology effectively and the pros and cons of using it. It's also very important they understand the results they get have to be closely checked, he said. That means students have to learn in order to know if they are getting accurate results from AI.

"Since I'm a teacher, I really focused on what can you do in classrooms with it," he said. "I mean, what should students be learning about ChatGPT and other AI programs? I'm not an expert on that, but I'm curious about it."

Johnson, along with other University of Arkansas faculty and staff, took an online course offered by Auburn University that took about 12 hours to complete.

"I learned a lot," he said. "You got a lot of different perspectives. I mean a whole gamut of people, some who are really hesitant, they just are looking for ways to keep AI out of the college classroom because they're worried about cheating and all those kinds of things, all the way to the folks who think that AI is here to stay. If the kids don't learn to use AI, then they are not going to be marketable."

Cautionary Tale

He described his first use of technology in class as cautionary. He fed the questions on a weekly homework assignment in an introductory course into ChatGPT.

"Then, I graded ChatGPT just like it was one of my students, and I shared the results with the students," Johnson recalled. "It scored 4½ out of 10 on the homework assignment. We had an in-class discussion about ChatGPT and, frankly, most of the students claimed not to be aware of it at that point and did not have ChatGPT accounts. I figured the students would be all over it because of the potential to use it to make college easier. But, according to them, they weren't."

The cautionary tale, he explained, is that, if you use ChatGPT to do your homework, you still have to understand the concepts well enough so you can check the answers.

"Whatever you turn in is your submission and, if ChatGPT makes a mistake, it's your mistake," Johnson told the students. "I told them, if I were a student coming into college right now as a freshman, the thing I would focus on throughout my entire four to six years, however long they are here, would be how to use ChatGPT and other artificial intelligence programs to add value in an ethical and responsible manner."

He said students will have to research how they can use ChatGPT in their intended career field. Instructors can guide them in the process.

"I tried to emphasize to them that, if the only thing you can do is what ChatGPT can do and you can't add value to it, ChatGPT is virtually free and you're a 50 or 60 or $70,000 a year employee," Johnson said.

Which will the employer choose?

"Why would I hire you for $70,000 when I can get the same output from ChatGPT for essentially nothing," Johnson posed.

In an in-class assignment, students discussed how if you don't put the exact query into an AI model, you won't get the exact results you need. The students could see how a better query would have resulted in less error by ChatGPT, he said.

In general, his students seemed a little distrustful of AI, Johnson decided.

"I think there are two parts of it," he said. "They have a little distrust of ChatGPT accuracy, but they also have a lot of doubt about, am I capable of writing a query that will elicit a correct answer to my question? I think that's an important teaching aspect.

"How do you write queries that get what you want to know? And, can you write a query that does what you need it to do about a subject that you don't know much about? And, if you can't, then you need to learn the subject that reinforces that."

Eventually, in a back-and-forth exchange, Johnson said, what the student learns improves ChatGPT's knowledge and the student is adding value to ChatGPT as a tool. ChatGPT can't do the work without your knowledge, but you can't use ChatGPT unless you understand the concepts and materials.

"They work together," he said. "You're better than ChatGPT and ChatGPT makes you better than you are alone so that's a net win. It's a win that makes you more employable. That's what I hope I'm giving students to understand."

Faculty Use

Faculty members have to invest time into their own education about AI to use it effectively, Johnson said, but they will gain time as they become more proficient with the technology.

"If I want to assign something and develop a rubric for grading it, I can just input the assignment into ChatGPT and ask it to generate four different rubrics for evaluation," he said. "A rubric takes me an hour or two to develop, but it'll give you four choices in seconds."

Johnson said he has heard of colleagues using AI to write letters of recommendation for students.

"I have not done that yet," he said. "I feel a little hesitant about doing that. If it's my recommendation, I should write it. But, I know folks are doing that and they turn out dandy letters, and to the student it's all the same. I'm still overcoming that hurdle. I guess I've been writing letters for so long that it just seems a little bit strange to do that."

He has considered the possibility of using AI for some writing but nothing that would appear in a published article another researcher might cite.

"The temptation is there but, so far, I have not used it (in writing)," Johnson said. "I have plugged some results into ChatGPT and said, ‘I think the three main results of this study are these. Evaluate the results and tell me what you think.' It has come up with some pretty good suggestions, and I'm not quite sure what to do with them."

Johnson said he was talking with a colleague recently, and they discussed the implications of a colleague asking a question that resulted in a new line of thinking. Would the researcher cite the colleague for that idea offered in passing? Probably not.

"So, what's the difference between ChatGPT and walking down the hall and talking to a colleague?" he said. "I'm just beating around in the forest with everybody else on this. I don't know the answer."

A technique Johnson said he would like to try this semester is asking his students to query ChatGPT to write five homework questions on the topic for that week. Students would submit their query, the ChatGPT-generated questions and their responses for a grade.

"I thought that would be interesting," he said. "I wouldn't have to write homework questions and everybody, theoretically, would have a different homework assignment.

"I don't think ChatGPT has alleviated college graduates or anybody of the responsibility to know stuff," Johnson said.

What About Research?

Johnson conducted an experiment with students in his introductory agricultural systems technology course last fall, dividing them into two groups as they performed a programming task with Arduino microcontrollers. All of the students were novice programmers; about half used ChatGPT in the exercise and the rest did not. Johnson wanted to see whether the AI would influence student performance as well as their interest and self-efficacy.

Some of Johnson's results:

• Mean rubric scores were higher for the self-programming group at 94.8% than the ChatGPT-programming group at 90.4%.
• A higher percentage (68.8%) of the Chat-GPT-programming group made a perfect score compared to the self-programming group (41.2%).
• Programming errors resulted primarily from incomplete or incorrectly worded ChatGPT queries.
• Observed means for self-efficacy and post-test scores were higher for the self-programming group than for the ChatGPT-programming group.

Questions remain, including would it be more effective to focus on improving students' basic programming skills or how to write complete and correct ChatGPT queries, Johnson wrote in a soon-to-be-published article on the experiment. Or, is a combination of instruction in both warranted? Because the students were largely unfamiliar with ChatGPT, having them use it could have increased the complexity of the task that was not experienced by the other group, Johnson wrote. Like any good university researcher, he identified numerous areas for further research.

"What gets me so excited is the possibilities," he said. "That doesn't mean that teachers and researchers are less important. This is just another tool that can make them better."

Online Learner Blog Home

 

Your
Thoughts?

Related Articles

• Instructional Designers Empower Educators, Students in Use of Artificial Intelligence Tools
• Instructional Designers Address Challenges, Promote Equity in Artificial Intelligence