Ben Riley visited the show on episode 093 The Artificial Intelligence (AI) Episode to discuss his recent report for educators considering applications of chatbots (a type of AI) in their teaching.
Michael Ralph 01:48
This month is our official AI episode. We are joined by Ben Riley, who wrote a guide for considering the use of AI for education. Our discussion considers the tasks for which AI might be useful, and the multiple concerns we have for its use as a substitute for thinking. Later, we read a study that shows AI can help people produce incrementally more creative task solutions. However, we are unconvinced that is ever the purpose of the educational process. Let’s get started. For our first segment, we read the educational hazards of generative AI,
Laurence Woodruff 02:25
This was written by Benjamin Riley and Paul Bruno,
Michael Ralph 02:29
and we are fortunate to have one of the authors joining us here today. And so welcome to Ben Riley, who is the founder of cognitive resonance, which is dedicated to helping people understand human cognition and generative AI, his favorite food truck in Austin, Texas, serves jambalaya on a stick, but like everything in Austin, he’s afraid it might be disappearing. Welcome. Thanks for joining us. Ben,
Ben Riley 02:51
Thanks for having me
Michael Ralph 02:53
So your report discusses the potential hazards of generative AI in education, particularly the risk of misunderstanding what AI can and cannot do well. They underscore the consequences of outsourcing deep thinking to technology based on the cognitive science of how humans learn. So can you tell us a little bit about what how did you come to decide to write this report?
Ben Riley 03:13
Yeah, great question. So I think, I think there’s no shortage right now of people in the education system, at all levels of the education system, who are incredibly excited about the potential for generative AI. I think, however, there’s also a large number of people also at all levels of the system who have concerns, and they have a wide range of those concerns, but some of those concerns just are about Like, maybe the technology can’t do all of the things that people are saying that it can do, or maybe it will have adverse impacts on education in a way that we should think more about. And the more that I saw the weight of hype and materials being produced in the pro AI in education camp, the more I became convinced that there was a need to just lay out some of the things to be at least aware of, if not concerned about. And so that was sort of the immediate Genesis. And I will say that a lot of it also was built off of, or stems from the fact that in my prior role leading an organization called deans for impact, I worked with Paul Bruno, then actually a classroom teacher and a cognitive scientist named Dan Willingham to articulate principles of cognitive science that would be useful for teachers to know and to connect those to specific classroom practices that we thought would be useful for teachers to be thinking about, and that document might be the most important thing I have ever done in my life, in the sense that it continues to be something that a large number of people all around the world look to for just basic insights into cognitive science. And you. Know, seeing that history and the impact of that, I thought, well, what could I do right now that could potentially have somewhere like that, some similar impact, in terms of just helping people understand sort of what these models are, large language models and generative AI are, and what are their strengths, and really, again, their limitations and the hazards that they pose. So it was both born of the moment, but also a little bit on, sort of my historical practice.
Michael Ralph 05:25
What is in the reaction or the the requests for supports from teachers in particular?
Ben Riley 05:32
It’s probably the case that the teachers that I’m connecting with are, number one, more inclined to pay attention to cognitive science than the average teacher. So I have to be aware that, like you know, my community, such as it is, are educators who already have, I think, a natural interest or disposition towards that. And I think because of that, or at least in correlation with that, I think those teachers are more inclined to also have at least some skeptical resistance to just being told, this is the bee’s knees.
“This” being generative AI. So, so the people who are reaching out to me are often sort of, you know, affirming, like, thank you. Like, we needed this. We wanted this. Some of it, though, is coming from places I wouldn’t have expected. I’ll give one example, and I won’t name by specific name, but like a professor at the Sorbonne in Paris reached out to me and felt like, you know, AI was being pushed in that university in ways that weren’t helpful. And we had an interesting back and forth over email about sort of how to navigate that one of my close high school friends, my prom date, who is now a professor at a community college, reached out to me because the community college that she teaches at in California is now pushing AI generative, AI tools, directly to the students, and they are claiming They are going to be doing a study, quote, unquote, of whether or not it improves GPA of their students. And she’s aghast, and is like trying to figure out how to have a conversation with the administrator saying, like, no, or at least like, if we’re going to do it, let’s do it in a much more thoughtful way. And I said to her, and would say to anybody listening, like the education hazards of generative AI is the document for you, like I very specifically had in mind, sort of this need to have something that you could pick up and read and go to the citations and say, look, there are a lot of outstanding questions that either we don’t know the answer to, or the answers are actually not the ones that the AI hype machine are willing to acknowledge and accept. So I think that’s sort of like, you know, the community that’s responding to what I’m putting out.
Michael Ralph 07:50
I think it’s worth maybe zooming in on you’ve said generative AI several times, but as we read your report, I was noticing it’s mostly talking about large language models, and I appreciate that you gave a pretty good pretty lay, friendly summary when we’re talking about AI, can you clarify what tools specifically are we talking about in this context?
Ben Riley 08:08
Yeah, well, within the context of the document, you know the education hazards of generative value, that’s exactly right. Like the focus is on large language models. And just in case people don’t swim in that terminology, like Chatgpt is a large language model. Anthropic puts out one that’s called Claude, which has gotten some traction and some attention. Like right now, there’s this very specific tool that everyone is at least curious about, and certainly is being pushed in educational settings. So let’s take that tool and try to figure out what’s good about it and what’s not so compelling about it.
Laurence Woodruff 08:47
Just to roll back a little bit about hype, I would like to talk about something that I’ve experienced. Two years ago, I went to a national, AVID Professional Development Conference, and we had a big, you know, keynote speaker session, and all of the attendees were there together. And the big reveal was that there was a new AI grading tool that they were going to start integrating. And, you know, it was all, you know, there was music and lights and cheers and applause, and I was sitting there with my eyebrows furrowed, thinking we’re why are we excited about this? Like, I’m not that. It’s not that obvious to me that this is clearly a benefit, and so I’m one of those, as you suggested earlier, like the skeptical folk. And I don’t think that’s a surprise to anyone who’s heard any prior episode of this podcast. And then in my school, there is a like the pressure is growing. It’s definitely a top down pressure. The district is telling admin and our building leadership team that they need to start slowly suggesting and advertising ways that teachers can use AI to do shortcuts and this and that or the other. And it’s not there’s no mandates and there’s no expectations, but it’s definitely a cultural let’s ease into this. Let’s find ways to make this convenient and approachable and and non-threatening to our teachers, so that we can integrate this. And again, like people, people on the on the leadership team were like giving me examples of ways that they used AI to generate a multiple choice test for themselves. And I, I ask why, and I’m really glad, And of course, everyone likes to read things that reinforce their worldview, which is exactly what happened to this morning when I read this paper. But it it did help me crystallize some of my concerns and ask some questions about why we’re doing what we’re doing.
Ben Riley 11:05
Well, that’s incredibly kind of you to say, and again, sort of affirms fundamentally what I’m hoping it will be useful for our teachers and educators in the situation like you’re finding in and that that you know, quite honestly, if the way your district is Hey, teachers, here’s some ways in which it might be useful to you, We’re not mandating anything, but you can think about using them in these ways. That’s actually not like terrifying to me, because the reality is, I think a lot of educators are probably going to, in fact, I know this is the case, that there are many educators who are going to experiment with this tool and and and sort of saying, like knowing that that’s the reality, and that there’s actually no way of stopping it, given this is like a direct to consumer product that they can just use at home if they want to. It’s not something a district can sort of control through its procurement processes. Then that’s at least better than what I have seen and am seeing in certain places. I wrote an essay that examines some AI guidance that was put out by Chicago Public Schools with support from an organization called AI for educators that was pretty remarkable in sort of laying out their vision of, sort of like what we want and what we don’t want, and literally have some boxes of sort of side by side pedagogical activities of what we want and what we don’t want, and under the want is the AI vision. And wonder what the don’t want is the not AI vision. And I’m sorry to say that, like, what we want is actually what is listed as what we don’t want, at least of what we want our students thinking carefully and thoughtfully about subject matter. And what we don’t want is what’s listed under the what we want, which is, like, all of this AI sort of nonsense and activities that aren’t going to be leading to productive thinking. So like, you know, I don’t have my finger on the pulse of this nation’s 15,000 school districts to the degree that I would like to, but there is no question that there are many of them that are pushing hard into this in ways that are completely unthoughtful and often reflect, I think, just a lack of a basic understanding of how these tools do what they do. So, you know, this is, again, sort of the whole bet behind my enterprise Cognitive Resonance, is to try to help people understand this, and especially looking at it side by side with human thinking, and in so doing, hopefully improve the quality of decision making and get away from sort of, this sense of inevitability, this sense of, if you’re not doing it, you’re a Luddite, like there’s a space, I swear there’s a space in between those two extremes. What is the goal? Is something that is, I think, really important to have in mind when we think about what a large language model is doing, what a chat bot is doing, the goal is to put, put out text that aligns to what it would predict would be in its training from the text that it was given. So it doesn’t have some larger goal than that. It literally is just, Hey, I’ve seen a bunch of texts then been trained on this text I’m totally anthropomorphizing by using the I so let me try to let me start that again. This tool gets a bunch of text inputted into it. It then runs and converts that text into data that it looks at all of the other ways in which it connects to all the text that it’s ever been trained on, and it says based on all of the text that it’s been trained upon, what should be outputted here? it’s a statistical next word prediction machine, or, if you want to get technical, it’s a next token prediction machine. And the one of the greatest papers I’ve read on this, which they literally just put out, sort of a shortened, peer reviewed version of, is called embers of auto regression, and it’s written by lead author Tom McCoy. Tom McCoy now at Yale University, who’s a linguist, and Tom and his co authors talk about having this goal driven teleological understanding of large language models. And what’s interesting is, once you get that, you can actually. Actually start to predict when they’re going to screw up. Because once you understand sort of that process that they’re going through, and it is a very complicated process, you can start to see where they’re going to have these, you know, sort of hiccups in their relationship between the text that they’ll produce because of what’s in their training versus what we would consider to be true.
Michael Ralph 15:18
And so as I read your report, there were, there were there were some things that resonated with me and there were some things that did not resonate with me. I think the idea of mistakes is much less compelling than than some of the other arguments that you have in that document, because a human tutor is also going to make mistakes, and textbooks and web searches are also going to have have the potential to make mistakes, and that’s all navigable like I think that’s something that is we can deal with that if we’re thinking about it. Whereas I think the most compelling reason for me that I would be very judicious in where I would use AI tools in the classroom is one of the things that you talk about later in the report, where, in essence, they are at their best when we are outsourcing some of our thinking. They are cheap thinking that is happening outside of our brains. And my number one job, when I think about teaching, is to help students think so. I’m not sure it’s super productive to be able to have a an incrementally better initial product right here, if the consequence of having that incremental improvement is to move the thinking outside of the brain that I am trying to develop. And so the long range consequence is that product is better at the consequence of less growth. And so it comes back that your comment, what is our goal? Is our goal this individual task, product, or is our goal productive struggle and thinking in the pursuit of growth of the brains. I think that that is something that a lot of the studies, the studies that I’ve read, the one that we’re going to talk about next, is looking at that incremental product progress. And I have seen very little thinking outside of your report that does engage with this of what are the consequences of outsourcing some of the deep thinking that is, I think, some of the best stuff that we’re doing in education.
Ben Riley 17:06
Okay, all right, thanks, Laurence, and I do want to hear what you have to say. So I appreciate the affirmation of the concern around what you know, some are calling in a term I have picked up, you know, the cognitive automation that large language models allow. It was actually a surprise to me to learn just recently that open AI, the purveyors of Chatgpt, are saying publicly that the majority of their total users are students like that is remarkable to me, so and we all sort of know, I think that students can and are using this tool, but the fact that they are actually the major use case at this point is telling, and the fact that the usage rates drop during the North American summer months when K-12 schooling is not in full season, is telling. So there’s no question that there is a very clear risk that is, I think, not risk. There is a very clear harm that’s happening right now that we don’t have a good grasp on the scale of and we may not find out until later, how damaging the lack of awareness we had to that has been
Laurence Woodruff 18:23
We, especially in the United States, have such a deeply baked capitalism, efficiency, mythology about how things should work, that the idea that we can make any process faster or quicker has this implicit superiority quality. And over and over again, I’ve said on this show, in my heart, in conversations everywhere, I believe that education should be difficult and problematic, and the experience that students have should be challenging with, with with points of success celebrated, but, but it’s a it’s a struggle. Growth requires struggle, and so whenever we prioritize convenience, we lose our purpose. And I think that that’s a at the heart of this issue, that convenience is awesome in business and a problem in education. And I think that’s true, not just for students, but I think it’s true for teachers too. Point two, let’s talk about the the parallel arguments. We said, hey, if, if information is available via Google. We don’t really teach anymore. That was that, that that argument had its day, and it’s largely passed. And if we look at what happened, um, if, if we look at what happened, we live in an age of really complex misinformation, like, not only. Only was that argument wrong, if we had all bought into it, by buying into it, we have contributed to a new wicked problem that we live in now about disinformation being disseminated at far greater complicated rates than actual information. It would be amazing if we could just look something up and it could be right, but we can’t, because it’s not so let’s apply that to this idea that we can use computers to teach our kids. We can use robots to teach our kids. Well, the robots meet expectations. That’s what the robots are doing. What do we expect the response to this question to be, which is not the same as what is the truth of this question? So that means, if we start relying on robots to teach, we are going to reinforce, as your paper suggests, misconceptions about teaching and learning, and we will reinforce them. So we will, like already there is this really entrenched as your paper references, uh, discussion about learning styles. And if we start popularly switching over to these things, they will become embedded in the culture. They will become the truth about the process. And number three, let’s talk about cheating in the classroom AI analyzers to say, Was this an AI generated thing or not? And there was a paper in Bloomberg that I have encountered that references a paper that actually you also reference in your work about English as second language, students being more likely flagged as cheating or AI generated the reflexive response that, oh no students are using computers to cheat is Now creating avenues for a new kind of injustice in disciplinary practices for academic dishonesty.
Ben Riley 22:08
Well, I liked what you said at the end there about the new forms of injustice. And I will say just quickly and pithily that we should not, under any circumstance, be using AI to do these sort of detection. I think the research is in on that it is totally unreliable and just don’t do it. But I want to get to your larger points, because I think both of them are quite interesting. And the notion that, you know, this is a very American slash capitalist push, I’m sympathetic towards that view. And one of the reasons for that is that there was this huge movement in education, not that long ago, in fact, very recently, to personalize education. And what that meant, typically, was to figure out a way in which every kid should get their own path of education and move at their own pace, and the technology should be used in order to figure that out. At the time that this was first gaining traction, the idea was that we would use machine learning algorithms in sort of their rawest sense, to determine that and and what happened was, is it didn’t work very well, and a lot of the money went up in smoke that was pushed towards that effort, and it was just about, I think, on the ropes as something that people were excited to try. And then along comes chat bots, as we now have them, and it’s all come back. Sal Khan, who has had 10 different theories of what was going to transform and disrupt education now, is convinced that it’s going to be personalized using AI. Never said it before. Never said that like you know, having this AI based tool is going to be the key. But now that we have it, now we know it’s not going to work. It’s not going to work for the same reason it didn’t work before. And I think it’s, you know, it is it capitalistic? Sure, sure, we could say that. I think it’s really more just American, individualistic, and so I think there’s something really true about what you were getting at, about sort of this, these systems are producing things based on what’s expected. Again, that is the common sense that they are imbued with. Now, ironically, sometimes what’s expected based on training doesn’t accord with what we actually think is true, and that’s where we get the so called hallucination. So even at what they’re designed to do, they’re not particularly reliable yet, and may never be, based on how the technology works, but there’s this big gap that fundamentally misses, I think, what is important about being human, about being creative and artistic and trying to push our thinking. I don’t think that these tools will ever be good at that on their own, and frankly, I’m skeptical that they’ll be very useful at helping us do that.
Michael Ralph 24:40
I think to follow that line of thinking. And one of my key notes out of this was, regardless of how the models continue to be refined by the people who are developing them, the essence of that like they are the common sense and you’re the paper makes the point of, if you just like Google curricular materials and look at all the things that are spread. Across the entire footprint of the internet, a lot of those materials are pretty low quality. And so if you like, put that all into a blender and make yourself a slurry of those materials and spit it out, it will similarly be low quality and biased and embedded with all these mistakes and all these other issues, which is navigable if you want to, if you have the expertise to go back in and say, Okay, well, this isn’t right, and I should, I should tweak those. I need to incorporate this new material. But at the end of the day, Laurence, back to your point about efficiency and this fetishization of efficiency of once you start to say, well, we can use this tool, but we need to do this, and we need to check that, and we need to vet that out. We’ll need to rewrite this. And then the reality of how much time you spent doing it, you didn’t actually save yourself any time, like it’s not actually more efficient to use it in a way that is even remotely responsible in most use cases. And so it’s not that it can’t work, it’s that it doesn’t work for its intended purpose, and it’s unlikely to ever do so, because you’ve got to have the human in the loop for some of those key points, those those key, key corrections and revisions, and so at the end of the day, you might as well just written it yourself, or gone to an established and reputable source for somebody who has already written it for you. And that’s doesn’t come as a shock to any of our listeners who are already saying we need high quality, viable curriculum. Sure do, like, that’s the end of the day. That’s what we’re talking about.
Ben Riley 26:24
Yeah, it’s such a great point. And I was listening to a podcast recently that featured Allison Gopnik from UC Berkeley, who’s written a lot about the cognitive science of emotion and in infants. And She invoked a metaphor that I’m blatantly going to steal now about large language models are the stone soup of technology of our times, and what she means by that. And we all remember the child story of Stone Soup. Like, you know, people come into this village, they’re like, we’re gonna make a soup. We’ll put the stones in it, but you have to each contribute an ingredient. And they’re like, Oh, okay. And then, you know, they think it’s the stone, but really it was the contribution. It’s very socialist, you know, very European and but the if you look at a large language model, it’s like, well, let’s start with number one, like it’s taking all of the knowledge that humans have created, right? So big contribution to the soup is coming from literally everything that humans have said that has been written down and stored somewhere on the internet. Number two, then there’s a massive amount of human training taking place to make them even more useful as a tool. This is sometimes called response to human feedback, or responsive learning to human feedback. So there’s like, even just on their own, based on that hoovering up of data, they still will say a lot of things you need to go in and start doing, like AB testing and having humans sort of say, yeah, that was a better response than that response. There’s a lot of work that goes on to that. And then after that, you then have the user and the so called prompt engineering that a user who types in. And it’s hilarious to me, because you will see people putting out these pre-written prompts saying, Look at what I can get this to do, and it’ll be like, after a three page written prompt. And it’s like, cool, but like, you spent how many hours and how much time writing that three page prompt? Like, how much did this just cover the last mile? Now I want to be clear, there are good use cases of it. There really are. And I don’t want to make it sound like there’s absolutely nothing of value. It would not have had the impact and import in our society if it was just something that was completely useless. It’s not it’s not the case. But nonetheless, there’s this sort of quality where we’re almost willfully denying how much human knowledge and capability and cognitive insight is being applied at every level of usage, and attributing it all to the technology.
Laurence Woodruff 28:47
I don’t know if this is gonna make tape, which is something that we say every single episode, every single segment, but I’ve been using, I like to use athletics and other types of metaphors for growth mindset and how it’s here in athletics and sports that like if you go to the gym, but you bring a robot that lifts the weights for you. When you are done, you have achieved nothing. That’s what it’s all about. Making things easier does not achieve the goal. What are our goals? And let’s use the right tools to do them. And if, and if you’re a student that wants to get better at writing papers, then you need to be the one writing the papers.
Ben Riley 29:34
Yeah, I think that’s well said. And I do think, while there are some disanalogies with calculators that we could go into, but, but I don’t think are necessarily critical at this point. Like, the reality is we adopt the technologies, and if they’re successful, they mostly fall into the background, right? If they are sort of like, you know, just solving some particular issue, and then we just don’t think about it anymore. I. Was doing a presentation to a group of education consultants, and one of the participants said, what if AI is just like email? And I have to say that, like that thought and idea has stuck with me, because I don’t know what the if you add it all up, how much time and the sheer volume of emails I produce in my life, but it’s probably one of the most significant things I’ve done in my life, just in terms of sheer amount of time spent doing it. But no one really thinks of that at least. I certainly don’t think of that as like, you know, some transformative thing to my experience as a human being. It’s just part of the fabric of the reality of which I have been presented. And it could be that AI will just end up being sort of like this thing that sort of gets folded into various aspects of our reality, that is clearly already the case, and you just don’t think or use it in a way that sort of, I think, where we are right now, which is still this, like, discovery and hype and belief that, like, oh, like, as many do, like, we’ve created some form of either proto-sentience, or is about to become fully sentient and have general intelligence. I think we’re in this moment of sort of Starry Eyed wonder that will probably give way to more realistic cynicism in the not too distant future.
Michael Ralph 31:13
Well, thank you so much for joining us. That brings us to the end of our time together. For any of our listeners who have enjoyed what you’ve had to share, or are interested in your resources. Where could they find more of your work?
Ben Riley 31:23
There is a document called education hazards of generative AI that is very free and very available on the cognitive resonance website filled with and I really tried to be deliberate about this, and you might not know it unless you dug into unless you dig into it, which is that the citations are all to documents that I think are accessible even if you don’t have a PhD. People might disagree. I mean, there’s some wonk in there, for sure, but I was very, very careful to try to select for research papers that influenced my own thinking and helped me understand these tools. And my co author, Paul Bruno, I know, was very attuned to sort of like, coming up with and identifying these like practical guidance for educators that you really have to think about in terms of using it for tutoring, using it for particularly teaching a particular subject, like all of that’s laid out in the document, I think, in fairly accessible and bite sized ways. So I would just underscore that if you’re out there as an educator, thinking, wondering, having some questions, Maybe some skepticism, check it out.