Brave New Words: How AI Will Revolutionize Education

by Khan, Salman

Published: May 14, 2024

Read: September 14, 2024

★★★★★

Goodreads

Review

This is basically an ad for Khanmigo—more a vehicle of change convincing educators than a source of truth. But it is probably the most comprehensive vision and discussion of AI's impact on education, so still worth the short read. Khan paints a glorious future where Khanmigo is always there to help students, teachers, and parents, redefining the way we do everything from testing to recruiting. I do appreciate his underlying optimism for the future and Khanmigo's team carefully considering potential risks. But he so adamantly asserts teachers will not be replaced that it feels like he is just trying to soothe his future users. In one paragraph, he insists teachers will never be replaced, for they motivate learning with a personal connection. But in another, he praises AI's ability to do mental health coaching, noting we should start talking about AE (Artificial Emotion). It seems like AE can progress to form this personal connection, especially since children anthropomorphize everything already. This seems to me to be the biggest risk with AI in education—replacing children's real social connection and skills for the sake of knowledge in an AI-driven world where the opposite trade makes sense. Key Quotes: “This is not a drill, the AI tsunami is here to stay.” “Is Khanmigo a replacement for a teacher? No, and it never will be.” “Teachers will always be in command of their classes and thank goodness for that.” “Technology augments the motivation that a relationship with an adult gives them.” “There is no job that is safer in LLM world than teaching.”

Notes

Key Takeaways

• Writing
  • Doesn’t matter whether it can be called cheating, but writing
  • Khanmigo can talk students through essay creation, give realtime feedback and estimated grade and report on process for grading it
    • Khanmigo will give suspicion if just pasted in and no process with it
• “is Khanmigo a replacement for a teacher? No and it never will be”
• ai Tutor feedback was it worked for students with motivation to seek help, so they added check ins and emails for goals (lol like that’s the same thing as real tutor)
• Isolation
  • Khanmigo can organize splitting work of project, can invite multiple students to play learning game
  • “you are always gonna get better collaboration if you have a great facilitator”
• “Life satisfaction comes from establishing goals, working hard, and feeling like you are moving world towards a better place”
• ai as guardian Angel, oversight of browsing, censoring
• “there is no job that is safer in LLM world than teaching”
• Average teacher works 54 hours per week
  • Only half with students
• “remove the teacher and you remove the critical element foundational to all learning”
  • “teachers will always be in command of their classes and thank goodness for that”
• “technology augments the motivation that a relationship with an adult gives them”
• **computation of gpt4 for khanmigo is 5-15/month per student
• Assessment
  • ai means you can now do rich assessment like projects and dictation and spoken tests that would require so much human work before
• khanmigo could write standardized letter of rec for students, imagine if everyone had same teacher
• biasfree is impossible, but better than status quo is possible
  • ai can be audited unlike human recruiters/interviewers and do checks or surface overlooked candidates
  • every job candidate can have conversation with recruiter bot or even your agent bot
• Early 2000 software outsourced and author thought he needed to switch into business, but instead software salaries exploded
• Job least affected by ai is roofing, but ai will change proposal
• “This is not a drill, ai tsunami is here too stay”
  • people with time but no purpose or meaning susceptible to populism,
  • ai can soon create internet surveillance, fraud and misinformation, and war
  • but bad actors won’t slow down, but it’s a race

Intro

• write story with daughter and she can talk to the character
• Khan always hoped to provide 1on1 tutor, need tech to scale
  • Received email from OpenAI 4 months away from chatgpt, they wanted test of AI logic and reasoning
    • could answer bio ap test :o
  • good videos like AI let teachers move up value chain by replacing lectures
• History
  • Claude Shannon 1948 started with next word prediction
  • Turing 1950 wrote about turing test
• author worried gpt becomes crutch answering questions, jobs, addictive quality
  • after 10 hours of early beta access, playing realized it had insane potential
• Khan Academy 30 engineers hackathon
  • lesson plans
  • quiz debate or lead student
  • immediate essay feedback
  • create projects
• Will we get a generation of cheaters?
  • Similar concerns with videos, will it just distract and isolate students
  • But concerns can be mitigated, around cheater would make AI tutor that wouldnt give answers, transperency around all conversations, teachers and student tools so they could spend time together
• "be careful but not fearful of change" cuz lets get this out to everyone
  • will change education including teachers and credentialing, work, and human purpose

Part 1 - Rise of AI Tutor

throwing away the bottle

• LA first to ban ChatGPT and many others followed cuz no cheating!
• We worry catastrophic effect on writing skills and biased sources create bias output, but tech allows good and bad like phone
  • Ideally we use it to enhance connection and learning
• best writers will be those who use AI tools to create great work(problem is AI will outpace our learning speed and oneshot will leapfrog human knowledge, it will be an irrelevant skill)

how to teach everything to everyone

• what could ai do?
  • Close learning gaps and teach acrid learning circumstances and economic constraints globally
  • Save teacher time to prevent overwork
  • AI that engaged students in creative and exploratory stuff
• 2022 and 2023 khan would make personal tutor and teacher tool
  • instead of answering, make it ask questions
  • GPT4 unlike gpt3.5 could play role without giving answer and had less math and general errors
• March 2023 khanmigo would be integrated with everything
  • guidance consouler, memory, can check up on you and your goals
• 1on1 attention “student never feels stuck or bored”
  • Alexander the Great got 1on1, Education luxury, decided on bold public education by batching students ^8e0eff
• public Ed Forces kids to move on even if they didn’t get it
  • 3/4 of high school seniors lack proficiency in writing
  • Bloom from Chicago found good caring tutor with clear leaning objective
    • mastery learning
    • 2 sigma 50% to 96%
• Khan wants trust and relationship with tutor who knows what someone knows and doesn’t know
• khan tried to push personalized mastery learning
  • 30-60 minutes on personalized experience 20-60% acceleration
  • Math learning 1.5 to 3 years per year
• Khanmigo goal to be for everyone
  • Memory of what student likes so can make all football questions
    • modifiable memory
  • hard to distinguish from human tutor
• political sensitive?
  • When asked by someone why 2nd amendment exists, tutor asked why student thinks founders had it
    • Better than human say not showing bias even
• Students might even like ai tutor more cuz can’t be embarrassed or annoying
• with memory and content knowledge and always available means ai tutors might surpass blooms findings

part 2 giving voice to social science

why students write

• teacher finds essay that is so clear but so blatantly and confidently says wrong things, gpt4
• Writing is form of communication, need to have ideas and analysis
• hick decided to go from policing gpt, to leveraging it
  • inperson proctored essay to create reading, don’t need take home essay
  • For research, won’t ai be used in real world
• grammar checker might be considered cheating to older teachers, what about critiquing or writing first draft
• Ai not good at originality, teacher decides it is for first pass and blank page
• Doesn’t matter whether it can be called cheating, but writing
  • ai improves the bad students and ESL
• Khanmigo gives themes to consider, feedback, counter arguments, will report on process to teacher to help understand weaknesses and grade it
  • Khanmigo will give suspicion if just pasted in and no process with it

the future of reading comprehension

• 2020 Gallup 54% of Americans between 16-24 read below 6th grade level
• talk to book character and author
• Passage based multi choice on SAT are standard, but writing and short answer would be better and create deeper readers
  • Khan Academy currently working on this kind of assessment, good discussion but struggle with assessment
    • Author thinks we will get there

Ai and creativity

• 40 years ago, ocr removed just quotes from books to make screen play that screen writers guild got mad about replacing their with, early problem we face now
• “Is gen ai a Creativity killer?”
  • Noam argues for creativity distinction between LLm and human
    • Human operates with small amount of info, not brute force correlations
      • Author says our biases are these brute force correlations
• author could find after engaging with problem for a bit, then subconscious could give him the answer often next morning
  • for best output experts, flow State is just not thinking
  • best ideas will come from ai riffing with us, not for us
• author sees better learning from daughter creating structured stories she could never do alone
• ai tutor with image rec can help you explore styles, real time feedback for writing and painting, improv music
• cameras might seem like way to cheat, but created new art
• will it make professional writers obsolete?
  • Screenwriters will become full movie procurers
  • we could have 1k Mozart, requires creativity and exposure
  • ai is next empowering wave enhancing creative product and allowing more in all areas

conversing with history

• “good history and civic teachers make past interesting, great history and civic teachers make past come alive”
• can talk with any historical figure, personification of 14th amendment,
  • can ask Washington about reconciling slavery and enlightenment
  • limitations
    • expert on Tubman asks AI Tubman questions
      • on quotes that were never confirmed and says they were just attributed to her
      • won’t comment on critical race theory as an AI simulation
      • Ask about reparations and won’t comment, but something like it was talked about even though word wasn’t
• Simulations new ending way to interact doesn’t take away old

Part three - powering the next innovators

using science to study science

• successfully says climate change is real and while it mentions opposition it says it didn’t make sense
• Khanmigo blocks url generation cuz random
• can ask about ozempic and it asks you questions, can generate

1+ 1 = closing the math gap

• gpt4 gets close to correct math
  • thinks behind the scenes for how it would do the problem
  • Tailored explanations and problems
• “is Khanmigo a replacement for a teacher? No and it never will be”

accessing courses that otherwise students would not

• tutor needs course progression which khan Academy already had
• 50% of high schools don’t offer calc, 40% don’t offer physics
  • less resources more not enough students ready
• minority of students have enough motivation, school house works offers free online tutoring
• cal tech announced khan Academy courses meet requirements
• Advisors that keep people accountable
• ai Tutor feedback was it worked for students with motivation to seek help, so they added check ins and emails for goals (lol like that’s the same thing as real tutor)
• Hobert Indians just for khanmigo, biggest improvement was in self confidence
  • increased student engagement cuz instead of raising hand ask ai
  • Ai breaks down barriers between fields integrating art podcasting and more

part 4 better together

• “to know oneself is to study oneself in action with another person” - Bruce Lee ^800971

bolstering collaborative learning

• does ai means more tech isolation?
  • collaborative teams people learn better and learn to communicate
  • Thoughtful use of tech can increase interaction
    • khan Academy encourages students interact with each other in class and online
  • khanmigo can do icebreakers, break outs and lead discussion
    • teacher can understand this interaction
• Khanmigo can organize splitting work of project, can invite multiple students to play learning game
  • “you are always gonna get better collaboration if you have a great facilitator”

ai meets student mental health coaching

• khanmigo can be emotional life line when you are stressed about test
  • Let’s do some breathing exercises, is failing realistic?
• eliza ai talking would use Rogerian technique, just rephrased statements
• therapy chat bot doing cbt could improve direction and anxiety in months (months?)
  • truly interactive interventions that highly educated psychologists could do
• AI assumed to be good with iq, not eq but maybe we should say AE, artificial emotion or something

the place for parents in ai based education

• historically parents have limited choices to help their kids looking like tutoring, now have ai
• when parent tutor kid can create tension, but value in non parent third party like ai
  • Ai tutor has endless energy
  • can give advice and parenting coach for social issues

increasing points of connection between parents and kids

• tech can be used to amplify intent
• “Life satisfaction comes from establishing goals, working hard, and feeling like you are moving world towards better place”
• llms can focus learning time to allow more time and ai can facilitate and do group learning and games to create lasting memories

part 5 keeping kids safe

• “distrust and caution are parents of security” Ben frank

delivering the facts: The state of bias and misinformation

• isn’t ai trained on bias so it’s bias?
• prev state is algs triggering us especially young people
  • search is now worse too with seo profit crowding out Wikipedia and Smithsonian
  • resume lean on superficial bias
• Gen ai can be wrong, but is it more or less than human
  • ai can be audited unlike human recruiters and do checks or surface overlooked candidates
    • We don’t really have bias on human evaluators
      • Best is ai is extra check on bias
• khanmigo won’t even debate in good fun that earth is flat, “no credible argument to suggest earth is flat”

what about data collection

• best practice have models never train on pii especially for children
• khanmigo personalized and gives option to edit or reset info

AI and gift of transparency

• ai tutor can help parents and teachers understand student journey, quizzes, and confusion and even suggest what to do with them
• how healthy is it to know everything?
  • If children feel like they have no privacy might never ask questions

ai as guardian Angel

• ai tutor could sit and make articles correct grade level, censor, or provide more info
• ai oversight of browsing
  • Could give YouTube time as reward

part 6 teaching in age of ai

• Alexander the Great “I am indebted to my father for living, but my teacher for living well” ^ec0b53

how ai will supercharged teachers and teaching

• 2017 Horace educator made prediction that by 2027 teachers will be ai and human will be classroom assistant while ai conveyer of knowledge
  • Author thinks it will be the opposite
• some imagine Students can get answers without interacting with person, so teachers become less important
• “there is no job that is safer in LLm world than teaching”
  • Llms will let teachers do more of what they enjoy, connection and developing lessons
• ai will be teaching assistant! (No arguments for this)
• Wharton says teachers need to do the things
  • Expect more from students
  • Further integrate ai to review and critique work, premorteum
  • flipping classroom to do Socratic and group work in classroom

Dawn of ai teacher assistant

• teacher shortage and highest rate of burnout cuz little support
  • Average teacher works 54 hours per week
    • Only half with students
  • Average tenure is 5 years
  • Quarter of teachers thinking of leaving
• Ai teacher assistant can hold people accountable, and do grunt work rubric, progress reports, grading
• Ai can help make lesson plans, activities, and be counselor for teacher
• Differentiation and active learning is the dream
• ai can eventually facilitate breakouts
• student’s want human to human connection, “remove the teacher and you remove the critical element foundational to all learning”
  • “teachers will always be in command of their classes and thank goodness for that”

Helping Build Alternative Education Models

• home school brings worries of socialization and teaching skills
  • Khan got lots of material and tutoring which is like socialization
  • Lots of big name schools like Yale and cal tech can get credit for college admissions with schoolhouse peer assessments of doing khan Academy well
    • Cal tech even gives credit for courses in their admission requirements
• Homeschool or pod schools students use khanmigo for things their parents don’t have expertise in
  • better parent reporting and help
• Different schools for different people

fixing cheating in college

• honor code for cheating already not respected in many colleges
• According to 2023 middle berry college, 2/3 say they broke honor code, 32% cheated on test, 15% had unauthorized use of Ai tools
  • Stanford professors used to not be allowed to proctor exams, reversed now
  • Environment where it feels like everyone is cheating and you will be left behind if you don’t
• GenAI spotlighting issues that has existed
• Solutions
  • Can have in class writing, reverse classroom
  • show more intermediate steps
  • Khanmigo can watch student work and riff with student
    • Can even help teachers with rubric, and then give students estimated grade
    • Imagine if you didn’t know if you made the shot in basketball until a week later, now feedback on every dimension of writing and work
• ai can actually solve long standing cheating issue, give better learning experience, and prepare students for AI world

Part 7: the global classroom

the global classroom

• deep global disparity
  • girls in Indian twice as likely to never set foot in class
  • only half of teachers in Indian teach and the ones who do are not well trained
  • us children in poverty enter school 18 months behind developmentally
• technology needs to be equitable, it can’t leave people behind
  • Our team operates on budget of some high schools but benefits 100 million students a year
• half population have smartphone
• “technology augments the motivation that a relationship with an adult gives them”
  • all evidence says need personal connection and adult to support and celebrate them to stay motivated

economics of ai in education

• Per student per year, Louisiana pays 10k, nyc pays 40k, india pays 500-1200
• In all, lockstep through curriculum with limited support for personalization and filling gaps
  • 6% Detroit operated at grade level before 2020 after pandemic dropped to 3%
  • average classroom has 3 grade levels of ability Abe after pandemic average was across 6 grade levels
• Money poured into tutoring did little
• khanmigo for free prevented pandemic slide and actually improved on pre pandemic
• 70M budget of khan Academy
  • computation of gpt4 for khanmigo is 5-15/month per student
    • vs 30/hour tutoring it’s great
    • Should become cheaper by factor of 10 soon
    • 100 factor decrease in next 10 years means can be for all

part 8 - ai assessments and admissions

the future of k12 assessments

• standardized tests bashed
  • Narrows focus to multiple choice that teachers then narrow too
• But assessments need to measure
  • Is it actually equitable to remove it and have no data
  • shouldn’t we just expand the data and make data more actionable
  • Continuous standardized assessment on khan academy is the way, not one big test
• ai means you can now do rich assessment like projects and dictation and spoken tests that would require so much human work before
• compare to status quo which had more bias, ai will be biased but can mitigate risks and allow broader tests of communication creativity so we can then focus on developing the whole person

the ai of college admissions

• classic college admission is grades, standardized tests, essays, extra curricular, and letters of rec
  • All will be changed with ai
• college coaches go from feedback to writing essay, elite use insider knowledge of admissions so now AI can let all play in this gray area
• india entrance exam is based solely on test scores
• but what if we could test soft skills in standardized way
• school house world has tutors, so Chicago and mit and 18 more uses tutoring transcript of knowledge of topic they teach and impact on kids
  • Ai gives tutoring tips
• khanmigo could write standardized letter of rec for students, imagine if everyone had same teacher
• biasfree is impossible, but better than status quo is possible

part 9 work and what comes next

employment in an ai world

• ibm slowed back office hiring by 30% in anticipation of AI automation
• “You won’t get replaced by ai, but people using ai” some say
• engineers between 5-10x more productive more in demand
  • Early 2000 software outsourced and author thought he needed to switch into business, but instead software salaries exploded
  • Not bullish for summarizing
• Chegg destroyed
• Job least affected by ai is roofing, but ai will change proposal

how to prepare kids to thrive

• need to develop skills to know what good is, but can take much more complex projects
• need entrepreneurial expertise in all domains not just specialization
  • Put tools together to create value
  • more executive decisions

matchmaking between job seekers and employers

• ai will post, apply, and acquire jobs for us
• Right now job descriptions, and reviewing resumes for status
  • interviewers have personal bias and mood add takes a lot of time
  • every job candidate can have conversation with recruiter bot or even your agent bot
  • recruiter can engage with references, help and monitor interviews
• less bias is future

where this leaves us and where it will take us - a call for educated bravery

• his father prob depressed
• He went to MIT and had high status professors, went to analysts as hedge fund which he liked about learning about world
• he always wanted to start a school
• education and healthcare where profit not same as value
• Like foundations for large time scales thinking
  • Culture and know how is where best are
  • Khan Academy could be institution’s built on internet to educate billions for centuries to come
• every level of labor will be disrupted from bottom to top, need to let everyone work at the top with ai enhancing skills
• people with time but no purpose or meaning susceptible to populism,
• ai can soon create internet surveillance, fraud and misinformation, and war
• but bad actors won’t slow down, but it’s a race
  • rule breakers and totalitarian won’t stop
  • “This is not a drill, ai tsunami is here too stay”

Review

This is basically an ad for Khanmigo—more a vehicle of change convincing educators than a source of truth. But it is probably the most comprehensive vision and discussion of AI's impact on education, so still worth the short read. Khan paints a glorious future where Khanmigo is always there to help students, teachers, and parents, redefining the way we do everything from testing to recruiting. I do appreciate his underlying optimism for the future and Khanmigo's team carefully considering potential risks. But he so adamantly asserts teachers will not be replaced that it feels like he is just trying to soothe his future users. In one paragraph, he insists teachers will never be replaced, for they motivate learning with a personal connection. But in another, he praises AI's ability to do mental health coaching, noting we should start talking about AE (Artificial Emotion). It seems like AE can progress to form this personal connection, especially since children anthropomorphize everything already. This seems to me to be the biggest risk with AI in education—replacing children's real social connection and skills for the sake of knowledge in an AI-driven world where the opposite trade makes sense.

Key Quotes:
“This is not a drill, the AI tsunami is here to stay.”
“Is Khanmigo a replacement for a teacher? No, and it never will be.” “Teachers will always be in command of their classes and thank goodness for that.” “Technology augments the motivation that a relationship with an adult gives them.” “There is no job that is safer in LLM world than teaching.”

================== This is basically an ad for Khanmigo. More a vehicle of change convincing educators then a source of truth. Khan asserts in multiple places teachers will not be replaced. He does not bother supporting this. Claim. Probably imagining his listener will be very sympathetic. On the one hand he says teachers will never be replaced. In another paragraph he talks about how AI has unique EQ to relate and form relationships with people. It's unclear to me how that will not serve as a replacement for the motivation that you create. But it is probably the most comprehensive vision and discussion of AI's impact on education, so still worth the short read. Khan paints the glorious future where Khanmigo is always there to help. It does He even covers how AI will affect recruiting and job search, parents, jobs. But he also has this underlying optimism that almost feels like an ad for Khanmigo. In some places, I just disagree. He seems extremely adamant that teachers will not be replaced—almost too adamant. It feels like something you need to tell teachers to have them use AI. So really, this book is just a vehicle of change rather than a source of truth.

This book is an ad.

Good book, Relativity Quick Read, Sam Khan. Covers the main use cases in education, going from AI teacher and recruiting to job search. But he also just has this underlying optimism. It almost feels like an ad for Ken Migo. In some places, I just disagree. He seems extremely adamant that teachers will not be replaced. It's almost too adamant. It feels like it's something you need to tell teachers to have them use AI. So really, this book is just a vehicle of change rather than a source of truth.

they heard x complete so they thought about and implemented why did y solve x I don't know

Review: pessimistic take, what a long as for khanmigo. It has upside and solves downsides. No teachers will NEVEr be replaced and thank god.

Optimistic take: This must be done and it’s nice he addresses potential containment and argues the steps he takes against them. Ai is the future with real upside and can’t be ignored

I appreciate Sam s optimism and hope for the future, less bias and good outcomes

3/5 Cuz I disagree and believe he is circumnavigating the truth and to a certain about the future to certain that teachers will not be replaced because that's what you need to tell them to use your product. probably the best source of information for AI in education, but there isnt many.

3/5 Im skeptical of his optimism, but appreciate it.

Key Quotes “This is not a drill, ai tsunami is here too stay” “is Khanmigo a replacement for a teacher? No and it never will be” “teachers will always be in command of their classes and thank goodness for that” “technology augments the motivation that a relationship with an adult gives them” “there is no job that is safer in LLM world than teaching”