Students Essays Graded by Computers

[Jsmith]

https://www.npr.org/2018/06/30/624373367/more-states-opting-to-robo-grade-student-essays-by-computer

 

This is terrifying.

I feel a full implementation of this right now would push the dividing line between creative little humans and those who are good at being in a hierarchy of following instructions too drastically, too quickly. The level of technology that is required to implement machines grading creativity is far from being within reach. It's like saying we already have general AI.

[Drak3]

Can't be worse than a US college professor. Or half of our teachers in K-12 schools.

[Jsmith]

3 minutes ago, Drak3 said:

Can't be worse than a US college professor. Or half of our teachers in K-12 schools.

Nobody should jump to something new if they cant figure out how to fix what they already have.

[captain_to_fire]

7 minutes ago, VegetableStu said:

wonder how long would the AI take for it to find out they're being messed with

 

nevermind the human graders

Teacher: Write a 500 word reflection essay about George Orwell's 1984 and To Kill A Mocking Bird excluding articles "a" and "the" but write it in a scantron paper and use #2 pencils only.

 

AI: Plagiarism detected. I recommend failing this lazy ass student. 

 

Edit: @Jsmith you need to follow posting guidelines or else the mods will move this somewhere or lock it down

 

[Dreamplay]

5 minutes ago, VegetableStu said:

wonder how long would the AI take for it to find out they're being messed with

 

nevermind the human graders

I don't know why but I started to imagine an AI getting graded by an AI in school, only to have that AI get spanked(remove 2 cores for 2 hours) by their parent AIs for not doing good at computing...

[S w a t s o n]

I wrote a computer graded essay to opt out of english in college, I got a perfect mark so I can't complain?

[Christophe Corazza]

An algorithm does not read. It looks for things that some human has decided correlate, on average, with "good" writing. Someone told it to count words, because in general better essays have more words. Same for other charactics it looks for. Couple that with writing methods that teach those things. It's all bogus.
But doesn't anyone remember the urban legend of the professors who threw essays up the stairs? this is the same thing.

 

Just what we need. More tests that sort for gaming the system, rather than sorting for actual skills.

[WereCatf]

3 minutes ago, Christophe Corazza said:

An algorithm does not read. It looks for things that some human has decided correlate, on average, with "good" writing. Someone told it to count words, because in general better essays have more words. Same for other charactics it looks for. Couple that with writing methods that teach those things. It's all bogus.
But doesn't anyone remember the urban legend of the professors who threw essays up the stairs? this is the same thing.

 

Just what we need. More tests that sort for gaming the system, rather than sorting for actual skills.

If it's all fully-automated and no teacher at any point actually reads the essays, it just becomes an exercise in how to cram as many different words from the dictionary into the essay while making the sentences look somewhat intelligible. The actual content can be complete gobbledygook. What's even worse is that it's really easy to write essay-writing bots that write just such gobbledygook for you!

[S w a t s o n]

7 minutes ago, Christophe Corazza said:

An algorithm does not read. It looks for things that some human has decided correlate, on average, with "good" writing. Someone told it to count words, because in general better essays have more words. Same for other charactics it looks for. Couple that with writing methods that teach those things. It's all bogus.
But doesn't anyone remember the urban legend of the professors who threw essays up the stairs? this is the same thing.

 

Just what we need. More tests that sort for gaming the system, rather than sorting for actual skills.

Why do you feel that way. At least for pure language classes I think it makes sense. You can tell a computer what correct grammar, conjugation, sentence structures and other factors look like. The actual content doesn't really matter too much.

 

Quote

"We have artificial intelligence techniques which can judge anywhere from 50 to 100 features," Foltz says. That includes not only basics like spelling and grammar, but also whether a student is on topic, the coherence or the flow of an argument, and the complexity of word choice and sentence structure. "We've done a number of studies to show that the scoring can be highly accurate," he says.

 

[WereCatf]

2 minutes ago, Swatson said:

"We have artificial intelligence techniques which can judge anywhere from 50 to 100 features," Foltz says. That includes not only basics like spelling and grammar, but also whether a student is on topic, the coherence or the flow of an argument, and the complexity of word choice and sentence structure. "We've done a number of studies to show that the scoring can be highly accurate," he says.

Yeah, that's what they claim, but it's bullshit. Even Google's voice-assistant still struggles to follow a three-sentence "conversation", and Google has a LOT more money and hardware to throw at researching such things than some education-market company.

[Ithanul]

Nothing really new.

My college English classes had such software in use.  Main reason I did not bother even pushing myself, since they used software to grade, hence I used software to spit the essay out.  Plus, felt like doing High School English all over again...

[Jsmith]

1 minute ago, Ithanul said:

Nothing really new.

My college English classes had such software in use.  Main reason I did not bother even pushing myself, since they used software to grade, hence I used software to spit the essay out.  Plus, felt like doing High School English all over again...

You.

[S w a t s o n]

3 hours ago, WereCatf said:

Yeah, that's what they claim, but it's bullshit. Even Google's voice-assistant still struggles to follow a three-sentence "conversation", and Google has a LOT more money and hardware to throw at researching such things than some education-market company.

voice recognition and text based language comprehension are NOT the same thing, not even close.

[Andreas Lilja]

Poli. Sci, would rather have had computers rate my work than professors who grade you unfairly because of their ideological biases. By the second year I gave up and started writing stuff they'd love outright.

[WereCatf]

26 minutes ago, Swatson said:

voice recognition and text based language comprehension are NOT the same thing, not even close.

You do realize that it still has to convert the speech into text and then try to understand it?

[S w a t s o n]

6 minutes ago, WereCatf said:

You do realize that it still has to convert the speech into text and then try to understand it?

voice recognition works by translating sound waves into words, that's the main problem surrounding it's comprehension.

 

if you're in a relatively quiet place and have a relatively modern smart phone, google will understand you almost 100% of the time.

 

Besides this, the computer grade is given based on correct use of the english language. Voice recognition needs to understand the content and execute the command AFTER the whole process of translating sound waves to words, a grammar marking bot does not.

 

Grammar bot cares if you use a comma properly, for google assistant, the comma may as well not exist

[WereCatf]

10 minutes ago, Swatson said:

voice recognition works by translating sound waves into words, that's the main problem surrounding it's comprehension.

 

if you're in a relatively quiet place and have a relatively modern smart phone, google will understand you almost 100% of the time.

 

Besides this, the computer grade is given based on correct use of the english language. Voice recognition needs to understand the content and execute the command AFTER the whole process of translating sound waves to words, a grammar marking bot does not.

 

Grammar bot cares if you use a comma properly, for google assistant, the comma may as well not exist

The claim is "whether a student is on topic, the coherence or the flow of an argument" -- that's a lot more than just knowing where commas and periods go.

[S w a t s o n]

37 minutes ago, WereCatf said:

The claim is "whether a student is on topic, the coherence or the flow of an argument" -- that's a lot more than just knowing where commas and periods go.

That's not the only claim, but yes there would be some comprehension of the topic, the point is that it's not translating sound waves from people talking in vernacular and trying to understand that. It's marking you on your use of the language. If you write an incoherent sentence that includes the topic keyword or the name of the person it's not going to be fooled is the main point.

[wasab]

Let machine replace all human labors, even creative thinking and innovation, then we can have a truly scarceless economy that is able to satiated endless wants. We will then have achieve the long term goal of perfect equality and a classless society. Futuristic communism for the masses! 

[Jsmith]

Can we talk somewhere about general AI, and how un-ready the world would be for such a scientific conglomerate?

[Teddy07]

I don´t have a problem with that as long as it works properly.

[GoodBytes]

I think it is a great idea! And I think Skillshare can replace these lazy ass teacher of today who uses this software.

They are many great teachers, but sadly, they are lazy ones, I am targeting those ones.

[Trik'Stari]

8 hours ago, Drak3 said:

Can't be worse than a US college professor. Or half of our teachers in K-12 schools.

I could not agree more.

[Jsmith]

There's a lot that needs to fall into place for this to be done right.  But if it happens, It could be revolutionary. But it still won't solve the "lazy teacher" problem, only help it really.

[Guest]

I use Siri to look for key words in the assignments that I mark. Works great.