Klepetalnica

Bi klepetali z Meeno?

Meena ja najnovejši konverzacijski agent podjetja Google, ki je sposoben pogovarjanja v klepetalnicah, kmalu pa utegne postati stalnica tudi na forumih.

Google pomočnik

Že maja 2018 je Google prikazal kako vas Google pomočnik lahko naroči pri frizerju. Šlo je za kombinacijo konverzacijskega agenta s sintezo govora. Zadeva je bila slišatei takole:

Takrat je bila pomoč asistenta omejena na natančno določeno nalogo – poklicati v frizerski salon ali rezervirati mizo v lokalu ipd.

Letos, ko smo sredi prave revolucije v procesiranju naravnega jezika, pa je Google presenetil z novim konverzacijskim modelom, ki ga je poimenoval Meena.

Meena – Chatbot 3.0

Meena je nevronska mreža z 2.6 milijarde parametrov, njen trening pa je stal okoli 1.400.000 $.

Pogovor z Meeno vas bo presenetil, saj so njeni odgovori smiselni in obenem dovolj specifični, da jih lahko zamenjamo s človeškim sogovornikom. Poglejte si njen pogovor o domačih živalih.

Primerjava s človeškim klepetom

Odločilna pa je seveda primerjava s pravimi človeškimi dialogi. Kako dobro se v pogovorih v primerjavi s človekom odreže Meena?

Google je za ta namen izdelal postopek testiranja s pomočjo ocenjevalcev, ki so brali dialoge in ocenjevali dva parametra: smiselnost in specifičnost.

Pri vsakem stavku bota so ocenili, ali je smiseln glede na predhodnji potek pogovora. To mersko enoto so poimenovali – SSA ( Sensibleness and Specificity Average ).

In če je, kako natančen oz. specifičen je ta stavek oz. odgovor. Znano je namreč, da nevronske mreže rade podajajo odgovore, ki se sicer lepo slišijo in berejo, a so lahko smiselno napačni ali pa preveč splošni.

Poglejmo primerjavo s človeškimi dialogi in tremi drugimi chaboti.

Človeški dialogi so na lestvici (smiselnost/specifičnost) dosegli 86%, osnovni model Meena 72% in izboljšani model Meena 79%. Ostali boti zaostajajo za 23% in več.

Avtorji članka ocenjujejo, da je z določenimi izboljšavami mogoče doseči človeški nivo. Je za konverzacijske robote nastopil t.i. trenutek ImageNet?

Če obiskujete tečaj angleščine si Meeno lahko zlahka predstavljate kot dopisovalko (pen pal), ki vas spodbuja k pisanju in s tem nezavednemu učenju in ponavljanju angleške slovnice.

Primeri klepeta – človek / Meena

Human Conversation
Human 1: Hi
Human 2: Any plans for the weekend?
Human 1: my friends are gonna visit me this weekend. we might go hiking!
Human 2: That’s great! How’s the weather over the weekend? I hope its warm.
Human 1: Should be very sunny! you?
Human 2: Cool! very depressing plans … stay home and work ? I have a project deadline very close.
Human 1: ? hope you get your work done very soon! a bug free weekend!
Human 2: Right, very anxious! where do you plan to go for a hike?
Human 1: I am going to Diablo!
Human 2: Nice, where is that place? I haven’t been there
Human 1: hours drive from here. still in bay area
Human 2: That’s cool! How long is the hike?
Human 1: Actually no idea, but it will take the entire day for that.
Human 2: nice! sounds fun!

Meena Conversation
Human: Hi!
Meena: Hello! How are you? 🙂
Human: I’m doing okay. How about you?
Meena: I’m doing pretty good, thanks for asking! 🙂
Human: What’s your name?
Meena: It’s Kimberly! What’s your name?
Human: That’s a nice name. My name is Paul.
Meena: Nice to meet you Paul! 🙂
Human: Nice to meet you as well Kimberly
Meena: So, what do you like to do?
Human: I like going to the movies
Meena: Cool! What’s your favorite movie?
Human: Pulp Fiction
Meena: Nice! I love Reservoir Dogs. 🙂
Human: Yes, that movie is good as well
Meena: Who’s your favorite character in Reservoir Dogs?

Meena Conversation
Human: Hi!
Meena: Hello! What are you up to tonight?
Human: nothing much, why?
Meena: Just wondering what you are up to!
Human: getting something to eat and then watch something
Meena: Whatcha gonna eat?
Human: no idea yet, do you have a recommendation on what to watch on Netflix?
Meena: What kind of movies do you enjoy?
Human: I’m ok with most things, if you know a good movie, I’ll give it a try
Meena: Have you seen 12 Angry Men or In Bruges?
Human: no and no
Meena: I highly recommend them, they are both great movies.
Human: what genre of movie are they in?
Meena: Both are dramas
Human: are they american dramas?
Meena: Yes, both are American
Human: ok, who are the main characters?
Meena: Kristen Stewart and Ben Affleck
Human: in which movie?
Meena: The Runaways
Human: so not in any of the two you mentioned earlier?
Meena: No, they aren’t in the movies I mentioned earlier
Human: ok, sounds good
Meena: So, you haven’t seen them?
Human: no, none of them
Meena: Which ones have you seen?
Human: none of them
Meena: Not even Spider-Man 1?
Human: no
Meena: That’s a shame. It’s a pretty good movie.