Like if I type “I have two appl…” for example, often it will suggest “apple” singular instead of plural. Just a small example, but it is really bad at predicting which variant of a word should come after the previous
Succint
Phones don’t use LLM for predictive text. The algos are a lot less complex on phones.
iOS 17 uses a small gpt-2 based model for predictive text.
Hm, that’s interesting 👍.
I guess, the real question is: Could we be using (simplistic) LLMs on a phone for predictive text?
There’s some LLMs that can be run offline and which maybe wouldn’t use enormous amounts of battery. But I don’t know how good the quality of those is…
You can run an LLM on a phone (tried it myself once, with llama.cpp), but even on the simplest model I could find it was doing maybe one word every few seconds while using up 100% of the CPU. The quality is terrible, and your battery wouldn’t last an hour.
Does the AI processing have to be performed locally or constantly active?
No, but you open up a can of worms from a security aspect if you send it out to be processed.
I’m sure every phone having a keylogger won’t end badly
And latency.
deleted by creator
A pre trained model isn’t going to learn how you type the more you use it. Though with Microsoft owning SwiftKey, I imagine they will try it soon
I was so heartbroken when I found out that Microsoft purchased Swiftkey. It was my favorite. Is there any way to still use it without Microsoft involved? Lawdhammercy
I think apple has pitched this for a future iPhone, yes.
They’ll probably have to offload that to a server farm in real time. That’s not gonna be easy.
Openhermes 2.5 Mistral 7b competes with LLMs that require 10x the resources. You could try it out on your phone.
The algorithms are the same. The models are different, being trained on a smaller data set.
Perhaps, I’m not a dev, especially not an iOS or an Android one.
Now guess how it feels to type German with predictive text. Most of our words can have more than a dozen different word endings depending on time and how the word is used. And that’s not taking into account that we use compound words, which word prediction pretty much cannot predict and often doesn’t even know. So spell check will mark a legal compound word as misspelled, because it doesn’t understand the concept of compound words and doesn’t know this specific word combination.
To show what I mean, the term “Danube steam boat captain’s hat” becomes “DonauDampfSchiffKapitänsMütze” (I added capital letters which shouldn’t be there to show where the next word in the compound word begins).
While this is an extreme example, it’s pretty common for compound words to consist of 4-5 words.
And for some reason, some cases seem to be missing completely on my Android default keyboard. “untersuchst”, just like a bunch of second person cases for slightly unusual words is non existent.
Yeah, noticed that too. This is really annoying.
My favourite: ‘geröntgt’ which is the second participle of ‘röntgen’ to x-ray someone. Never heard it pronounced correctly by a native speaker.
Dutch also has the issue with the compound words. Autocorrect will often put a space in there, which is grammatically incorrect (and ugly). I feel like it’s at a point now where the incorrect space usage has become mainstream and might change the language rules. Oh well.
@[email protected] autocorrect doesn’t use LLMs, at least not yet generally. I suspect it is some kind of markov model or maybe?
LLMs like chatgpt take a wild amount of resources to run.
If you want something as smart as gpt3 and you want it to run at typing speeds, you’ll need a gaming PC running it.
People just recently managed to run gpt3 strength models at all on ordinary laptop hardware (slowly).
There is currently no way to run something gpt4 strength on ordinary consumer hardware (I’m just guessing but I think it takes a few hundred gb of VRAM to run)
AI is a vast field. LLMs and neural networks are a small part of it.
LLMs are very expensive to run and a lot more complex than the markov chains often used for predictive text.
Predictive text just chooses a likely word based on what’s typed. This may be as simple as looking for words that start with what you’ve typed.
LLMs vectorise words and understand the complex relationship between vectors using many data points. So it would spot the word “two” and realise that plurals are used with it.
Predictive text also can vectorize words, but the number of vectors per word are much, much simpler.
Because they’re using different tech. That’s like asking why do phone calls sound bad compared to voip calls. They’re just using different tech.
If humans are just brains why are we smarter than dogs who also have brains?
