Hey everyone, I’m looking for a way to use an open source local large language model (LLM) on Linux, particularly on low-spec hardware like Raspberry Pi, to generate lengthy, coherent stories of 10k+ words from a single prompt. I recall reading about methods described in scientific papers such as “Re3: Generating Longer Stories With Recursive Reprompting and Revision”, announced in this Twitter thread from October 2022 and “DOC: Improving Long Story Coherence With Detailed Outline Control”, announced in this Twitter thread from December 2022. These papers used GPT-3, and since it’s been a while since then, I was hoping there might be something similar made using only open source tools. Does anyone have experience with this or know of any resources that could help me achieve long, coherent story generation with an open source LLM? Any advice or pointers would be greatly appreciated. Thank you!
More generically, this sounds like “Let’s not put any effort into anything: the machine will do it for me.”
That is my opinion and I’m aware it’s a minority point of view these days. That’s why I fully expect to be modded down immediately after posting this. Because arguing with people who have a different opinion is also not in fashion anymore.
So you are not using a calculator, I presume? Only math done on abacus is not being lazy?
There is a conveniently omitted difference here. When doing learning exercises, the result is almost irrelevant. The way towards said result is what matters. So if you put a function into a calculator and learn that the result is X1/2=3π², you have really gained nothing. What about text creation have you learned when you let an AI spew out some text “from a single prompt”?
In many cases I don’t want nor need to learn that. I just need volume about the key points
When it makes sense I do. More often than not, for basic arithmetics in daily life, and back of the hand engineering calculations, I use my noggin because it’s quicker.
Why an LLM is any different?
Let’s say I want my RPG players to find a corporate mail that gives them some plot info. Why not ask an LLM to write the boilerplate around the info I want to give them? Just as example
Agreed. All I said was, the way the OP framed his question sounds like they’re trying to weasel out of putting some effort into something that sounds worth putting some effort into.
It’s not like they said “I want to draft boilerplate legalese that I can go back and adapt to each customer.” He said “I want to use an LLM to create full-blow 10,000 word long, coherent stories.”
But that depends what do they need it for
Personally I don’t see a difference between legalese boilerplate and 10k word story. But that discussion might lead us nowhere
The difference is between spewing out functional text for the purpose of covering your ass in court and human creativity.
I have no problem with people using AI to generate Excel sheets, meeting records, summaries of articles… I really, REALLY have a problem with people who ditch what makes us human to the wayside and delegate what is ours and ours alone to do to the machine out of laziness.
But machine will not do the creative part. It can only fill in the time-sinks around our creative ideas. Ask an LLM to tell you a joke no-one has ever heard before and then google it. The creative part still has to come from humans
EDIT; and the truth is that we very rarely come up with something creative. We mostly just recompile previously met combinations
I keep a bookmark to this AI detector on my browser bar https://gptzero.me/
I just do random checks for AI generated stuff, especially if it’s a news site I’ve not seen before. There are use cases for AI in text creation but I think it’s very limited - a summary production (with references), for example. Instead I am exited for medical advances, space study, cases where there are huge datasets of good quality and not datasets that are the result of huge dragnets of shite from the internet.
People are quite right to be cynical about AI created stuff; we’re near the zenith of a hype-cycle at the moment.
You’ll be getting no down vote from me.
Thank you for that link. I needed that!