This concerns me:

stories of 100,000+ words from a single prompt

An LLM excels at making passable derivative work. It does not, by definition, come up with original ideas.

What are you going to do with 100,000+ words of 100% derivative writing where anything potentially original can be summed up in a prompt of a few dozen words?

Will this be published or sold somewhere? Undercutting or crowding out original works?

You need to use an LLM with a very long context length, potentially 1 million+ tokens. I don’t know if any local LLMs can even go that far, and if they can, you’ll need an outrageous amount of ram and vram.

But honest question… Why? If you’re planning on generating fake books or stories, it’s not going to happen, you’ll create the most generic barely coherent text.

And fair warning, if you’re trying to sell AI generated stories you’ll quickly be permabanned from any store, so don’t even try it.

LLM generations of that length tend to go off the rails - I think generating it in chunks where you can try and guide the model back onto the rails it probably a more sane technique.

There are several open source llms to lean on - but for long generations you’ll need a lot of memory if you’re running it locally.

Thw 100k word part is relatively easy.

The coherent story part is not possible with today’s LLMs, even with a much smaller word count.

Hell, lots of human writers fail at making their stories coherent.

my guy you gonna plagiarize a whole-ass book using an llm for something THAT big

Don’t.

Develop real AI, don’t rely on bullshit LLMs.

I am confused as to why you’re going through all this struggle! You’ll get the same results just copy-pasting big chunks of other books that humans have already put time and effort into writing :) best of luck!

Creating a 100,000-word coherent story using an LLM with a limited context window requires strategic planning in how you manage the narrative flow, continuity, and character development over multiple sessions. Here’s a strategy tailored for this scenario:

1. Detailed Plot Outline:

   Expand the Outline: Break down the story into smaller, manageable arcs or segments (e.g., each act could be split into several chapters). Each segment should have its own mini-outline: Major plot points Character development for that segment Setting changes Key interactions or conflicts Micro-Outline for Each Chapter: For each chapter within these arcs: Opening scenario Middle conflict Resolution or cliffhanger Character arcs within the chapter

2. Session Management:

   Context Management: Due to the limited context window, you’ll need to manage how much information is retained from session to session: Summarize Previous Content: Before each new prompt, provide a concise summary of the previous narrative sections. This summary should include: Key events Current state of characters Unresolved conflicts or mysteries Setting and time Prompt Structure: Start with a Summary: Begin each prompt with a summary:

    Previous chapter summary: [insert summary here]. Now, write the next chapter where [describe the key elements from the micro-outline].
   
    Specify Tone and Style: If the story has a specific tone or narrative style, remind the LLM of this:
   
    Maintain the [tone/style] from previous chapters. 
   

   Length of Each Segment: Estimate how many words you can comfortably fit into one session. If your LLM can handle around 2,000 tokens (which could be around 1,500 words, depending on the model), you might aim for each session to produce a chapter of 1,500 words.

3. Continuity and Cohesion:

   Character Consistency: Keep a running document of character details, relationships, and developments outside the LLM context. Use this to ensure consistency: Character sheets Timeline of events Plot Devices: Use recurring elements or plot devices to maintain cohesion: Recurring themes Foreshadowing elements from earlier segments Feedback Loop: After each session, review the output for: Continuity errors Character voice consistency Plot holes

   Use this feedback to adjust your next prompts or summaries to address any discrepancies.

4. Incremental Development:

   Iterative Refinement: As you generate content, refine your prompts based on what works better to certain styles of prompts or require more detailed instructions.

   Draft and Revise: Treat each segment as a draft. After generating a section, you might need to: Revise for coherence with the previous content. Adjust for pacing to keep the narrative engaging without overwhelming the reader. Enhance character development if it feels lacking. Dynamic Outlining: Be prepared to adapt your outline as the story progresses. Sometimes, the LLM might produce content that suggests new directions or deepens certain aspects of the plot or characters in ways you hadn’t initially planned.

5. Technical Considerations:

   Token Management: Since LLMs count tokens rather than words, be aware of how much each prompt and response consumes. Words with multiple tokens (like proper nouns, rare words) can quickly fill up your context window. Prompt Efficiency: Keep prompts concise but informative. Avoid redundant information in prompts to maximize the space for story development: Use bullet points or lists for summaries when possible. Focus on key points rather than detailed narratives in your instructions.

6. Final Assembly and Editing:

   Compilation: After generating all segments, compile them into a single document. Here, you’ll have a larger context to: Check for continuity Ensure narrative flow Address any plot holes or character inconsistencies Editing: The final step involves editing for: Grammar and style Pacing and tension Theme consistency External Tools: Consider using external writing tools or collaborators for a final polish, especially if the LLM has limitations in areas like nuanced character development or complex plot twists.

7. Iterative Feedback:

   Review and Adjust: If possible, after a few segments, review the overall narrative to see if adjustments are needed in how you’re prompting the LLM. This could mean changing how you summarize past events or specifying more about character motivations and interactions.

By employing this strategy, you can leverage the capabilities of an LLM to create an expansive, coherent narrative even with the limitations of context window size. Remember, this process is iterative and might require several attempts to get the balance right between creative generation and maintaining narrative integrity.