"Smart" Nested Folders
in progress
Jordan "Jado" Finnigan
In addition to the feature defined in the Multi-Level Folders feature request (i.e., the one that prompted me to come make this request), I would be curious to hear others' thoughts on having an (optional) inherited system instructions feature. There could be a Global System Instruction, and then folder-specific instructions.
For example, the
Coding
folder has its own instructions, Go
underneath has further instructions, and the conversation within Go about Gin has instructions (which exists now). The System Instructions in the Gin conversation would be updated to be evaluated as follows:{{ Global Instructions }}
{{ Coding Folder's Instructions }}
{{ Go Folder's Instructions }}
{{ Gin Conversation's Instructions }}
This would solve a problem I have currently, where I have a "base" coding conversation that I "clone" when working with a specific language/framework. If I need to make changes to this "base" instruction, I have to edit each of the conversations that were cloned from that base conversation. Using this inherited system, I would be able to make changes to the "Coding Instructions" (by editing the Coding folder's System Instructions), and the
Go -> Gin
, PHP -> Laravel
, CSS -> Tailwind
conversations that live underneath Coding would automatically use those changes.This would also help with a slightly related challenge where I have a base prompt that describes how I want GPT to respond, but it also has a bunch of coding-specific instructions in it. This means that when I'm discussing "Cooking", those coding instructions are included (despite the fact that they aren't used/relevant).
Cascading instructions would allow me to make changes at each level and have the conversation prepend each level along the way (it would probably be useful to show a preview of the full instructions in the conversation).
Daniel Nguyen
This has been released in the beta channel. Please help me test the beta release :D
To opt for beta updates, go to Settings > Advanced > Software Updates.
Jordan "Jado" Finnigan
Daniel Nguyen I don't see that option. I'm using the SetApp version of the app. Is it possible to switch to the beta from there?
Daniel Nguyen
in progress
Daniel Nguyen
Yeah. So my plan is to introduce a new concept called "Project". It's like an "advanced folder" but it has multiple other features:
- Custom "base" configurations that new chats within this project would inherit from
- Pre-attached documents so any new chat within this project would be able to query them without re-uploading
I'm not sure if we should introduce this new project, or make it an option for the current Folder.
On one hand I don't want to make it more complex to use (so avoid introducing new concept) but on the other hands, people would treat a folder as a simple group of chats like in the Finder.
What do you think?
Daniel Tango
Daniel Nguyen, project is fine. It could also live inside a folder.
Jordan "Jado" Finnigan
Daniel Nguyen The two options I can think of off the top of my head are:
_Note: I'm saying the following with the assumption that either option would allow nesting._
Option 1
Integrate the ideas behind "Smart Folders"/"Project Folders" into the existing folder structure, with settings allowed for each folder.
Upsides:
- By default, the folders could just be dumb folders and wouldn't have any additional functionality beyond what they do today. This would make it easy for existing users to continue to use Folders without needing to learn a new process.
- Users could discover the advanced settings for the folders on their own and in their own time.
- Users could have a mix of dumb and advanced folders, e.g., Work [dumb folder] -> Code [advanced folder]
Downsides:
- Deleting an advanced folder becomes MUCH more risky, since it's not just the container for organizing conversations that's being deleted, but also the configurations attached to it.
- It muddies the water to have them associated. i.e., How do you visually differentiate between a dumb and advanced folder? You might have to check the child folders to see if any are advanced so that you can warn the user, "There are 3 advanced folders contained inside 'Old Company'. Do you want to delete them?"
Option 2
Add the advanced folders as a new organizational type (in the same way that macOS has Folders and Smart Folders).
Upsides:
- Folders continue to be folders. Those who want this feature can use it, and those who don't wouldn't have to think about it.
- I suspect it would be easier to include a separate feature in a higher price tier than the other option (an advanced version of a feature).
- Easier to add new features unrelated to folders to this option.
Downsides:
- Based on your description, it sounds like Projects would introduce another, separate organizational structure to use. e.g., If I want to use the Projects feature, would I need to organize my Projects (in whatever way that you imagined) in addition to organizing my folders? Would I have to choose between organizing my projects and organizing my folders? Would I have to mirror the structure between the two?
I'd be curious to hear your thoughts. I think the ability to "nest" instructions would be the most useful to me, no matter the implementation.
James Shostak
Daniel Nguyen Projects sounds incredible!
Daniel Nguyen
Jordan "Jado" Finnigan: I decided to go with the project option.
Quick question. After starting a chat within a project, do you want it to sync the configuration with the project, or the project serve as a template only (starting point)?
- sync configuration with project: a bit annoying if you want to customize the chat further, since you will need to do it at the project level
- use project configuration as a starting point, then you're freely to customize the chat: this can be tricky when you update the project, would the chat follow the updated project or not?
It's almost finished BTW. Here is a teaser :D
Jordan "Jado" Finnigan
Daniel Nguyen I think the former. If it’s in that folder, it is overridden by the project with a notice. Exciting stuff!
Daniel Nguyen
Jordan "Jado" Finnigan Ah sorry. It's only available in the website version. I will fix the remaining bugs and release a stable version soon.
A
Adam Ram
Daniel Nguyen This is great improvement. Out of this topic, I already try this new feature with Project Files. I put 4 files in it and unfortunately it costs >600k tokens. Would be nice if there's an option to use embedding models than baked into system prompt, since it will be much like a bunch of files inside Project Files. Some tasks are much better with huge system prompt, but some are simply needs RAG.
Daniel Nguyen
Adam Ram Ah you ok with using a cloud RAG service? I don't plan to support local RAG: too much work and just too difficult to maintain. I did this with PDF Pals and it's a pain. Not going to do the same this time with BoltAI.
A
Adam Ram
Daniel Nguyen No, you don't have to build local RAG. I have my own local embedding+reranker endpoint using infinity backend (https://github.com/michaelfeil/infinity). Using works just like cloud RAG services. So you just need to add a setting to setup embedding API. It needs only a 8-12 VRAM GPU to run embedding+reranker model.
Daniel Nguyen
Adam Ram I see. Can you share more about your setup? Not installation etc but the api endpoints and how to use it. AFAIK there is no standard around this
A
Adam Ram
Daniel Nguyen The request will go to /v1/embeddings
You can check the settings on my OpenWebUI RAG settings, might be BoltAI settings can also has these variables.
Not the installation, but just for reference how I run infinity backend:
infinity_emb v2 --model-id ~/models/indo-bge-m3/ --model-id ~/models/bge-reranker-v2-m3/ --served-model-name tugasi-embed --served-model-name tugasi-rerank --port 5001 --api-key xxxx
Daniel Nguyen
Adam Ram Hmm this is not an end-to-end RAG system. It’s an endpoint to compute embeddings.
There are still a lot of work for storing these embedding in a vector database, inference and other maintenance operations.
A
Adam Ram
Daniel Nguyen Yes, I mean RAG system on BoltAI, excluding public endpoint. I just show how Open WebUI does it. That's okay if it's not feasible to develop, this suggestion based on experience of newly feature "Project Files" that can't process of 4 files because it's 600k tokens. Anyway, this feature is still great to be an expert system leveraging system prompt. Thanks, great job.
Daniel Tango
I like the idea. Can you share your prompts and how that makes a significant difference for you?
Jordan "Jado" Finnigan
Daniel Tango Here's a gist: https://gist.github.com/JadoJodo/b205ad2cbbd5fc8d18dc8140a998a128
I'm still testing it, but I'd love to hear any feedback you might have. It seems to work (generally) well, but I know that it is fairly long, which I want to improve. I am also not 100% sure of which portions should be optimized to more closely reflect the way that GPT likes/needs to be instructed.
Jordan "Jado" Finnigan
To reiterate the issue that prompted this feature request: The coding rules in that gist could be entirely dropped for, say, conversations about cooking (with different rules being applied for that. e.g., "Show me Metric AND Imperial when displaying measurements)" or one where I'm trying to learn a foreign language. I may want the base rules, but then have rules around cooking, and then another, additional ruleset around cooking, e.g., Korean food.
Daniel Tango
Jordan "Jado" Finnigan - Wow, very exhaustive. It works with GPT4o-mini, but not with Perplexity Sonar Large. Interesting.
Jordan "Jado" Finnigan
Daniel Tango is it rejected or it just doesn’t follow the instructions?
Daniel Tango
Jordan "Jado" Finnigan - it just does what it does and ignores the instructions. But i actually often dont use detailed prompts and instead rely on the base model behavior. So i couldn’t say why or what.
Jordan "Jado" Finnigan
Daniel Tango Interesting. I cleaned it up quite a bit, if you want to give it another go. I had a lot of stuff in there that it either already does by nature of GPT, or that just wasn't helpful and added too much fluff. The issue I have now is that GPT-4o-mini seems to have trouble following the formatting rules :/