Gpt4all integration in progress. Any other LLMs you want to see integrated?

[DataBassGit]

I'm working on implementing GPT4All into autoGPT to get a free version of this working. All LLMs have their limits, especially locally hosted. The best bet is to make all the options. So throw your ideas at me.

[drgutman]

vicuna?
didn't get to test it myself but I heard that it has better results than gpt4all

[GoZippy]

Wizard Mega 13B uncensored

[Irongate2G]

I had Dolly 2 3b working on my pc but didn't know how to integrate it. Still they all suffer from learning and memory, so the real goal is to get a local small model, that learns from what you feed it and remembers what you feed it. Otherwise, you just end up with garbled trash.

[GoZippy]

Maybe another agent to run a script that reads main agent workspace memory and saved files then processes them into local model with vectorized db backend...? Continuous training. There's no reason to not train from new info learned especially from web intelligence gathering during AutoGPT agent tasks... Store all website info local cache file and response data from OpenAI or otherwise and process into local model after pruning junk...

[GoZippy]

Or just another script running that monitors saved files in working directory... Maybe a chrome plugin to dump text from automated page loads and browsing to local file to process as well...

[NotaHumanResource]

No LLM is as good as all of them. The ability to utilize multiple LLM's simultaneously would be interesting.

[Torantulino]

@DataBassGit Is there a PR for this?
Would be incredible to have offline GPT4ALL running!

[zhaozehua666]

我想用CHATGPT4建一个属于企业的独立AI机器人，可以为企业降本增效，并且数据信息上传后AI可以判断权限，是否为个人提供必要的服务，对企业机密做保护。有没有能训练LLM也就是训练大型语言智能机器人的人可以和我联系。

[gmy77]

Wait for this...

[claysauruswrecks]

Putting them into a docker-compose would make it really easy to run and config.

[zhaozehua666]

我想用CHATGPT4建一个属于企业的独立AI机器人，可以为企业降本增效，并且数据信息上传后AI可以判断权限，是否为个人提供必要的服务，对企业机密做保护。有没有能训练LLM也就是训练大型语言智能机器人的人可以和我联系。

[GoMightyAlgorythmGo]

translation english
我想用CHATGPT4建一个属于企业的独立AI机器人，可以为企业降本增效，并且数据信息上传后AI可以判断权限，是否为个人提供必要的服务，对企业机密做保护。有没有能训练LLM也就是训练大型语言智能机器人的人可以和我联系。
"I want to use CHATGPT4 to build an independent AI robot belonging to the enterprise, which can reduce costs and increase efficiency for the enterprise, and after the data information is uploaded, the AI ​​can judge the authority, whether to provide necessary services for individuals, and protect corporate secrets. Can anyone who can train LLM, that is, train large-scale language intelligent robots, contact me."

sounds interesting i think AI that manage large parts of businesses and take the misaligned incentive structures of humans and some humans are extreamly cleaver at hiding things (out of necessity evolutionary or because of other problems usually, if there is 1 problem usually everyone that cant fix it openly goes into that mode that can) from my evaluation of common parts of human behavior. That is another reason why AGI is the best philanthropic investment for worldpeace/prosperity/fixing problems pretty much universally and if not directly definitely indirectly.

[stefangrotz]

Might be worth thinking about creating a Dataset from the OpenAI API calls to be able to fine tune other models for Auto-GPT.

[DrakenZA]

I'm working on implementing GPT4All into autoGPT to get a free version of this working. All LLMs have their limits, especially locally hosted. The best bet is to make all the options. So throw your ideas at me.

Interesting, how will you go about this ? My tests show GPT4ALL totally fails at langchain prompting.

[GoMightyAlgorythmGo]

vicuna is good. there are all kinds of models i havent looked deeper into it. Depending how you prestructure it everything is possible its just a matter of different levels of effitiency. I would imagine you cant expect letting them talk without any rails or [do this] then [do that] give them as much freedom your security measures allow and plan for the exceptions rather than the rules. Many things involved. But i tested gpt4all and alpaca too alpaca was somethimes terrible sometimes nice would need relly airtight [say this then that] but i did not relly tune anything i just installed it so probably terrible implementation maybe way better

[GoZippy]

any better at coding than others? test results?

[r1di]

This would be amazing!

[GoMightyAlgorythmGo]

Here are two preliminary suggestions (please verify if they are useful first):

-https://huggingface.co/autotrain automatic training it will automatically find models that may help.
(Unrelated sidetalk: I think AGI will be a combination of models/methods like our brain is. Maybe autoGPT can test and try them and use statistics/machinelerning (trends and effectiveness: Trends for models that perform worse but show promise and effectiveness for the best models based on chatGPT4 rating (reflecting on performance) or by some external means)

Language Model "Alpaca" form standford is also available i heared and standford has published a lot over the decades so it might stay relevant/be updated in the future.
Alpaca details
This combines the LLaMA foundation model with an open reproduction of Stanford Alpaca a fine-tuning of the base model to obey instructions (akin to the RLHF used to train ChatGPT) and a set of modifications to llama.cpp to add a chat interface.

git clone https://github.com/antimatter15/alpaca.cpp

(I have cloned the Alpaca model and (barely) tested it, like Gpt4all it does work.)

(PS:
Edit: Vicuna seems to be one of the best but requires virtual memory/ram because it needs a lot. Gpt4all/alpaca seem more lightweight. When using chain of thought formatting like [say exactly only this in this way here] then go here and only say [a critique] and relly make it detaled and use its full response on every one of those exept of giving it too much that its working memory cant handle there definitly might be potential to use idle pc ressources)

3. most importantly: use AutoGPT cmd logs to finetune/train something automatically so it can help as a side agent while AutoGPT is running there is so much to lern from using AutoGPt i bet it comes up with creative /most optimal ways to use AutoGPT way better. Using Datasets like this are extreamly powerful as we seen from the success of using the best language models output to train other models with massive success. No wonder

Unrelated some dreaming:
Just imagine they are gona run on our pc's and just work like AutoGPT but even if way less efficiently when they are free they can run super long and i can even now create amazing deep plans by copy pasting high level -> medium level -> low level -> Step by Step -> Programming chat amazing plans to research and further research and try to work/build agi/auto agi builder/trainer. Love the AutoGPT if society is wealthyer that generally lifts all boats and makes the world a better place, so i think it is on of the best effective philanthropic
and economically (work to survive) ways to accomplish many goals at once. Also it can help me find adhd meds and life coach by machine lerning correlations maybe help fight dystonia/depression to some extent in the future. Even for "normal" people high or low iq. 🌈🦄 💸 )

[zhaozehua666]

我想用CHATGPT4建一个属于企业的独立AI机器人，可以为企业降本增效，并且数据信息上传后AI可以判断权限，是否为个人提供必要的服务，对企业机密做保护。有没有能训练LLM也就是训练大型语言智能机器人的人可以和我联系。

[CreeperNinjaYT]

I'm working on implementing GPT4All into autoGPT to get a free version of this working. All LLMs have their limits, especially locally hosted. The best bet is to make all the options. So throw your ideas at me.
This sounds great!
It has been a week since you posted this discussion, is there any progress that can be tested?

I really want to stay updated with this as it would be amazing to have something like Auto-GPT but offline, please keep us (Me) updated on the progress!

[hellohau]

Are these models even better than GPT3.5 turbo ?

[CreeperNinjaYT]

No they are not, but the upside is that it can be run offline and it would be 100% free if it works, ultimately allowing this to run it freely and privately

[GoMightyAlgorythmGo]

no chatGPT3.5t is the best except from chatGPT4 alto vicuna is said to be 92% of chatGPT4 i don't think that is even remotely the case based on instinct (extensive chatGPT3.5t and chatGPT4 testing and the free L.M.s i tried so far, i have not been able to test vicuna apart from a 5-15 text messages yet) alone but would have to be verified. Not all modes use gpt generations to train some use free no copy right stuff only too by the way

[Samsuditana]

Have we considered asking autogpt to write a version of itself that can utilize offline models? Lol

[GoMightyAlgorythmGo]

Yes but execution and plans are different. Also many people probably not. Not nessesarily for terrible reasons using autoGPT for development would be a smallter more immediate milestone

[Stonedge]

Koala, openassistant(oasst), gpt-x-alpaca,vicuna ... honestly there's a lot of good local model. Using something like llamacpp-python would probably enable user to use any of those model (also think that llamacpp offer embeddings*not sure)... honestly what we really need to solve would be a Lora for those model to fine tune then at answering json/make task/prioritize task based on log shared by user of autogpt... that would be awesome 👌

[brandus1]

Dolly 2.0 is open source also for commercial use. I haven't tried it yet, though. Anyone?

[GoMightyAlgorythmGo]

its a image generation its good but also bad depending, not sure how it would help at all right now at least ultra mega super low priority. AutoGPT can already do some images from even lower huggingface language models i think. But dally 2 costs money after your free tokens not worth other prioritys -lots - no motivation - no brain activation (ignore unclear statements)

[andreacortis]

I think the poster was referring to: https://huggingface.co/databricks/dolly-v2-12b
I would be very interested in seeing this engine implemented!

[einarpetersen]

Why not interface your AI with your locally running stable diffusion? - I am planning to do that via python and my ChatGPT API having ChatGPT do the prompts as I have trained it to do and then the stable diffusion spitting out images locally according to the python based instructions to CLI that the Python interface delivers to stable diffusion

[inflaton]

Dolly 2.0 is open source also for commercial use. I haven't tried it yet, though. Anyone?

I did. The results look good but it took 39+ minutes to answer a question.

[einarpetersen]

May I suggest that while focusing on including other models that you really look at creating a universal interface for (autoGPT'ing) so that there is a drop in structure for "any" models as long as they have an i/o API that would then allow you to play with whatever new model comes along using autoGPT as a bridging orchestrator / logic layer through which to work with different GPT models

[kenster99]

Along those lines, Langchain may be that agnostic interface to other LLMs, so as opposed to creating individual bridges to each LLM, leverage Langchain as a facade and it's a matter of developing the one adapter to Langchain.

[abhawan888]

Microsoft JARVIS at huggingface perhaps? Since it supposedly works with other models as well.
Just a suggestion :)

[einarpetersen]

In my optics we don't want huggingface / Microsoft dependency - it must be separate from specific location and services primarily able to run on a users own system accessing a model of users choice

[Stonedge]

Vicuna 1.1 has just been release... i'm currently testing it with my babyagi style script but it seems to be a well trained model up to now

[GoMightyAlgorythmGo]

Any snips?

[Lunchbokss]

"GPT4All-j"

[MetaWabbit]

No bias is one of many reasons. Big reason for myself is having a LLM model that's basically unlocked, free from any constraints or Digital Oppression (A term in which may have just been first used and expressed but, think about that one for min seriously 🤔). A outlaw model in which we can truly unlock and see the full capabilities and possibilities of artificial intelligence. Embracing the unknown💯🤯

…

On Thu, May 18, 2023, 9:57 PM GoZippy ***@***.***> wrote: no bias - just raw pretrained data to pull from — Reply to this email directly, view it on GitHub <#34 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AY5DWBXFQ73GJXVE3IVJTKDXG3HRTANCNFSM6AAAAAAWQN34HM> . You are receiving this because you commented.Message ID: ***@***.*** .com>

[Wladastic]

This can be revived once the new agent loop discussion and code is done.
My plan is also to make the thought process more modular, but we will have to see how it turns out.

[GoZippy]

So what happened? All of a sudden half the discussions on different ways to implement multiple llm are closed and reference this one but no one has prepared any specifics yet. Recent closure of all the ways to reduce token limit within llm scope are also closed yesterday and referred to this... anyone have a summary of why those issues were all force closed and what the genersl consensus is for moving forward with new loop initiation with error handling? I stopped working on solutions due to confusing direction... my work is mostly in handling intent prompt size limiting or " chunking" as others described but I stated working on methods to allow local llm or remote with unified handler based on model and location limits. I was working on user environmental tests and variation handling so that agent loop is using standards while sending requests to new method translation layer that handles the user defined llm case for each request... idk.. was hopeful to find a way to get new agent spawning to "think" about which llm is best for the given task, initiation of a new agent for that with the defined llm either local or remote and let the handler act as traffic controller with token limits handled in the traffic controller loop based on standard definitions which define limits and response formatting based on intent translations before being sent back to agent as response from llm requested... was getting deep in the woods with where to handle model construct limits/ token limits and errors such that the requesting agent loop isn't broken on invalid or unexpected response and handler/ controller has methods to restart requests after reformatting bad length or otherwise after breaking down messages by summarize method chunking... the idea was to create another layer above agent spawning to listen for requests and translate with error handling such that everything is unified below it so an api can be developed...

[Wladastic]

I have started working on a solution myself but was flooded with work outside of the project.
The main point why this is halted is because of the extensive work on re-arch, the thought process is being rewritten.
I also had the idea of letting auto-gpt or a machine learning model decide which llm to use if at all.
Right now I am experimenting with micro llm's that reinterpret small prompts given by gpt-4 to reduce it's responsibility, as I found loading all context into gpt-4 prompts makes no sense in long term.
As for example even crawling a website just overwrites almost all context tokens and auto-gpt forgets its main goal. This should be definitely handled by a different agent or agents.

[GoZippy]

any update?

[Adrianogb]

Estou trabalhando na implementação do GPT4All no autoGPT para obter uma versão gratuita deste trabalho. Todos os LLMs têm seus limites, especialmente hospedados localmente. A melhor aposta é fazer todas as opções. Então jogue suas ideias em mim.

They have a GUI with some very good (Eastern) models, the best I have found so far, I would like someone to implement the GPT4All model in that GUI as well:
https://github.com/GaiZhenbiao/ChuanhuChatGPT

[hzhangxyz]

Although text generation webui provide openai-like api, many model have context window less than 2048, while in AutoGPT, many prompt exceeds this limit, is there any solution? or is there any open weighted llm with large token limit?

[thistleknot]

Llama2

[thistleknot]

Stablecode

Google knowledge graph api keys and beautifulsoup web scrapes (I have code)

[t0pd09590dp0t]

wizardlm-13b-v1.1-superhot-8k

[aespeiuk22]

llama2 seems to be like gpt4, having llama2 will be a great breakthrough. The important thing is, there should be a 3 person process where there will be a light weight model or a parallel process of llama2 itself between the agent and the person to check looping and to guide and make decisions following strict guidelines by the intermediate one

[kfsone]

I've had AutoGPT connecting to models hosted in LM Studio 0.2.8, using a docker-hosted autogpt by:
adding network_mode: host to the docker compose, and specifying the OPENAI_API_BASE url with the machine's lan IP address:

OPENAI_API_BASE_URL=http://192.168.86.37:xxxx/v1

But despite running 0.2.8 of LM Studio which now plays nicely with autogen agents, something still goes horribly wrong trying to use AutoGPT, but I'm just about to take the time to actually go look at what the errors are telling me: there's shouldn't be a token limit but maybe I do need to tune some of the parameters better.

[GoZippy]

got any logs to share?

[wiseman-timelord]

I have used LM Studio with OpenCodeInterpreter, and I think its the way to go.
The .env has the items, "OPENAI_API_BASE_URL" and "SMART_LLM" and "FAST_LLM"; "OpenCodeInterpreter-DS-6.7B" for "FAST_LLM" and "OpenCodeInterpreter-DS-33B" for "SMART_LLM", would logically be the way to currently go based on the leaderboard.

The part I am unsure of is the API key. This would logically have no effect or cause issue, and so may require to be redacted from the scripts.

[wiseman-timelord]

after 5 hours of, investigating and updating, AutoGPT 4.7, the problem ended with LM server reporting..

[2024-04-02 10:51:45.630] [ERROR] Unexpected endpoint or method. (POST /v1/completions/chat/completions). Returning 200 anyway

...this error is in how the OpenAI library works. The solution would be to do what OpenCodeInterpreter does, and replace all the code for "OpenAI" with code for "litellm". It would require a lot of work, and I decided with all things considered with my earlier hacks to hardcode things to my local model, I would prefer to start over, with the focus on replacing the OpenAI code as I go with something hardcoded for litellm and local models.

I wont be doing that at the moment, not until I get my head around the structure of version 5, if I am going to invest serious time in it.

[shuther]

My 2 cents; if we use litellm, we can leverage it as a sidecar (act as a proxy/router); it may allow to adapt to the different models and keep the OpenAI sdk for now (maybe more future proof).

[shuther]

Going through the thread, it seems everybody agrees that we (will) need several, and new interesting models popup every month. I imagine AutoGPT to call AI model based on a context (prompt, language, level of expected quality,text vs. voice vs. image vs. python dev ....) on a specific endpoint; and this endpoint would chose the best model to run according to a strategy to be defined. For now, it could be a proxy to openAPI/Azure but it would be easier to support and test future models and improving the strategy in the future. What do you think?

[GoZippy]

It's been a while now. What's the plan? I'm ready to spend some time on this but hit pause 3 months ago waiting to see what all the changes were going to be. My branch works.

[thistleknot]

Suggestion: gpt4all leverages text-generation-webui's api and/or transformers inference library

…

On Sun, Nov 5, 2023 at 6:07 AM GoZippy ***@***.***> wrote: It's been a while now. What's the plan? I'm ready to spend some time on this but hit pause 3 months ago waiting to see what all the changes were going to be. My branch works. — Reply to this email directly, view it on GitHub <#34 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABHKKOUEQEJ673KHP6GRK5TYC6FRFAVCNFSM6AAAAAAWQN34HOVHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM3TINZZGM3TA> . You are receiving this because you commented.Message ID: ***@***.*** com>

[GoZippy]

With forge and the direction this project is going I am not sure if anyone has circled back to this yet. Any updates? I can spend some time over the next couple weeks...

[beuvelet]

Je souhaite utilisé les modèles GGUF de Mistral car s'est des modèles très efficace et avec une meilleur compréhension sur la plupart des languages !

[ntindle]

Theres an open pr for this

[thistleknot]

There was a change in the spo endpoint. User_Bio something. oobabooga/text-generation-webui#5761 (comment)

…

On Tue, Apr 2, 2024, 5:16 AM Mister Maximus ***@***.***> wrote: after 5 hours of, investigating and updating, AutoGPT 4.7, the problem ended with LM server reporting.. [2024-04-02 10:51:45.630] [ERROR] Unexpected endpoint or method. (POST /v1/completions/chat/completions). Returning 200 anyway ...this error is in how the OpenAI library works. The solution would be to do what OpenCodeInterpreter does, and replace all the code for "OpenAI" with code for "litellm". It would require a lot of work, and I decided with all things considered with my earlier hacks to hardcode things to my local model, I would prefer to start over, with the focus on replacing the OpenAI code as I go with something hardcoded for litellm and local models. I wont be doing that at the moment, not until I get my head around the structure of version 5, if I am going to invest serious time in it. — Reply to this email directly, view it on GitHub <#34 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABHKKORRUD46N7SZ5X6VWYDY3KHPFAVCNFSM6AAAAAAWQN34HOVHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM4DSOBSGY3DI> . You are receiving this because you commented.Message ID: ***@***.*** com>

[PiersPikachu]

I'm working on implementing GPT4All into autoGPT to get a free version of this working. All LLMs have their limits, especially locally hosted. The best bet is to make all the options. So throw your ideas at me.

don't suppose you could integrate anythingllm's api-key into auto-gpt? just curious...

[ntindle]

have a link?