Standalone Chat Interface.

[gsaivinay]

Hello, greetings.

I'm self hosting text generation server with Pythia 12b model. I'm also developing a chat interface.

Looking for some inspiration from this project and would like to know about what components should I look to just have a chat interface, supporting backend code along with plugins, without having all the other components like dashboards, human data collections etc.

Thanks for the support in advance.

[olliestanley]

At the moment it's easy to run the inference backend alone (--profile inference in Docker compose) but to use the website as a chat interface you also need to run the data collection backend, currently. Running the website with an alternative inference backend isn't something we can support