[laion5B is down] Create indices of other datasets to make the demo work

[yzou2]

Thanks for the great work on the OpenCLIP search index interface in https://rom1504.github.io/clip-retrieval/?back=https%3A%2F%2Fknn.laion.ai&index=laion5B-H-14&useMclip=false

But the host is down and we cannot search anything in that host. Would you kindly tell when the server will be back to normal status?

[rom1504]

https://laion.ai/notes/laion-maintanence/ The laion5B dataset is currently down

…

On Fri, Jan 5, 2024, 7:13 AM Yang Zou ***@***.***> wrote: Thanks for the great work on the OpenCLIP search index interface in https://rom1504.github.io/clip-retrieval/?back=https%3A%2F%2Fknn.laion.ai&index=laion5B-H-14&useMclip=false But the host is down and we cannot search anything in that host. Would you kindly tell when the server will be back to normal status? — Reply to this email directly, view it on GitHub <#324>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAR437T5YECGOAWQSU2LP4LYM6KXHAVCNFSM6AAAAABBN62RQSVHI2DSMVQWIX3LMV43ASLTON2WKOZSGA3DMNZYG4ZDENA> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[yzou2]

Thanks, is there an ETA for the search interface to get back to normal?

[rom1504]

For laion5B to come back sadly I cannot really give an ETA, there's a lot of stuff going on there, with some not depending on us.

For any index to be available in this demo, there are some alternatives. We could create a few indices for:

• cc3m
• cc12m
• datacomp1b
• coyo700m

I guess that would help people.

What do you think?

[yzou2]

Thanks! If we can have some indices for datacomp1b (including people images), or the coyo700m, they will be definitely really helpful for many people.

[helblazer811]

Is there another index that can be used as an alternative in the mean time?

[rom1504]

No not yet

[RobbeSneyders]

We created an index for the datacomp-12.8M dataset using Fondant and published it on the huggingface hub. You can find more details and info on how to use it in this short post.

[nannigath]

Any updates on datacomp1b and coyo700 indexing?

[chenyangzhu1]

Is there another index that can be used as an alternative in the mean time?

I created an index on a LAION subset and implemented a simple script for CLIP retrieval. Please try it and give me feedback. : )

[Thomas2419]

Any update on this? I'm a bit surprised it's down so long with no alternatives, it's insanely useful for targeted research into domain specific models. Especially when keeping model finetuning in mind. It was a great tool that served open-source everywhere allowing people to analyze it and get url lists.