A modular framework for vision & language multimodal research from Facebook AI Research (FAIR)

streamline the fine-tuning process for multimodal models: PaliGemma, Florence-2, and Qwen2-VL

Code for "Aligning Linguistic Words and Visual Semantic Units for Image Captioning", ACM MM 2019

JoyCaption is an image captioning Visual Language Model (VLM) being built from the ground up as a free, open, and uncensored model for the community to use in training Diffusion models.

CapDec: SOTA Zero Shot Image Captioning Using CLIP and GPT2, EMNLP 2022 (findings)

Audio Captioning datasets for PyTorch.

VisText is a benchmark dataset for semantically rich chart captioning.

Fully-Convolutional Point Networks for Large-Scale Point Clouds

Python code for handling the Clotho dataset.

A Tennis dataset and models for event detection & commentary generation

A Base Tensorflow Project for Medical Report Generation

Medical image captioning using OpenAI's CLIP

What and How Well You Performed? A Multitask Learning Approach to Action Quality Assessment [CVPR 2019]

Remote Sensing Temporal Vision-Language Models: A Comprehensive Survey

Positive-Augmented Contrastive Learning for Image and Video Captioning Evaluation. CVPR 2023

[CVPR21] Visual Semantic Role Labeling for Video Understanding (https://arxiv.org/abs/2104.00990)

A gradio based image captioning tool that uses the GPT-4-Vision API to generate detailed descriptions of images.

A Pytorch implementation of Attention on Attention module (both self and guided variants), for Visual Question Answering

Using LLMs and pre-trained caption models for super-human performance on image captioning.

Audio captioning baseline system for DCASE 2020 challenge.