The ultimate guide to RL environments: building and scaling them in the LLM era
📝
164
Building and scaling RL environments for LLM training
Tirupati Venkata Sri Sai Rama Raju Penmatsa PRO
rajuptvs
AI & ML interests
None yet
Recent Activity
liked a Space 7 days ago
AdithyaSK/rl-environments-guide upvoted an article 4 months ago
LightOnOCR-2-1B: a lightweight high-performance end-to-end OCR model family commentedon an article 7 months ago
How I Trained Action Chunking Transformer (ACT) on SO-101: My Journey, Gotchas, and LessonsOrganizations
liked a
Space 7 days ago
upvoted an article 4 months ago
Article
LightOnOCR-2-1B: a lightweight high-performance end-to-end OCR model family
lightonai
• • 94 commented on How I Trained Action Chunking Transformer (ACT) on SO-101: My Journey, Gotchas, and Lessons 7 months ago
Incredible read!! tempting me just a bit more to get my own So 101 😀 .
upvoted an article 7 months ago
Article
How I Trained Action Chunking Transformer (ACT) on SO-101: My Journey, Gotchas, and Lessons
sherryxychen
• • 70 upvoted a collection about 1 year ago
liked a
model about 1 year ago
published a
Space about 1 year ago
reacted to Jaward's post with 🔥👀 about 1 year ago
Post
3745
Implemented a custom multimodal GRPO trainer that scales for Small VLMs, supports cpu and gpu with vllm + flash attention. Using SmolVLM-256M-Instruct reference & reward model, wasn’t trained for long btw, still got some sparks of “thinking”:)
Code: https://github.com/Jaykef/ai-algorithms/blob/main/grpo_multimodal_reasoner.ipynb
Code: https://github.com/Jaykef/ai-algorithms/blob/main/grpo_multimodal_reasoner.ipynb
- 1 reply
liked a
model about 1 year ago
updated a
collection about 1 year ago
liked a
model about 1 year ago
reacted to merve's post with 🔥 over 1 year ago
Post
2840
What a week! A recap for everything you missed ❄️
merve/nov-22-releases-673fbbcfc1c97c4f411def07
Multimodal ✨
> Mistral AI
released Pixtral 124B, a gigantic open vision language model
> Llava-CoT (formerly known as Llava-o1) was released, a multimodal reproduction of o1 model by PKU
> OpenGVLab released MMPR: a new multimodal reasoning dataset
> Jina has released Jina-CLIP-v2 0.98B multilingual multimodal embeddings
> Apple released new SotA vision encoders AIMv2
LLMs 🦙
> AllenAI dropped a huge release of models, datasets and scripts for Tülu, a family of models based on Llama 3.1 aligned with SFT, DPO and a new technique they have developed called RLVR
> Jina has released embeddings-v3: new multilingual embeddings with longer context
> Hugging Face released SmolTalk: synthetic dataset used to align SmolLM2 using supervised fine-tuning
> Microsoft released orca-agentinstruct-1M-v1: a gigantic instruction dataset of 1M synthetic instruction pairs
Image Generation 🖼️
> Black Forest Labs released Flux 1. tools: four new models for different image modifications and two LoRAs to do image conditioning and better steer generations
Lastly Hugging Face released a new library Observers: a lightweight SDK for monitoring interactions with AI APIs and easily store and browse them 📚
$ pip install observers
merve/nov-22-releases-673fbbcfc1c97c4f411def07
Multimodal ✨
> Mistral AI
released Pixtral 124B, a gigantic open vision language model
> Llava-CoT (formerly known as Llava-o1) was released, a multimodal reproduction of o1 model by PKU
> OpenGVLab released MMPR: a new multimodal reasoning dataset
> Jina has released Jina-CLIP-v2 0.98B multilingual multimodal embeddings
> Apple released new SotA vision encoders AIMv2
LLMs 🦙
> AllenAI dropped a huge release of models, datasets and scripts for Tülu, a family of models based on Llama 3.1 aligned with SFT, DPO and a new technique they have developed called RLVR
> Jina has released embeddings-v3: new multilingual embeddings with longer context
> Hugging Face released SmolTalk: synthetic dataset used to align SmolLM2 using supervised fine-tuning
> Microsoft released orca-agentinstruct-1M-v1: a gigantic instruction dataset of 1M synthetic instruction pairs
Image Generation 🖼️
> Black Forest Labs released Flux 1. tools: four new models for different image modifications and two LoRAs to do image conditioning and better steer generations
Lastly Hugging Face released a new library Observers: a lightweight SDK for monitoring interactions with AI APIs and easily store and browse them 📚
$ pip install observers
- 3 replies
i totally missed the observers.. thanks a lot!!
upvoted a collection over 1 year ago
updated a
dataset almost 2 years ago
Unable to do inference of finetuned lora model
➕ 1
4
#69 opened almost 2 years ago
by rajuptvs
Adding `safetensors` variant of this model
#1 opened over 2 years ago
by 
SFconvertbot
updated 2
models over 2 years ago