Spaces:

---

IndianAIDevs

/

README

like 2

Running

Has anyone pre-trained LLM model from scratch ? If yes then share your experience, things to consider while training, notes, tips etc.

Hi i am also intrested into LLM Model , i am about to start this reserach from next week please give any inputs

Hi i am also intrested into LLM Model , i am about to start this reserach from next week please give any inputs

Hey @Shashank2k3 , if you want your own LLM model, first you need huge data. You can start with fine tuning already available good LLM models like Gemma, Phi, LLAMA, mistral etc with your dataset. Start with small models of sizes like 4 to 7B parameters. For pre-training LLM from scratch you need enormous data, good resources like heavy duty GPUs and CPUs and also have knowledge of training techniques, NLP, etc . You can always brainstorm with ChatGPT to get more knowledge.

Hey @kalashshah19 , thanks for the input! I already have a solid foundation in these areas from my Bachelor's degree in AIML, and now I’m looking to dive deeper into the world of LLMs.

Hey @kalashshah19 , thanks for the input! I already have a solid foundation in these areas from my Bachelor's degree in AIML, and now I’m looking to dive deeper into the world of LLMs.

Great !

Yupp so what you guys do, i mean profession!!!

Yupp so what you guys do, i mean profession!!!

I am an Associate Data Scientist at Casepoint.
What about you ?

Happy New year

Happy New Year !

@JDhruv14 Amazing work brother
https://tatva.info/ is just amazing. Would love to work on something together 🤝

@JDhruv14 Amazing work brother
https://tatva.info/ is just amazing. Would love to work on something together 🤝

Woahhh thanks. I had forgotten about sharing about my latest project tatva. How did you found about it? Also, if possible please vote for me on Peerlist. I want to make tatva better that's the aim.

@JDhruv14 Amazing work brother
https://tatva.info/ is just amazing. Would love to work on something together 🤝

Woahhh thanks. I had forgotten about sharing about my latest project tatva. How did you found about it? Also, if possible please vote for me on Peerlist. I want to make tatva better that's the aim.

i forgot from where i found it but yesterday i noticed the developer name and saw the posts on X, thought i should promote it. i upvoted tatva on producthunt & peerlist

https://huggingface.co/Shaligram-Dewangan/Dhi-5B-Base

my senior (3rd year) trained this model from scratch

Sarvam AI launches Indus, an interface powered by their 105B model
try here: https://indus.sarvam.ai
Also, you can use its mobile app here: https://play.google.com/store/apps/details?id=ai.sarvam.indus

New Open Model Release — Param2-17B-A2.4B (Thinking)

BharatGen released a reasoning-optimized MoE model designed for multilingual + Indic intelligence at the India AI Impact Summit.

Why it’s interesting:
• Strong performance across reasoning, factual QA & Indic benchmarks
• MoE architecture → ~2.4B active params → efficient inference
• Built for culturally grounded & multilingual tasks
• Solid balance vs reasoning-heavy DeepSeek R1

This signals serious progress toward sovereign, deployment-ready AI for Indian languages and real-world workflows.

Model: https://huggingface.co/bharatgenai/Param2-17B-A2.4B-Thinking