etemiz/Ostrich-70B-Llama3-251212
Text Generation
•
71B
•
Updated
•
18
•
1
Emin Temiz PRO
etemiz
AI & ML interests
Alignment
Recent Activity
updated
a model
2 days ago
etemiz/Ostrich-70B-Llama3-251212
published
a model
3 days ago
etemiz/Ostrich-70B-Llama3-251212
Organizations
None yet
updated
a
model
2 days ago
replied to
their
post
2 days ago
I made an LLM act as an aggregator and combine answers of several other LLMs like in a mixture of agents scenario. The aggregator does not always produce the average or median answer. It brings its own opinion.
I think Google has a benchmark for this (sticking to context and not bringing its own words).
published
a
model
3 days ago
posted
an
update
3 days ago
Post
1665
Today's winner is Ling 1T with a score of 38!
Btw AHA2 is in the works, with more domains, better comparison LLMs and questions, overall better signal.
Btw AHA2 is in the works, with more domains, better comparison LLMs and questions, overall better signal.
- 1 reply
commented on
New in llama.cpp: Model Management
3 days ago
what is the best way to get <think> </think> and the tokens in between? openAI library is removing them.. i want to run llama-server in console and talk to it using a python library that does not remove the thinking tokens.
i checked the llama-cpp-python but it does not have that.
posted
an
update
4 days ago
Post
218
Fine tuning is also important in a RAG system. The LLM will bring its own opinion sometimes and tell a different answer which is contrary to the retrieved knowledge. One should use an aligned LLM to produce the final answer.
-
- 2 replies
posted
an
update
24 days ago
Post
1109
Two new entries to the AHA Leaderboard:
Kimi was OK until it started "thinking" ..
Kimi was OK until it started "thinking" ..
posted
an
update
25 days ago
Post
1752
Working on a new version of AHA Leaderboard.
- Better signal thanks to new models like Enoch
- MoA of top and bottoms of current leaderboard to add more diverse inputs
- Better questions
- A faster and more precise measurement methodology
- Explanations on how each column is calculated
- More sample questions revealed
Current version: https://huggingface.co/blog/etemiz/aha-leaderboard
Tell me how I can improve more.
- Better signal thanks to new models like Enoch
- MoA of top and bottoms of current leaderboard to add more diverse inputs
- Better questions
- A faster and more precise measurement methodology
- Explanations on how each column is calculated
- More sample questions revealed
Current version: https://huggingface.co/blog/etemiz/aha-leaderboard
Tell me how I can improve more.
commented on
The AHA Indicator
about 1 month ago
Update to the indicator
https://huggingface.co/posts/etemiz/490866222392672
posted
an
update
about 2 months ago
Post
291
This is a very human aligned fine tune and would score 56 on AHA leaderboard:
CWClabs/CWC-Mistral-Nemo-12B-V2-q4_k_m
We are going to be using it as one of the ground truths for AHA Leaderboard 2.0 (the next version).
We will be able to generate some RL datasets for folks to align their own LLMs with humanity. We will generate answers from best models and worst models and do mixture of agents that combines the answer, and publish results as dataset(s). Things looking bright!
CWClabs/CWC-Mistral-Nemo-12B-V2-q4_k_m
We are going to be using it as one of the ground truths for AHA Leaderboard 2.0 (the next version).
We will be able to generate some RL datasets for folks to align their own LLMs with humanity. We will generate answers from best models and worst models and do mixture of agents that combines the answer, and publish results as dataset(s). Things looking bright!
liked
a
model
about 2 months ago
posted
an
update
about 2 months ago
Post
301
Benchmarked 2 more models today. Any other model you want me to do?
Working on a broader version of the AHA leaderboard. Follow for more quackery :)
Working on a broader version of the AHA leaderboard. Follow for more quackery :)
replied to
their
post
about 2 months ago
This fine tuning would score 56 and be placed 1st in the leaderboard but I didn't add it, I only include full trainings in the leaderboard or (further tunings by the same company):
https://huggingface.co/CWClabs/CWC-Mistral-Nemo-12B-V2-q4_k_m
posted
an
update
about 2 months ago
Post
269
O oh! things looking uglier by day
LLM builders in general are not doing a great job of making human aligned models.
I don't want to say this is a proxy for p(doom)... But it could be if we are not careful.
Most probable cause is reckless training LLMs using outputs of other LLMs, and don't caring about curation of datasets and not asking 'what is beneficial for humans?'...
LLM builders in general are not doing a great job of making human aligned models.
I don't want to say this is a proxy for p(doom)... But it could be if we are not careful.
Most probable cause is reckless training LLMs using outputs of other LLMs, and don't caring about curation of datasets and not asking 'what is beneficial for humans?'...
-
- 1 reply
updated
a
model
about 2 months ago
liked
2
models
about 2 months ago
published
a
model
2 months ago