It fails other benchmarks vs Mistral-7b. https://twitter.com/Teknium1/status/172... | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

		ple13 on Nov 21, 2023 \| parent \| context \| favorite \| on: Orca 2: Teaching Small Language Models How to Reas... It fails other benchmarks vs Mistral-7b. https://twitter.com/Teknium1/status/1726846755344634020 (There is some doubts about the validity of the comparaison in the comments)

eurekin on Nov 21, 2023 [–]

Also, worth mentioning the next tweet:

  Update, I benchmarked 13b Orca 2, its still not surpassing gpt4all score of

  Base Mistral or OpenHermes 2.5 7B:

  Hermes 2.5 7B Mistral score: 73.12%

  Mistral Base 7B score: 71.16%

  Orca 13B GPT4All score: 70.58%

https://twitter.com/Teknium1/status/1726833004117635414

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact