Microsoft’s Orca 13B model is being touted as a small OpenSource model that imitates GPT-4 in the right way, focusing on reasoning rather than style. The lack of rigorous testing in other small models makes them seem as good as GPT-3.5, but Microsoft argues that their Orca follows “good” imitation. The model shows promise but the release of its weights is still being worked on.

source update: Imitating GPT-4 the “Right” Way – Towards AI