Am I alone in struggling to trust closed-source model evaluation (especially to compare them to open-source)? I feel like for closed-source: - you don't know if the evaluation dataset isn't in the training dataset - you don't know if you're evaluating a raw model output or a

Awesome development in open-weight llms!

🔥Congrats to Maitrix.org for their new library: LLM Reasoners v1.0🔥 Great to see Shibo Hao and LLM360 members Yi Gu and Zhiting Hu’s continued contributions to open-source LLMs! Check them out: llm-reasoners.net

❄️Congrats to Snowflake for openly releasing Arctic!❄️ Arctic is available to all with an Apache 2.0 license! Great to see LLM360 member Aurick Qiao and the whole Snowflake AI Research’s team's amazing contribution to the open-source LLM community!

Thank you Lee Carter and Carlo Ruiz for inviting us to talk with the NVIDIA EMEA team earlier today! It was great sharing about our work together on the LLM360 with MBZUAI.

🔥Congrats to Maitrix.org for releasing Pandora, a World Model that can predict and simulate the world’s states in visual space, controllable by language 🎮 Excited to see the MaitrixOrg is indeed building something like the Matrix, great work by Jiannan Xiang, Guangyi, Yi Gu

🎉 Congratulations to an awesome fully open source model, by the m-a-p team! Paper: 📎arxiv.org/abs/2405.19327 Includes great info on: -Data Curation -Infra details -Intermediate checkpoints -Scaling law LLM360 is happy to work with this thriving community on open source AI.