New project! Mosaic is a framework for scalable and interoperable visualization that unifies visualization toolkits with scalable visualization methods and query optimizations. uwdata.github.io/mosaic/what-is…

I'm extremely excited about this project with Jeffrey Heer. It brings together some key visualization techniques and tools to enable smooth interactive visualization experiences. Lots of cool details to talk about here. Take it for a spin and let us know what you think!

You might ask how this relates to the new Falcon Vis library from x.com/donny_bertucci…. Falcon Vis is useful when you need a lean library (no DuckDB needed). It's also leaner to add crossfiltering to existing charts in an app with it.

💡Our new TMLR paper presents a systematic evaluation of "blindspot discovery methods" – algorithms to discover "hidden" subgroups where an image classifier underperforms🔎 We also outline open directions for future work on behavioral evaluation of ML!

I'm happy to announce my latest project, Data Navigator! 🎉 I've been frustrated these past few years with all of the problems that visualization toolkits face making their visualizations more accessible. I built a tool that I hope will help. Read more: frank.computer/blog/2023/09/i…

A * sneak peek * at my favorite feature in the upcoming revamp of Zeno 𝙕𝙚𝙣𝙤 𝙍𝙚𝙥𝙤𝙧𝙩𝙨 Author reports with interactive charts grounded directly in your data and tell stories of how AI systems behave. Example report on LLMs and translation: hub.zenoml.com/report/cabrera…

Using OpenAI Whisper models for audio transcription? 𝐖𝐡𝐢𝐜𝐡 𝐯𝐞𝐫𝐬𝐢𝐨𝐧 𝐬𝐡𝐨𝐮𝐥𝐝 𝐲𝐨𝐮 𝐮𝐬𝐞? In our second Zeno report, we find big performance differences between model versions, especially *English* versions! Check out the report! hub.zenoml.com/report/cabrera…

Zeno Reports now support 𝐢𝐧𝐬𝐭𝐚𝐧𝐜𝐞 𝐯𝐢𝐞𝐰𝐬! Display interactive data instances and model outputs directly in your reports. This is especially powerful for media data, such as audio, where users can play the data. Example: hub.zenoml.com/report/cabrera…

We updated the audio transcription and translation reports with 💠 Zeno's new instance views to highlight patterns like model hallucinations. Take a look! Audio transcription report: hub.zenoml.com/report/cabrera… GPT translation report: hub.zenoml.com/report/cabrera…

🗓️ Next week I will be at #CSCW2023 to present Eye into AI 🧿🤖: a game that can help researchers collect human evaluations of explainable AI techniques at scale #XAI (session XAI I) 🌐 Play our game, Eye into AI dig.cmu.edu/eyeintoai/ 📰 Read our paper dl.acm.org/doi/10.1145/36…

📣📣 CMU School of Computer Science PhD students: I'm so excited to share that Charvi Rastogi, Sanae Lotfi, Yaniv Yacoby, and I are bringing this amazing workshop to CMU! Join us to reflect on PhD student mental health, community-building🫂, and academic culture. (1/n) 👇

Thank you SO much to all of you who joined our “User AI Audit” workshop at #CSCW23 (cscw-user-ai-auditing.github.io)! I had a wonderful time interacting with such a diverse and dynamic group of scholars ✨. Please stay tuned for our upcoming blog post synthesizing our workshop insights!

I'll present our work on 𝐛𝐞𝐡𝐚𝐯𝐢𝐨𝐫 𝐝𝐞𝐬𝐜𝐫𝐢𝐩𝐭𝐢𝐨𝐧𝐬 this afternoon at #CSCW2023 in the Human-AI Collaboration I session at 2:30! Looking forward to meeting & chatting with people about AI tooling, evaluation, and collaboration this week 🤖🧑

Sad to miss #ieeevis this year, but I’m very excited about our new paper from PAIR!

🤗 Hugging Face released a 𝐦𝐚𝐬𝐬𝐢𝐯𝐞 update to their OpenLLM leaderboard with 3 new tasks. Using Zeno we explore each task and discuss what this means for the leaderboard's validity. TL;DR it's a 𝐦𝐮𝐜𝐡 better measure of performance. Report: hub.zenoml.com/report/cabrera…

One of the recurring questions in the XAI sessions at #CSCW2023 was about incorrect and unfaithful XAI 📖 Philipp Spitzer and I are excited to share our #CSCW2024 paper on Imperfect XAI (arxiv.org/abs/2307.13566) 👀 So, how does Imperfect XAI impact human-AI decision-making? 1/4

Life update: very happy to be joining Georgia Tech Georgia Tech School of Interactive Computing in the Fall as a PhD student advised by Alex Endert.

So proud of you, Donny! I truly enjoyed mentoring you Oregon State University. Very excited about your new journey GT Visualization!

Very excited to open-source LLM Comparator! This new #visualization tool lets you analyze LLM responses side-by-side. It’s been used for evaluating LLMs Google, and we're proud to release it as part of Google's Responsible GenAI Toolkit. medium.com/people-ai-rese…