Stef Ruinard

🧠 Can you catch an AI’s mistakes by looking inside its head? That’s the idea behind probing: looking at a language model’s “brain” for signals that it might be about to hallucinate or ignore an instruction. While it sounds almost like an AI lie detector, in reality, it’s much less straightforward. Our ML engineers Sanne Wielinga and Jorrit van der Laan tested the idea in two very different settings: healthcare and banking. 🔎 In Jorrit’s experiment, the probe was better at identifying conversations that were more likely to go wrong than predicting a specific mistake. In Sanne’s research, probing caught some errors that other uncertainty checks missed, but worked best when the two were combined. That makes probing less of a hallucination detector and more of a risk signal: something that could help decide when an answer needs an extra check or human review. 👉 In the article, Jorrit and Sanne break down both experiments and where they think probing can be useful today. Read it here: https://lnkd.in/eB9qFxkB #MachineLearning #LLMs #AIResearch #GenerativeAI

11

𝐖𝐡𝐲 𝐌𝐨𝐬𝐭 𝐃𝐚𝐭𝐚 𝐏𝐫𝐨𝐣𝐞𝐜𝐭𝐬 𝐅𝐚𝐢𝐥 𝐨𝐧 𝐌𝐨𝐧𝐝𝐚𝐲 𝐌𝐨𝐫𝐧𝐢𝐧𝐠? Monday morning. Kick-off meeting. Dashboards are ready. Tools are in place. And yet… everyone feels something is off. Most data projects don’t fail because of: ❌ Cloud platforms ❌ Lakehouses ❌ AI tools ❌ Lack of technical skills They fail because governance was never clearly defined. No clear answers to simple (but critical) questions: • Who owns the data? • Who decides priorities? • Who arbitrates when business, IT, and security disagree? • What matters now vs later? When governance is missing, teams improvise. When teams improvise, complexity grows. And complexity kills delivery speed. 👉 𝐆𝐨𝐨𝐝 𝐝𝐚𝐭𝐚 𝐠𝐨𝐯𝐞𝐫𝐧𝐚𝐧𝐜𝐞 𝐢𝐬 𝐧𝐨𝐭 𝐛𝐮𝐫𝐞𝐚𝐮𝐜𝐫𝐚𝐜𝐲. It’s a decision framework that creates clarity, alignment, and momentum. The most successful data initiatives I’ve seen all start the same way: • Clear roles • Clear decision paths • Alignment between business, IT, security, and architecture Data projects don’t need more tools. They need more clarity. This is precisely how I support organizations in structuring their data strategies: aligning business ambitions, governance, and architecture to transform data into a reliable, decision-ready strategic asset. What are your biggest data governance challenges right now? #DataGovernance #EnterpriseArchitecture #DataStrategy #DataLeadership #ITStrategy #DigitalTransformation #ProjectManagement #MondayGovernance

9

2 Comments

From our own TPUs next to offering choice of GPUs, Google fully owns a fully integrated AI stack that gives us unique advantages for our customers. Gartner recognises this, making us the top leader in AI infrastructure. You can try our Agent Platform and experience the difference, even running models other than Gemini.

17

When ‘Irrelevant’ Products Sell: The Paradox of Semantic Search in E-Commerce We recently noticed something curious in our search data. When customers search for “MacBook Pro”, quite a few end up buying a MacBook Air. At first glance, that looks like a failure of relevance. The system retrieved something that doesn’t exactly match the query. But the economic outcome tells a different story: people are finding and buying what they actually want, even if it’s not what they typed. It's possible that they were still in the buying decision process, and we helped them by showing both models in the search results, making it easier to compare. This pattern has become more common as we’ve shifted from strict keyword-based search to semantic/hybrid search. These models are great at understanding intent and similarity: “close enough” products that are, let’s call it, semi-relevant. In the logic of Amazon’s ESCI classification (Exact, Substitute, Complement, Irrelevant), these products would be considered Substitutes or Complements. Now we face a dilemma: - If we filter out these semi-relevant products, we might lose sales and hide options users actually want. - If we keep them, we risk frustrating customers who expect precise, predictable results — feedback we are already receiving. It’s a tension between relevance and revenue, precision and discovery. Maybe the question is not: “Did the search results exactly match the query?” but rather: “Did the user find what they needed?” We’re exploring how to measure this balance better, blending traditional relevance metrics with behavioral and satisfaction data. How do you approach this tradeoff in your search system? René Kriegler Jens Kürsten Andreas Wagner Doug Turnbull Levente Schoenherr Mika Wolle Andreas Wagenmann Marcel Kollmar Richar Gründer

105

23 Comments

Based on what I've seen these past months : yes, totally agree. What shouldn't be underestimate is the data work behind to have a clear data warehouse and semantic layer (or any kind of context layer). The big shift is that there are much more people now that can query data and get insights, so governance is key.

60

3 Comments

Unfortunately for newcomers to the field, many data scientists believe that they are only providing value to a company when they are using the latest and “greatest” tech that comes along. Instead of focusing on the buzz surrounding some new approach catalogued in a seminal whitepaper or advertised heavily in a blog post, a seasoned data scientist realizes that the only thing that really matters is the act of solving problems, regardless of methodology. As exciting as new technology and approaches are, the effectiveness of a data science team is measured in the quality, stability and cost of a solution that they provide.

53

Join me on the 25th of June for a new PyLadies workshop! Feed an AI-agentic beast with proper data. Every one uses LLM's, but how is it that results vary greatly? Is it my prompt, that must be it, right? Or can it be your data?

16