Renzhi He

You want to do Machine Learning on large scale Marketing solutions ? Check below 🚀

3

A common misconception with AI and machine learning: “all of the resources, infrastructure and technology" it takes to get started. With Databricks ML Flow, that’s simply not the case. In the video ⬇️ Kenny Shaevel breaks down how Databricks makes it easier to manage and track your models while ensuring full transparency, traceability, and governance—without the headache. Kenny covers everything from: - Simplifying Machine Learning Implementation - Utilizing AutoML for Quick Data Evaluation - The Importance of Feature Engineering - Leveraging Feature Store for Reusability - Governance and Traceability with the Model Registry - Seamless Integration and Application of Models Excited for more? Kenny will be diving even deeper into Databricks in our upcoming live stream series. Keep an eye out for details coming soon! 👀 #machinelearning #databricks #dataanalytics #MLFlow #AutoML #featurestore #dataengineering #techinnovation

5

Join PyTorch at Snowflake's #DataCloudDevDay on June 6 in San Francisco to learn how to accelerate GenAI performance with Native PyTorch. Register: https://hubs.la/Q02xQStt0

18

Chip Huyen on Twitter today below. I 100% agree. Metric hygiene is system hygiene. LLM-Generated metrics require a higher standard of care than they're currently receiving, and much more scrutiny. Mind the compasses. 🐦  A big issue I see with AI systems is that people aren't spending enough time evaluating their evaluation pipeline. 1. Most teams use more than one metrics (3-7 metrics in general) to evaluate their applications, which is a good practice. However, very few are measuring the correlation between these metrics.If two metrics are perfectly correlated, you probably don't need both of them. If two metrics strongly disagree with each other, either this reveals something important about your system, or your metrics just aren't trustworthy.2. Many (I estimate 60 - 70%?) use AI to evaluate AI responses, with common criteria being conciseness, relevance, coherence, faithfulness, etc. I find AI-as-a-judge very promising, and expect to see more of this approach in the future.AI-as-a-judge scores aren’t deterministic the way classification F1 scores or accuracy are. They depend on the judge's model, the judge's prompt, and the use case. Many AI judges are good, but many are bad.Yet, very few are doing experiments to evaluate their AI judges. Are good responses given better scores? How reproducible the scores are -- if you ask the judge twice, do you get the same score? Is the judge's prompt optimal? Some aren’t even aware of the prompts their applications are using, because they use prompts created by eval tools or by other teams.Also fun fact I learned from a (small) poll yesterday: some teams are spending more money on evaluating responses than on generating responses 🤯 https://lnkd.in/gYnTRZW3

2

We launched structured data retrieval in Bedrock Knowledge Bases last week at re:Invent. While embeddings-based semantic search has worked wonders for unstructured data, we felt we need a new approach for structured data. We needed a way to translate customer natural language queries to an expressive query language that traditional databases already support. https://lnkd.in/eTPStNif #rag #text2sql #aws #bedrock

29

🎉 Big congratulations to our incredible team (Haoyu Chen, Nick Greenquist,Yuxing Wang, Sridevi Maharaj, Ruiyang Wu and many more) for delivering several quarters-long impactful features on the Play Store! 🚀 We’re thrilled to have launched key updates like Search Home, Sports Moment, and WearOS recommendations, all designed to enhance personalization and help billions of users worldwide search and discover content they love. 🌍✨ Check out the Play Store Q3 updates: https://lnkd.in/dn5wx8Mi #GooglePlay #Personalization #Recommendation

74

1 Comment

Building Text-to-SQL from Scratch using DAGs 🛠️🧱 Most existing pre-packaged text-to-SQL modules are not suitable for production - inevitably you’ll have to go in and customize the prompts, and define custom orchestration. An underrated feature of LlamaIndex for advanced LLM developers is query pipelines: the ability to define very custom LLM pipelines as DAGs, letting you mix both prepackaged modules you don’t want to write (e.g. retrievers, SQL integration) as well as custom functions. Here Aruna Withanage shows you how to compose an advanced text-to-SQL pipeline:  1. Index the table schemas via a vector database 2. Retrieve relevant table schemas given a user query 3. Given schemas, convert query to a valid SQL statement 4. Retrieve relevant rows from the SQL database 5. Define a custom response synthesis prompt https://lnkd.in/gkhTJqjt

575

21 Comments

Thats the exact explanation I gave to a group of investors last week. (… it’s just Precision/Recall, attributes vectorization as “embedding”, K-Nearest Neighbor, User Search Intent, DAG, ML algorithms, etc all over again …). LOL and they asked what’s new this time around. And I was like not much, ML is still the backbone, and AI is the inference and actions based on the model. It works better when there is a feedback loop in a closed system. But the current state of the art like ChatGPT is based on scraping the Internet (garbage in, garbage out). Kinda like early search engines. What is a better analogy is that NVIDIA/Snowflake etc are selling tools, aka pickaxes to prospectors. The gold is up to you to find. And the opportunity right now is to monetize industry specific (healthcare, travel, financial services) clean data to build models with (aka a treasure map for the prospectors). From an enterprise perspective, what’s out there today aka generative AI isn’t very useful. Too much hype. To build useful engines will require an effort similar to search revolution 2 decades ago. From

16

3 Comments

Replacements are crucial for online grocery shopping. Check out our latest blog post on how we've leveraged machine learning to generate replacement recommendations for out-of-stock items !! Great work by Ahsaas Bajaj and team 👏

27

2 Comments

ChaosSearch + Databricks 📈 Deliver on the best of Databricks (open Spark-based data lakehouse) and ELK (efficient search, flexible live ingestion, API/UI) via ChaosSearch on Databricks. ChaosSearch, as a Databricks Technology Partner, brings ELK use cases to Databricks ecosystem enabling centralized log and event analytics for observability and security. Learn more: https://lnkd.in/dSAXRgEF

2

I'm excited to share two papers from the Retrieval team at Walmart that were presented at CIKM last week. They're both about improving the relevance of Embedding-Based Retrieval. 𝗥𝗲𝗹𝗲𝘃𝗮𝗻𝗰𝗲 𝗙𝗶𝗹𝘁𝗲𝗿𝗶𝗻𝗴 𝗳𝗼𝗿 𝗘𝗺𝗯𝗲𝗱𝗱𝗶𝗻𝗴-𝗯𝗮𝘀𝗲𝗱 𝗥𝗲𝘁𝗿𝗶𝗲𝘃𝗮𝗹. https://lnkd.in/gNQe7QhA The first is about how to intelligently choose a query-dependent cutoff on the cosine score to remove irrelevant items from the retrieved set. Since cosine scores aren't comparable across queries, we introduce a "Cosine Adapter" to map the scores to interpretable scores. 𝗘𝗻𝗵𝗮𝗻𝗰𝗶𝗻𝗴 𝗥𝗲𝗹𝗲𝘃𝗮𝗻𝗰𝗲 𝗼𝗳 𝗘𝗺𝗯𝗲𝗱𝗱𝗶𝗻𝗴-𝗯𝗮𝘀𝗲𝗱 𝗥𝗲𝘁𝗿𝗶𝗲𝘃𝗮𝗹 𝗮𝘁 𝗪𝗮𝗹𝗺𝗮𝗿𝘁. https://lnkd.in/gdqkc2hR The second is about model training techniques we learned to improve the relevance of embedding models. In particular, we describe how to use a teacher model to provide relevance labels for training Juexin (Joy) Lin Nicholas Rossi, PhD Feng LIU Zhen Yang Sachin Yadav Praveen Reddy Suram Satya Chembolu Prijith Chandran Hrushikesh Mohapatra Alessandro Magnani Ciya Liao

148

2 Comments

Quickly index, search, and visualize your dataset with ApertureDB Docker. Get started in under 10 minutes!

4

Check out our latest blog to see how our Notification Team harnesses the power of Graph Neural Network (GNN) models to create personalized restaurant recommendations for user notifications.

63

1 Comment

Kumo helps data scientists build highly accurate predictive AI models that predict user and customer behavior by combining graph learning over enterprise data and generative AI models trained on public data—all entirely within Snowflake. If you want to create better machine learning models faster with Kumo, try our Snowflake Native App now. 🔗 You can request a free trial at the following link: https://hubs.ly/Q02HgQ1P0. If you need additional support and more details, contact us. We'd love to get in touch with you and help you query the future: https://hubs.ly/Q02HgGvG0. #AI #MachineLearning #PredictiveModels #DataScience #GraphLearning #GenerativeAI #Snowflake #TechInnovation #KumoAI #FutureOfData

9

🚨 Is your data engineering workflow holding you back? 🚨 MotherDuck and Dagster Labs are teaming up to discuss the secrets of modern data engineering. We'll cover: - Crafting an end-to-end data platform with Bluesky data, MotherDuck, and Dagster - Proactively solving problems with "shift-left" testing to catch issues early - Practical automation strategies to streamline your workflow and boost productivity 📅 January 14th ⏰ 9:00 AM PT ☑️ Registration link in the comments

13

1 Comment