Jialu Zhu (朱嘉璐)

🎉 Big congratulations to our incredible team (Haoyu Chen, Nick Greenquist,Yuxing Wang, Sridevi Maharaj, Ruiyang Wu and many more) for delivering several quarters-long impactful features on the Play Store! 🚀 We’re thrilled to have launched key updates like Search Home, Sports Moment, and WearOS recommendations, all designed to enhance personalization and help billions of users worldwide search and discover content they love. 🌍✨ Check out the Play Store Q3 updates: https://lnkd.in/dn5wx8Mi #GooglePlay #Personalization #Recommendation

74

1 Comment

Come join us tomorrow in SF to learn more about the latest updates to run AI workloads on Google Cloud!

11

1 Comment

Chip Huyen on Twitter today below. I 100% agree. Metric hygiene is system hygiene. LLM-Generated metrics require a higher standard of care than they're currently receiving, and much more scrutiny. Mind the compasses. 🐦  A big issue I see with AI systems is that people aren't spending enough time evaluating their evaluation pipeline. 1. Most teams use more than one metrics (3-7 metrics in general) to evaluate their applications, which is a good practice. However, very few are measuring the correlation between these metrics.If two metrics are perfectly correlated, you probably don't need both of them. If two metrics strongly disagree with each other, either this reveals something important about your system, or your metrics just aren't trustworthy.2. Many (I estimate 60 - 70%?) use AI to evaluate AI responses, with common criteria being conciseness, relevance, coherence, faithfulness, etc. I find AI-as-a-judge very promising, and expect to see more of this approach in the future.AI-as-a-judge scores aren’t deterministic the way classification F1 scores or accuracy are. They depend on the judge's model, the judge's prompt, and the use case. Many AI judges are good, but many are bad.Yet, very few are doing experiments to evaluate their AI judges. Are good responses given better scores? How reproducible the scores are -- if you ask the judge twice, do you get the same score? Is the judge's prompt optimal? Some aren’t even aware of the prompts their applications are using, because they use prompts created by eval tools or by other teams.Also fun fact I learned from a (small) poll yesterday: some teams are spending more money on evaluating responses than on generating responses 🤯 https://lnkd.in/gYnTRZW3

2

Want a preview of what I'm going to be talking about in my keynote for RubyConf? You can catch a glimpse of it here!

10

Check out the latest in our "Debuggability" blog series! This blog is authored by Kevin Downey Jason Johl and me and talks about how we built a Debuggability Paved Road at Intuit using Ephemeral Containers and Argo Workflows. We'd love to hear your thoughts on the topic. Also, what tools do you build as-a-platform to help debug service, compute and network issues in Kubernetes at scale? #intuit #kubernetes #debug

23

What does it feel like to be an engineering manager at Amazon: Since the RTO announcement it became popular to write shit about Amazon, but it's still one of the best companies around the world. A few weeks ago, Gilad Naor shared with the 'Leading Developers' readers 3 lessons from his time as an engineering manager at Amazon. Here is the first one, read the full article here: https://lnkd.in/dHBxv2Ha 𝟭. 𝗦𝗼𝗹𝘃𝗲 𝗽𝗿𝗼𝗯𝗹𝗲𝗺𝘀 𝘄𝗶𝘁𝗵 𝗺𝗲𝗰𝗵𝗮𝗻𝗶𝘀𝗺𝘀 My first meeting at Amazon was baffling. I entered a room with about a dozen Software Development Managers (SDMs) and Product Managers (PMs). Everyone sat in silence and looked at their laptops or the printed papers they had before them. It was my second day at Amazon, and I picked up one of the extra sheets and started reading. After those first ten awkward minutes, everyone finished reading and the meeting began. Why do Amazon meetings always start this way? This is a specific example of how Amazon approaches problem-solving. It is the "Good Intentions Don't Matter" approach. In a famous internal video, Jeff Bezos talks about why good intentions are irrelevant. When things don't work, it does not matter what people wanted. All that matters is fixing the problems with mechanisms. For example, many busy managers come to meetings without first reading the prep material. They all intended to read the 6-Pager before the meeting. They just got sidetracked by the latest emergency. Scolding the managers won't solve the problem. A mechanism will. This is why all Amazon meetings start with a study period. The remaining meeting time is then spent on high-quality decision-making. Meta solves the same problem by creating cultural norms. I can't count the number of times that I heard a senior leader cut off the presenter: "Stop, assume that everyone here already read the slides. Let's get to the point." This is a culture-first approach. I have seen this work well in some teams, and not work at all in other teams. At Amazon, the solution is part of the company's "operating system". ------------------------- Anton here - by now everyone is familiar with the 6-pager concept, but that "operating system" thinking can be used almost anywhere. Think about something annoying that your developers do, and how can you solve it. For example, I remember being annoyed that people don’t update the Jira tickets. Finally, after requesting multiple times, now we just take 20 seconds from the standup to update them.

56

8 Comments

Sharing my thoughts on How Waymo (formerly known as the Google Self-Driving Car Project) is Pioneering Autonomous Driving with AI. Waymo has become a trailblazer in self-driving technology, and at the core of it all is AI. Leveraging advanced machine learning and deep neural networks, Waymo’s AI-powered system continuously improves through real-world data, making autonomous vehicles smarter, safer, and more reliable. Perception: AI allows Waymo’s cars to "see" and understand their surroundings by processing data from sensors like cameras, LIDAR, and radar. This helps detect pedestrians, other vehicles, and objects in real time. Prediction: The AI system predicts the behavior of everyone around the vehicle—whether it’s cars changing lanes or pedestrians crossing the street—ensuring smooth and safe driving decisions. Planning & Control: AI makes split-second decisions about accelerating, braking, and steering based on the analysis of complex traffic patterns, enabling Waymo vehicles to safely navigate roads. With millions of autonomous miles driven, Waymo’s AI is constantly learning and improving, helping create the future of safer and smarter transportation. The road ahead is AI-driven! www.rxsmeta.com #AI #AutonomousVehicles #Waymo #SelfDrivingCars #MachineLearning #SmartMobility #TransportationTech #FutureOfTransport #AIinAction #Innovation

2

One of the best things about being in San Francisco 🌉 is the ability to drop in on great talks. Last week I had a chance to see NVIDIA's CEO Jensen Huang (JH) being interviewed by Stripe's CEO Patrick Collison. Jensen is among the longest-serving founder-CEOs in Silicon Valley at the moment (30+ years). With that comes a special clarity of mind and some hot takes. 🌶👇 🗣 Jensen doesn't believe in 'praise in public and criticism in private'. Instead, he reasons that most information in the organization should not be privy to certain people. By giving feedback in public, you allow everyone to learn. 'What is better than learning from your own mistakes? It's learning from someone else's mistakes!!' 🔥 JH also doesn't like to fire people. He would rather "torture them into greatness". He invests in people and he doesn't like to write off his investments. 👨‍💻 Asked whether he likes his job every day, he answers "no". The perfect job doesn't give you happiness 100% of the time. "No great things were done, that were easy all the way." ... "Even if I come home from a day where I did things I didn't enjoy, it could still be have been a great day." 👩‍👧‍👦 Jensen famously has 60 direct reports. "This allowed us to cut out probably 7 levels of management". His subordinates don't necessarily have such large teams. "It doesn't scale down, less senior people need more guidance". Looking at the numbers, this approach allowed NVidia to reach ~2T market cap with ~28.000 people vs. Microsoft's 3T with ~220.000 people and Apple's 2.6T with 160.000 people. ❔ Jensen answers almost every question in the same way: "Let's break it down, let's reason about it". This is a great response on so many levels. First, it allows him to not be put on the spot. Second, it allows him to teach his workforce how he reasons through a problem so that they can emulate it and create alignment. Third, it gives people a chance to point out flaws in his reasoning. He admits he is not right all the time. ON AI 👨‍🔧 "You will not lose your job to AI, your company will go out of business because another company used AI." "If you're not using AI you're doing it wrong." ⏱ The extent to which AI will enter many processes is largely underestimated. Right now we are focused on language - sounds in a time sequence. "Any time sequence can be tokenized." This will include robotic movement. 🥛 JH also expects that we won't be satisfied with foundation models "You're going to want to tune it into perfection because you care so deeply. Say if it's already at 99% you'll want to get it to say 99.3%" This last piece is also why we created Query Vary (YC W22). AI will never feel truly intelligent until it completely understands your data and context. It needs to be taught, regardless of how powerful the foundation model is. Finally, besides insights, it was remarkable how funny Jensen was, cracking a joke at roughly a 1 per 3 min rate. It was a very entertaining talk. #ai #sf

61

8 Comments

I found this post pretty insightful (in terms of qualifying the differences), though I think it misses what the difference actually is When you're interviewing someone for 45 minutes, what kind of data can you get? Can you really conclude this person is going to be a good leader? Or are you simply measuring how well the person can bullshit? Coding & problem solving, on the other hand, cannot be bullshit They are a good predictor & signal of actual real-world performance (at least, how good of a coder & problem solver the person is) Whereas if you interview someone & they "seem to be a good leader", what you're actually probably measuring is that they're a really good bullshitter If I had to assign probabilities: A) Probability that a "leadership interview" leads to a hire which is: A good bullshitter: 90% Competent: 20% Hardworking: 20% A good leader: 20% B) Probability a technical/problem-solving interview leads to a hire which is: Competent: 90% Hardworking: 50% A good leader: 20% A good bullshitter: 20% I'll take (B) any day

My team at Meta (In-Feed Recommendations) is hiring a number of ML engineers in the US We are building one of the most advanced recommendation engine in the industry to power discovery experiences on Facebook newsfeed with a diverse set of public content (Reels, videos, photos, web links, conversations, etc) available on our platform. You can make a big difference by working on a wide range of projects, including but not limited to user/content interests understanding, next-gen recommendation systems and modeling technologies, new social discovery experiences to help create communities of like-minded people. Check out the following links to learn more: The AI Behind Unconnected Content Recommendations on Facebook and Instagram https://lnkd.in/gZ7XV_am Meta Rethinks the Philosophy of the Facebook Feed https://lnkd.in/ggJzk3g3 If interested, please DM me. ==================== Team / Role Summary We are part of the FB Feed Relevance Org, and our team’s mission is to create joyful discoveries and meaningful connections via personalized recommendations on people’s home feed with deep understanding of their interests. We are responsible for building the E2E recommendation system to power all the interest-based public content discoveries across FB Feed. Why is This Impactful IFR is a central product in Facebook’s long term vision, Discovery Engine. Each day we are sending more than 50B posts to our users. From Mark Zuckerberg's Q2 earnings call: "AI-recommended content from accounts you don't follow is now the fastest growing category of content on Facebook's feed. Since introducing these recommendations, they’ve driven a 7% increase in overall time spent on the platform. This improves the experience because you can now discover things you might not have otherwise followed or come across." Your Role As an ML Engineer, you will help build one of the world’s largest recommendation engine, implement state of the art ML algorithms and modeling techniques, directly influence billions of users’ News Feed Experience. You will also learn how to build an ultra large scale Recommender System from end to end and grow your career as a domain expert, and lead our efforts to build emerging new experiences on the Newsfeed Recommendations platform.

64

1 Comment

How you can make your LLM applications better!

5

Many folks often say that LLMs are just pattern matching… and contrary to some, I would actually agree with that sentiment! However, I really don’t think that scaling LLMs alone will help us breakthrough the next intelligence plateau. Surely, bigger models running faster and faster on smaller hardware will help.. but let’s take a moment to draw a comparison to human cognition and the brain 🧠 Pattern matching is the essence of reasoning but take the human brain for example - even though pattern matching is inherent to everything our brain does, we still have dedicated “modules” for various functions on top of pattern matching. And this is exactly the kind of system design we’re heading towards with agentic systems and cognitive architecture! Let's compare... > Prefrontal Cortex: Helps in analyzing trades offs, long-term consequences and planning complex actions (LLM Compiler/LDB, Self-Discover) > Hippocampus: Involved in forming new memories and integrating past experiences (stored as patterns) into current reasoning processes (GraphRAG + LGGMs + CLIN and other long-term memory systems for agents that use LLMs to pre-process, cluster and route to certain memories) > Basal Ganglia: Contributes to pattern recognition and procedural learning, helping automate certain processes based on recognized patterns (Automatic routing to SLMs, deterministic workflows and non-deterministic action planning) > Parietal Cortex: Integrates sensory information from different modalities, helping with spatial reasoning and the understanding of numerical patterns (multi-modal models, tools, environmental grounding and routing to prompting techniques specialized for math or other more nuanced capabilities) > Anterior Cingulate Cortex: Monitors conflict, makes decisions when there are competing patterns or information and adjusts strategies (Continuously learning models and systems inspired by CLIN and MedAgent-Zero + guardrails) > Default Mode Network: Active when the brain is at rest but also plays a role in reflective thought, daydreaming and consolidation of patterns into larger conceptual frameworks (Layered graph-of-thought consolidation in clusters / abstractions of thought with MCTS) I'm no neuroscientist so take what I say with a grain of salt... but I do think of LLM exploration of latent space and pattern matching as an essential substrate. However, it's the interplay between pattern matching and other cognitive processes that leads to authentic reasoning ✨ In 1-2 years time with enough speed + compute, these post training optimization techniques with environmentally grounded agentic systems will be what brings about authentic, human-like reasoning. And once these reasoning traces and agent trajectories get baked into the underlying models, this will only get better and better…

4

2 Comments

Three insights from the TEDAI San Francisco X Factory X AI at Meta CodeGen Hackathon. Had the pleasure of judging a fantastic hackathon in SF yesterday. Here's the biggest things I took away from iteration on the cutting edge of GenAI for code: 1) 𝙂𝙚𝙣𝘼𝙄 𝙘𝙤𝙙𝙞𝙣𝙜 𝙨𝙩𝙖𝙧𝙩𝙚𝙙 𝙬𝙞𝙩𝙝 𝙖𝙘𝙩𝙞𝙫𝙚𝙡𝙮 𝙖𝙨𝙨𝙞𝙨𝙩𝙞𝙣𝙜 𝙙𝙚𝙫𝙨 𝙖𝙩 𝚠̲𝚛̲𝚒̲𝚝̲𝚎̲ 𝙩𝙞𝙢𝙚, 𝙗𝙪𝙩 𝙂𝙚𝙣𝘼𝙄 𝙖𝙜𝙚𝙣𝙩𝙨 𝙣𝙖𝙩𝙪𝙧𝙖𝙡𝙡𝙮 𝙛𝙞𝙩 𝙖𝙩 𝙙𝙞𝙛𝙛 𝚛̲𝚎̲𝚟̲𝚒̲𝚎̲𝚠̲ 𝙩𝙞𝙢𝙚. Iterative agenic workflows can run at review with automated vetting of generations, and only serve edit suggestions shown to improve a tested outcome (i.e. memory efficiency). Bonus is that this can scale with inference time compute to try more or less variations or revision cycles. 2) '𝘾𝙤𝙙𝙚' 𝙖𝙨 𝙖𝙣 𝙖𝙗𝙨𝙩𝙧𝙖𝙘𝙩 𝙘𝙤𝙣𝙘𝙚𝙥𝙩 𝙤𝙥𝙚𝙣𝙨 𝙖 𝙬𝙤𝙧𝙡𝙙 𝙤𝙛 𝙥𝙤𝙨𝙨𝙞𝙗𝙞𝙡𝙞𝙩𝙞𝙚𝙨 𝙛𝙤𝙧 𝙣𝙤𝙣-𝙨𝙤𝙛𝙩𝙬𝙖𝙧𝙚-𝙙𝙚𝙫 𝙚𝙣𝙜𝙞𝙣𝙚𝙚𝙧𝙞𝙣𝙜 𝙙𝙞𝙨𝙘𝙞𝙥𝙡𝙞𝙣𝙚𝙨. Many tools in these areas involve cumbersome UI's that ultimately translate points and clicks into complex config files, and LLM's are pretty good at editing these underlying files directly. 3) 𝘾𝙤𝙙𝙞𝙣𝙜 𝙖𝙗𝙞𝙡𝙞𝙩𝙞𝙚𝙨 𝙞𝙣 𝙢𝙤𝙙𝙚𝙡𝙨 𝙖𝙧𝙚 𝙜𝙚𝙩𝙩𝙞𝙣𝙜 𝙢𝙤𝙧𝙚 𝙘𝙤𝙢𝙥𝙖𝙘𝙩 𝙩𝙝𝙖𝙣 𝙚𝙫𝙚𝙧. One hackathon team was able to use a combination of synthetic data creation/refinement and model distillation (in this case from my near and dear Llama 405B) to fit highly advanced code translation ability in a 14B model. I have a feeling this trend will continue, which will open up more inference compute heavy coding opportunities like broad repo summarization. Bonus here - the resulting model was open sourced by Jason Lu: https://lnkd.in/gswMbCR3 Shout out to SHACK15 and Cerebral Valley for hosting. Incredible views and vibes.

19

3 Comments

I can't believe the key takeaway I had for middle and high school girl students this weekend in a programming camp was something even expert software engineers forget sometimes: "software engineering is much more than coding". I had the chance to talk to these young bright minds at Santa Clara university. I explained, hopefully in an intuitive way, how humans coordinate at large scale to build products like YouTube. And mainly I wanted to bust the myth that software engineering is a job where you code heads down all day and don't need to talk to people. Three main qualities I highlighted in a good engineer: - knows their users - team player - problem solver That made me think though. The meetings and all those office hours, and 1:1s spent "socializing", while on the surface seem like unproductive hours, they're indeed my job now. And that's true of anybody who's growing up the ladder in organizations. Human coordination is a beautiful problem :)

139

3 Comments

Waymo is probably both more expensive and longer waits on average than Uber or Lyft today, but they are close enough, and rapidly getting better. These markets will flip to fully autonomous when the tech is ready.

2

🍜Anybody up for some ramen? I am excited to share that I am back in the garage, building again (it’s actually a house in the Berkeley Hills). We are taking the wraps off of Supercog AI, a new startup focused on solving Application Integration using LLM powered agents. My co-founder in this new venture is a great friend of mine (and a former co-founder partner), Alex Osborne. Alex and I have worked around the application integration space for a long time, and we are unrealistically excited about applying the GenAI stack to this problem. Not many people relish the task to get some bits to move from system A to system B. But it’s a critical job to be done, and that’s led to a lot of brittle scripts, a myriad of inflexible “no code” tools, and a lot of hand crafted SQL. But the power of the Large Language Model, trained on a huge corpus of information, offers a remarkable tool to solve this problem. The LLM you use today already knows the APIs of hundreds of popular systems. It knows the SQL dialects of every major database. It understands the semantics and data schemas of many popular SaaS systems. It understands much of the specific domain in which your business operates. I like to call the AI revolution “the last platform shift”. That may turn out to be hyperbole, but I definitely believe we are in the very early stages of seeing what this new stack can enable. I know we are hardly the first folks to claim that “amazing stuff is coming!”. But this is why Alex and I have decided to focus on a real and hard problem. If we can prove that LLMs can power a new way to solve this problem, it will be the first successful *new* approach in 15 years.. We will have much more to share about this new platform soon. In the meantime, if you’re interested in getting a peek - or even better if you have application or data integration tasks that you would like some help with - please reach out because we want to help. #genai #startups #backinthegarage

164

37 Comments