Pengfei Hu

"Build in primitives" -Andy Jassy, #Amazon CEO https://lnkd.in/gHMYt7NX Happen to take the time to read this year's Annual investor letter from Andy Jassy. Deep in 2nd half, it talks about how Amazon started thinking about "building in primitives" and how it has been game-changing for them and their customers. Probably old news for some, but eye-opening for me. Definitely a good read!

2

Project update! My application tracking extension is now doing fuzzier matching and injecting new information into LinkedIn (and other sites) to keep me from doubling up on applications. Next up: local storage and bulk import/export operations...

12

2 Comments

Today I'm sharing this 🔥 video from our recent Engineering onsite at Demandbase HQ in SF! https://lnkd.in/eFAFR6QG We were so fortunate to host Zhengxuan Wu and Aryaman Arora, two upcoming AI researchers from Stanford University Department of Computer Science. Their approach to fine tuning, called ReFT, targets the representations emitted by neural net layers rather than the weights of the layers themselves. This makes it very flexible when using the same base model for swapping multiple fine tunes. The memory overhead is also very low for their intervention weights. Another highlight is how powerful it can be. One POC they showed forced the LLM to speak in only emojis exhibiting ReFT's power to control output. Control is crucial when, for example, you want to prevent your bot from offering customers unauthorized refunds!! If you're already familiar with LoRA, it should be pretty simple to use with only a few extra hyperparameters -- details in the video. Here's a link to the repo. Give them a star! https://lnkd.in/eJScNbzK Thanks again for letting us host your first external talk Aryaman and Zen. It's very exciting tech!!

17

4 Comments

Check out the latest in our "Debuggability" blog series! This blog is authored by Kevin Downey Jason Johl and me and talks about how we built a Debuggability Paved Road at Intuit using Ephemeral Containers and Argo Workflows. We'd love to hear your thoughts on the topic. Also, what tools do you build as-a-platform to help debug service, compute and network issues in Kubernetes at scale? #intuit #kubernetes #debug

23

We launched structured data retrieval in Bedrock Knowledge Bases last week at re:Invent. While embeddings-based semantic search has worked wonders for unstructured data, we felt we need a new approach for structured data. We needed a way to translate customer natural language queries to an expressive query language that traditional databases already support. https://lnkd.in/eTPStNif #rag #text2sql #aws #bedrock

29

Come join us tomorrow in SF to learn more about the latest updates to run AI workloads on Google Cloud!

11

1 Comment

Want a preview of what I'm going to be talking about in my keynote for RubyConf? You can catch a glimpse of it here!

10

"Females can act like glue within a team, contributing to the overall success the same as men, while injecting a different perspective." - Jian Cheng, Senior Full Stack Developer at Edge Signal. We had the pleasure of speaking with Jian and other female leaders within our ecosystem in celebration of #IWD2024. With 15+ years of software programming experience, Jian is in a unique situation – being a senior developer, female, Chinese, an immigrant, and currently the only female on a small start-up team that is tasked to build an AI-powered edge computing platform, Edge Signal. Read the article to learn more: https://lnkd.in/eUyBJrVg. #tech #femalesintech #womenintech #edgesignal

31

Over the holidays, I read Gergely Orosz’s book Building Mobile Apps at Scale to see how his insights matched my experience at ClassPass. The first part provides an overview of the challenges that are unique to mobile. One standout quote from Chuck Rossi who worked on a mobile release team at Meta captures one of these challenges: “It was the most terrifying thing to take 10,000 diffs, package it into effectively a bullet, fire that bullet at the horizon and that bullet, once it leaves the barrel, it’s gone. I cannot get it back…” Mistakes are hard to revert. Once a binary is released, it’s out there forever–there’s limited recourse to fix it once it’s out (e.g., expedited review, feature flags). This is why we find it crucial to thoroughly test and monitor for crashes and critical bugs before a full rollout. We achieve this by dogfooding the app with employees, automate end-to-end tests, stage rollouts, monitor and alert on new crashes, and smoke test the app with our QA engineers.

12

From my conversations, I'm noticing different trends between how AI engineers deploy evaluation models (model as a judge) in production monitoring vs application testing. 💡 Value - Monitoring: provides real time alerts of performance or used to preemptively block problematic outputs from surfacing to the user -Testing: regression testing applications to establish baseline performance before pushing to deployment ⚡ Speed - Monitoring: near real time, requires extremely fast token processing - Testing: batch, async, often processing large test sets, lower token processing speed required 🎯 Accuracy - Monitoring: overindex on false positives, better to flag something as problematic and be wrong than to let something slip between the cracks - Testing: high correlation with human graders, be as accurate and robust as possible 💰 Cost - Monitoring: prefer cheaper models, otherwise the additional inference cost per generation will add up exponentially - Testing: willing to take on higher costs, calculation is accurate testing leads to better performing outputs 🛑 Risk - Monitoring: if your monitoring judge gets something wrong, there is a high consequence because it directly impacts the end user - Testing: if your testing judge gets something wrong, async review gives some potential to remediate the issue At the moment, it seems using model as a judge for testing is a little more popular due to the high cost constraints of constant monitoring, but that is subject to change as more specialized models come out. As the industry matures, I strongly believe we're gonna see specialization between models for monitoring and models for testing. If you're currently using model as a judge techniques, would love to chat more! Let me know what you think. Twitter Handle: https://lnkd.in/eAsQvtiu

26

Hey everyone, I am an investor and advisor to Katanemo.... They are reinventing the infrastructure primitives for GenAI and launched Arch - https://lnkd.in/gCrbbkEV - an intelligent prompt gateway. Give the project a look on Github, and star the project if it resonates

88

3 Comments

How you can make your LLM applications better!

5

If you were looking to optimize costs when using generative models on Vertex AI please, the post that I wrote with my colleague Nim (Nimesha) Jayawardena, can be helpful. The post explores how to use Vertex AI’s context caching to reduce the cost of using Gemini models with large, repeated contexts. It shows criteria for using context caching together with the code samples. You can read the post in https://lnkd.in/g6XyRiJy or in Nim's blog on Medium. #vertexai #gemini #google_cloud

4

Thats the exact explanation I gave to a group of investors last week. (… it’s just Precision/Recall, attributes vectorization as “embedding”, K-Nearest Neighbor, User Search Intent, DAG, ML algorithms, etc all over again …). LOL and they asked what’s new this time around. And I was like not much, ML is still the backbone, and AI is the inference and actions based on the model. It works better when there is a feedback loop in a closed system. But the current state of the art like ChatGPT is based on scraping the Internet (garbage in, garbage out). Kinda like early search engines. What is a better analogy is that NVIDIA/Snowflake etc are selling tools, aka pickaxes to prospectors. The gold is up to you to find. And the opportunity right now is to monetize industry specific (healthcare, travel, financial services) clean data to build models with (aka a treasure map for the prospectors). From an enterprise perspective, what’s out there today aka generative AI isn’t very useful. Too much hype. To build useful engines will require an effort similar to search revolution 2 decades ago. From

16

3 Comments

The challenges of scaling and recovering AWS MSK clusters are now effectively mitigated, representing a significant advancement in cluster management.

10

Highly recommended book...

9

Meet the StarTree All-Stars: Yupeng Fu 🌟 As a Principal Engineer at Uber, Yupeng leads the real-time data platform and search platform at Uber. He is an Apache Pinot PMC and has been an essential part of its community for many years. Learn more about Yupeng and let us know where his favorite movie quote comes from: 𝗤: 𝗪𝗵𝗮𝘁'𝘀 𝘆𝗼𝘂𝗿 𝗳𝗮𝘃𝗼𝗿𝗶𝘁𝗲 𝗺𝗼𝘃𝗶𝗲 𝗾𝘂𝗼𝘁𝗲? A: “Don’t chase success, chase excellence and success will follow” 𝗤: 𝗪𝗵𝗮𝘁’𝘀 𝘆𝗼𝘂𝗿 𝗳𝗮𝘃𝗼𝗿𝗶𝘁𝗲 𝗣𝗶𝗻𝗼𝘁 𝗳𝗲𝗮𝘁𝘂𝗿𝗲 𝗮𝗻𝗱 𝘄𝗵𝘆? A: Upsert. Fast analytics with upsert over fresh data is a powerful feature that differentiates Pinot with other many OLAP stores 𝗤: 𝗪𝗵𝗮𝘁 𝗶𝘀 𝗮 𝗰𝗼𝗺𝗺𝘂𝗻𝗶𝘁𝘆 𝗽𝗿𝗼𝗷𝗲𝗰𝘁 𝗼𝗿 𝗰𝗼𝗻𝘁𝗿𝗶𝗯𝘂𝘁𝗶𝗼𝗻 𝘆𝗼𝘂’𝗿𝗲 𝗽𝗿𝗼𝘂𝗱 𝗼𝗳? A: Upsert. Upsert is technically challenging, and I’m proud of the innovative solution developed by Uber and StarTree. It exemplifies the power of collaboration between companies within the community. 𝗤: 𝗪𝗵𝗮𝘁'𝘀 𝘀𝗼𝗺𝗲 𝗮𝗱𝘃𝗶𝗰𝗲 𝗳𝗼𝗿 𝗱𝗲𝘃𝗲𝗹𝗼𝗽𝗲𝗿𝘀 𝗹𝗼𝗼𝗸𝗶𝗻𝗴 𝘁𝗼 𝗺𝗮𝗸𝗲 𝗮𝗻 𝗶𝗺𝗽𝗮𝗰𝘁 𝗶𝗻 𝘁𝗵𝗲 𝗰𝗼𝗺𝗺𝘂𝗻𝗶𝘁𝘆? A: Simplicity is a great virtue but it requires hard work to achieve it and education to appreciate it. #StarTree #Community #ApachePinot

106

2 Comments

A data-driven approach to driving wound healing outcomes Marcella Gomez, University of California, Santa Cruz https://lnkd.in/gAkWZYJT Meet: https://lnkd.in/g4AAWkcK YouTube Stream: https://lnkd.in/gwtqaTmf Talk details: https://lnkd.in/gWn3B3cU Subscribe for talk announcements: https://lnkd.in/g5ciuNuX Abstract: Precision medicine requires an ability to predict the response of an individual to a prescribed treatment regimen a priori. Thus, advancement in the field is challenged by a lack of predictive models and, arguably, a lack of time-series information for a highly dynamic system. Here, we discuss work in wound healing, for accelerating wound closure. We argue that timing of treatments is as critical to consider as the choice of drug or therapy. Due to system size and complexity, data-driven methods need to be explored to develop multi-dimensional quantifiable indicators tracking systemic changes. In this work I discuss how bioelectronic devices enhanced with deep learning can help facilitate real-time sensing and actuation for automated decisions in treatment for wound healing and preliminary work in transcriptomic based classification of wound states. Bio: Marcella M. Gomez is an associate professor at UC Santa Cruz in the department of Applied Mathematics and Associate Dean for Diversity, Equity, and Inclusion for Baskin Engineering. She received her PhD from Caltech in 2015 and a B.S. from UC Berkeley in 2009; both degrees in Mechanical Engineering. Her research interests are in the broad field of bio-control leveraging methods in machine learning and control theory. Applications range from controlling single-cell response to driving complex systems such as wound healing. #modeling #simulation #medicine #healing #trauma #predictivemodeling #datascience #data #woundhealing #biology #bioengineering #systemanalysis #sensing #omics

3

💥 We recently announced Temporal Nexus - the feature that connects durable executions across team, namespace, region, and cloud boundaries. Nexus endpoints are API proxies for Nexus services that abstract Temporal primitives like Workflows for seamless integration and collaboration across teams 🤝 
 
Benefits for developers: - Familiar API programming model that supports both sync and async patterns for connecting durable executions using existing queue-based Workers. - Modular architecture that allows teams to independently develop, scale, and maintain parts of an application with clear API contracts backed by robust Nexus machinery. - Enhanced security, troubleshooting, and fault isolation that reduces the risk of misbehaving workers, since each team can run their workflows in separate namespaces. Use the Temporal SDK to add Nexus services to your existing workers with clean API contracts for others to use. Read more: http://temporal.io/nexus

111

2 Comments