Yang Liu

Last week, Steven Astorino challenged Tanmay Bakshi and I to demonstrate building and deploying an application from scratch at the IBM Canada Townhall. We made this possible by leveraging watsonx Code Assistant to accelerate our workflow, allowing us to focus on the inventive components of software development. Since the Grammy Awards are coming up, we decided to build a polling platform where the audience votes for nominees in each of the major categories so we could determine IBM Canada's predictions for the outcomes. watsonx Code Assistant helped us write boilerplate code, SQL queries, data structure transformation logic, and more! As a result, we had a fully deployed application in ~15 minutes and collected over 1400 votes from attendees. Turns out, IBMers share my opinion that Taylor Swift will win Album of the Year 😊

187

8 Comments

Yesterday, I had the chance to attend a presentation on Apple Intelligence at Apple Montreal’s office. First of, thanks to Apple for hosting such a nice event. It was a good recap of what’s to come. While I don’t think it showcased anything beyond what was already covered in the WWDC24 videos, it gave everyone the opportunity to dive deeper into Apple Intelligence, which promises to be the biggest leap forward for the Apple ecosystem in years. Here are my main takeaways: - Find app-specific content from anywhere on your device: The potential here is immense. With new capabilities to index app content through AppIntents, Siri and Spotlight can now find app-specific content during a generic search on the device. For instance, we could make specific products or categories easily accessible through a Spotlight search or Siri query. While there may be limitations on the amount of data we can index on the device, this is still a very promising feature. - Siri awareness of on-screen content: Siri will now understand the context of what’s displayed on screen. Combined with the ability to make our app-specific content transferable (Apple’s method for serializing content to popular formats such as PDF, image, or text), this opens up possibilities for app content to interact with other apps through Siri. Although initially limited to a few domains like photos and emails, the potential is exciting! - Easier than ever to deploy on-device AI models: To power Apple Intelligence, Apple has optimized its hardware for on-device AI operations. They are also simplifying the deployment of machine-learning models for tasks related to vision, speech recognition, natural language processing, and sound recognition. Tools such as Create ML make it straightforward to train models with our own datasets. Additionally, we can still import existing models to Core ML from popular sources like Hugging Face or those built with TensorFlow or PyTorch. As I mentioned in my previous post about WWDC24, I’m excited both as a builder and as a user. Let’s build!

24

8 years ago, I started at Google, drawn to the job and mission of supporting customers, building cloud architecture, and creating tools. Back then, our team of 35 focused on "Cloud Platform products like App Engine and Compute Engine". Funny enough, there was no mention of Kubernetes or GKE, which is now a cornerstone of cloud architecture and my specialization! My Googleversary always brings reflection, especially after the intensity of the last few years. Each year feels like a checkpoint; an opportunity to analyze what went well so far and how to build on it. I thought it would be really cool to have someone summarize for me the last 8 years working for Google, my team, and shed some light on what does a Solutions Engineer exactly do? I turned to Google's new notebookLM for this. notebookLM is a web app similar to Google Docs, but it ingests documents, sources, and videos, allowing you to ask questions in natural language. I fed notebookLM, my resume, recognitions, awards,  assessments (including growth feedback), conference notes, presentations, product requirement docs, trip reports, ~8 years worth of performance evaluation and a few rants ;-). notebookLM has a podcast feature that creates conversations from your uploaded documents. You can give it guidelines, but you can't heavily influence the output. I asked it to stop using internal code names, customers, partners, and other names. I also had to cut sections where it tried too hard, "the biggest company in X sector in Y region", that's just too revealing. The summaries were surprisingly insightful. Sometimes, it mentioned things I'd forgotten, which I then fact-checked. notebookLM responds with the exact section in the document where it found something. I appreciated this perspective on my contributions, especially the balance it showed between my technical skills and leadership, training and developing my team. This gave me valuable insights into my own strengths and how I contribute to my team. I must also say a lot of these achievements can't happen without the fantastic teammates, partner teams, customers etc that help us build the cloud community. Incredibly blessed to have spent the last 8 years doing this, and very thankful for all of it. In the end notebookLM gave me a beautiful product, but it's also not lost on me the times i've stumbled, wasn't too sure if I was focusing on the things that would ultimately make my team successful. There was a broadcast/podcast that was talking about a sport team and saying "having pressure is a privilege"...  and in the moment, I myself may not fully believe that, but it's a sentiment I'd love to embody someday! Very thankful for the 8 years of growth. *If you're an AI practitioner, and you have an idea on how to take this audio and generate something cool with it! shoot me ideas! Here's the podcast summary it generated! https://lnkd.in/gfH4zwQQ

155

7 Comments

ZGC: Generational Mode is proposed to be the default ZGC mode for JDK 23 :-) Highly recommending you try ZGC!       https://lnkd.in/gAJC6GYY #zgc #java #openJDK #GC

34

1 Comment

AI tools like Amazon Q Developer are evolving rapidly, with new features and capabilities reshaping how we approach software development. But it’s not all about exponential progress - there are nuanced stories and limitations worth exploring. Here are some of the upcoming features for Amazon Q Developer: - Gradle support, expected in Q1. - Java 21 compatibility, also slated for Q1. What’s more interesting is the overall approach Amazon Q Developer takes. Each action, like "transform" , is handled by a dedicated agent designed specifically for that task. General-purpose agents just don’t cut it. The limitations imposed on these agents are equally fascinating. To avoid shortcuts that might create bigger problems, agents are given a restricted toolset. For example, the transform agent doesn’t have bash access. Why? Because instead of carefully resolving issues with Java library upgrades, it might just delete files to "fix" things. And yet, even with months of development and fine-tuning, these specialized agents deliver only 10-50% time savings. The complexity of the codebase or the task itself heavily influences their effectiveness. The more complex the work, the more human oversight and manual intervention are required to ensure success (or even getting it to build). Now, before the AI bros jump in with "this will get exponentially better," let’s inject some realism. Progress doesn’t come for free. As we’ve seen with o3, scaling these agents up for more complex scenarios gets significantly more expensive. Handling something as intricate as a legacy migration is far more challenging than churning out the millionth iteration of the same app. The cost and time investment for building highly specialized agents don’t magically disappear. Looking ahead to 2025, we’ll undoubtedly see more agents emerge, but they’ll be increasingly niche and take significant effort to create to be useful. Even then, humans will remain an essential part of the equation, especially for tasks that go beyond repetitive, predictable workflows. AI can assist, but it’s not about to replace the need for expertise and adaptability in complex scenarios. Let’s celebrate the progress, but let’s also keep it grounded in reality.

3

So I became fascinated by C++ coroutines late last week, and have spent a good portion of the weekend figuring out how they work. The TL;DR of my discoveries so far is that C++20 coroutines: 1) follow the best and the worst of C++ practices, and 2) are a really clever way to decouple stackless functions from their execution model. For (1), well, it's C++. No crisp & clean syntax, lots of weird compilation errors, and even weirder runtime errors. I hated them with passion for the first several hours. But then they grew on me, because the flip side is also true. The flip side is backwards compatibility and "only pay for what you use". And introducing a totally new concept — the stackless execution model, effectively, [proto-] green threads — into a mature and complex language is one hell of a pain! For (2), I do believe it's a genius solution for the language to separate stackless functions from their execution model. And C++20 did exactly that. There is literally no event loop in C++20! The claim (as I understand it) is that a tiny microcontroller may want to use C++20 coroutines, and the authors of the Standard are very explicit about not constraining possible runtimes. All that C++20 coroutines do is introduce stackless functions into the language as first-class citizens. And yes, stackless immediately implies resumable. So it is one step about the most-clever-possible syntactic sugar over `setjmp()`. Full one step. But not a millimeter more than this one step. Whether you "execute" your coroutines in a dedicated single-threaded event loop, or a thread-pool-based event loop, or by individual steps via some timer event or even a network packet — that is entirely your call. The stackless functions are there, `co_await` (as well as `co_yield` and `co_return`) is there, no callback hell is needed, and all the RAII and scoped initialization/ownership model is fully intact. Now, I'm still elbows deep in my experiments, but here's the first big LinkedIn-worthy question. Why don't we write our C++20 [unit] tests as corourines? Say, you want to limit some test to five seconds. And fail it as "timed out" within these five seconds. Before coroutines, doing this without instrumenting the very code of the test involved `fork()`-ing the process, effectively relying on the operating system's primitives to ensure proper resource deallocation. With C++20 coroutines, as long as the slow test is slow on non-blocking I/O (i.e. not in an infinite loop, but on something that `co_await`-s stuff), it is possible to time-box each individual test to some five seconds very much organically. Because a coroutine frame, unlike a stack frame, can be `.destroy()`-ed safely and cleanly. So, are we doing it yet? If no, why not? If yes, what would be some next-gen `gtest` to look into? Also, I'll be writing some code and blog posts on C++20 coroutines soon. If you are reading this text and are an expert, DM me pls — I'd love to pick your brain!

19

4 Comments

For the first time I am happy about certificate which I received today that remind me the new learning, foundation for next year, and the hours I spent over the weekend for the 10 weeks. Little bit about course: NLP with deep learning The best part is the hands on exercises where we learn the foundations of deep learning, neural networks and the NLP. But it will be useful as a starting point and set the direction in that path.

50

10 Comments

Meet AGUVIS: A pure vision-based framework for autonomous Computer/Mobile-use GUI agents, operating seamlessly across web, desktop, and mobile platforms without UI code. Key Features & Contributions - Pure Vision Framework: First fully autonomous pure vision GUI agent capable of performing tasks independently without relying on closed-source models - Cross-Platform Unification: Unified action space and plugin system that works consistently across different GUI environments - Comprehensive Dataset: Large-scale dataset of GUI agent trajectories with multimodal grounding and reasoning - Two-Stage Training: Novel training pipeline focusing on GUI grounding followed by planning and reasoning - Inner Monologue: Explicit planning and reasoning capabilities integrated into the model training - Generalization to OOD (Out-of-Distribution): AGUVIS demonstrates impressive generalization on OSWorld benchmark even without desktop trajectory data during training. Project Page: https://lnkd.in/guzkwpbB Paper: https://lnkd.in/gs7wXyJS GitHub: https://lnkd.in/gWeSkFmK Collaboratiion among Yiheng Xu Zekun Wang Junli Wang Dunjie Lu Tianbao Xie Amrita Saha Doyen Sahoo Tao Yu Caiming Xiong

156

6 Comments

To measure what I call Total Factor Reliability, we don't just ask: is Temporal robust? We must ask: do our *users* write robust Workflows? These improvements to Signals and Updates will help you ensure the latter. My blog, Robust Message Handlers, explains. We crafted them with love. Hope you enjoy!

7

1 Comment

Leverage StarCoder 2, a powerful LLM for code generation, completion, summarization, and documentation. Try the now available NVIDIA-hosted StarCoder 2 API, packaged as NIM inference microservice for free from the NVIDIA API catalog. https://bit.ly/3y8fyvB

1

I am excited to share the latest blog, co-authored by myself, Junhwi Kim and Vyas Sekar, detailing how we managed to successfully localized and improved LLM by combining Fine-Tuning and RAG. https://lnkd.in/gKDkgGPq

70

3 Comments

Check out the latest in our "Debuggability" blog series! This blog is authored by Kevin Downey Jason Johl and me and talks about how we built a Debuggability Paved Road at Intuit using Ephemeral Containers and Argo Workflows. We'd love to hear your thoughts on the topic. Also, what tools do you build as-a-platform to help debug service, compute and network issues in Kubernetes at scale? #intuit #kubernetes #debug

23

Hey everyone, I am an investor and advisor to Katanemo.... They are reinventing the infrastructure primitives for GenAI and launched Arch - https://lnkd.in/gCrbbkEV - an intelligent prompt gateway. Give the project a look on Github, and star the project if it resonates

88

3 Comments

Performance is key! As we cannot share the detailed work with (former) customers of Rockset (acquired by OpenAI), we decided to implement the Rockbench benchmark to check how streaming ingest between CrateDB and Rockset (acquired by OpenAI) compares. The results are very insightful. We see: ✅ CrateDB achieves 6-9x lower latencies than Rockset for streaming ingest. ✅ CrateDB outperforms Rockset on the same hardware while saving about 20% costs. ✅ When volumes increase, the latency increased linearly in Rockset, while remaining mostly flat in CrateDB. As a plus: 1️⃣ Achieve full high availability due to the distributed nature of CrateDB (benchmark results are achieved including replication in CrateDB) 2️⃣ Achieve higher throughput on the same amount of CPUs as Rockset (at 50% of memory needs) 3️⃣ Full open-source solution that prevents you from market shifts On side note: If you are thinking to switch to Elastic/OpenSearch, this benchmark has been created to compare Rockset vs. Elastic, so CrateDB is also faster than Elastic for streaming ingest cases! Read the full comparison here: https://lnkd.in/d_nEMDkx

29

1 Comment

At Carta, I've had the opportunity to mentor many sharp and hungry engineering co-ops that come from the University of Waterloo. I've found that the integration of externships into the school's curriculum has led to well-rounded, technologically agile, and effective software engineers. One document that I think that every new grad and intern should study is the Future Ready Talent Framework. This framework really does set a lot of coops forward in terms of their skills relative to the rest of the market. Check it out!

27

Can KV cache optimization go beyond just reducing memory footprint? Our latest work, SwiftKV, does exactly that—cutting prefill computation by half and significantly accelerating enterprise inference workloads! We’ve open-sourced the SwiftKV-optimized model checkpoints for the Llama 3 family and vLLM implementations, enabling efficient inference and making it easier than ever to integrate these advancements into your applications. - Tech blog: https://lnkd.in/gZbPBum3 - Paper: https://lnkd.in/g__9NpBd We hope SwiftKV inspires new possibilities for enterprise LLM efficiency for you!

460

5 Comments