Oliver Koo’s Post

With Honeycomb, Pinterest enhanced their CI/CD pipeline in the following ways: + Automated debugging : Developers can use Honeycomb's query language to automate debugging processes. + Faster feedback loops : The platform enables faster feedback loops by providing real-time visibility into service performance. + Improved collaboration : Honeycomb facilitates collaboration between developers, engineers, and product managers by providing a shared understanding of the application's behaviour. https://bit.ly/3DOVlxi

I've posted about Pinterest utilizing honeycomb.io before, but I love this article from InfoQ. 📌 x 🐝 "Using Honeycomb's features, such as derived columns and fast query performance, the team can process millions of daily events and identify anomalies in build times or pipeline performance in near real-time. This allows them to diagnose issues quickly and implement targeted improvements." Full article below ⬇️

I specialize in optimizing performance at scale, and I agree many organizations bleed ridiculous amount of money on the infra side. Most of that is even often a recurring cost, which helps making it feel "business as usual", further reinforced by "that's how we always did it". Meanwhile I alone often enabled improvements by 1-2 orders of magnitude several times throughout my career: - reduced infra TCO by 75% (=4x capacity on same hardware) - increased network throughput to 80x on same hardware (=98.75% waste before) - reduced build resource usage 60x (=98.33% waste before) - reduced server provision time by 90% (~10x faster) - made terabyte-scale ETL operations finish in 40 minutes instead of 11 hours (=16x faster, or ~94% waste before) I saved several times the amount of my wages or consulting fees on several occasions. Now imagine enabling a whole team to do similarly. I'd love to introduce your team to minimum viable infrastructure as well. It's the least I could do.

7 Essential System Design Projects Every Developer Should Master.     Want to build systems like Uber, Netflix, or Facebook?   Mastering System Design (HLD + LLD) is key to creating scalable and reliable architectures. These 7 projects will help you understand real-world challenges like load balancing, caching, and database scaling.   Pro Tip: Combine traditional system design skills with the power of GenAI. Tools like GitHub Copilot, RAG (Retrieval-Augmented Generation), and LangChain can supercharge your development workflow!   Thanks to Learnbay for this guide.   Whether you're preparing for interviews or building complex applications, this guide will take your skills to the next level.   Learn System Design, GenAI tools in our specialized programs for software developers.   Details here: https://lnkd.in/eNtGWRdQ   Join their free demo sessions and get career guidance and interview tips directly from FAANG mentors!   #SystemDesign #SoftwareEngineering #DeveloperTips #CareerAdvice #ad

How can organizations effectively monitor and optimize their CI pipeline performance while managing millions of daily build events? Pinterest 's journey with honeycomb.io showcases an innovative approach to CI observability that transformed their build process monitoring. By implementing Honeycomb for real-time analysis of build metrics, they've created a sophisticated system handling 1 million events daily while enabling sub-second query response times for complex build analysis. The technical transformation centered on integrating Honeycomb with Buildkite, using AWS EventBridge to process build logs and create detailed traces. This allowed Pinterest to break down builds into granular segments, track performance metrics, and implement automated error categorization - significantly improving their ability to identify and resolve CI pipeline issues. Key Learnings: - Implement trace-based observability to break down CI jobs into actionable segments - Leverage correlation analysis to quickly identify root causes of build failures - Automate error categorization to route alerts to appropriate teams - Use custom instrumentation to track specific build segments within scripts What makes this solution particularly compelling is its approach to error categorization, which automatically routes issues to relevant teams while providing deep visibility into build performance. This significantly reduces noise in on-call rotations while maintaining comprehensive system oversight. #observability #continuousintegration #devops

At Fastly, we believe your tech should work for you, not against you. When Stuff's platform was holding them back from innovating new products for their customers, they knew they needed to switch. So how did Fastly help empower Stuff's platform team? Read about the good stuff here 👇🏻😉

Get familiar with latest trends by following below Engineering blogs links 👇👇 ♻️♻️♻️

𝐋𝐞𝐞𝐭𝐜𝐨𝐝𝐞 𝐁𝐢𝐰𝐞𝐞𝐤𝐥𝐲 𝐂𝐨𝐧𝐭𝐞𝐬𝐭 137: 𝐄𝐝𝐢𝐭𝐨𝐫𝐢𝐚𝐥: 𝐐1-𝐐4 𝑯𝒆𝒍𝒍𝒐 𝒆𝒗𝒆𝒓𝒚𝒐𝒏𝒆, 𝑰 𝒑𝒂𝒓𝒕𝒊𝒄𝒊𝒑𝒂𝒕𝒆𝒅 𝒊𝒏 𝒍𝒆𝒆𝒕𝒄𝒐𝒅𝒆 𝒃𝒊𝒘𝒆𝒆𝒌𝒍𝒚 𝒄𝒐𝒏𝒕𝒆𝒔𝒕 137 𝒂𝒏𝒅 𝒈𝒐𝒕 𝒂𝒏 𝒐𝒗𝒆𝒓𝒂𝒍𝒍 𝒓𝒂𝒏𝒌 𝒐𝒇 106 𝒃𝒚 𝒔𝒐𝒍𝒗𝒊𝒏𝒈 𝒂𝒍𝒍 𝒒𝒖𝒆𝒔𝒕𝒊𝒐𝒏𝒔 𝒖𝒏𝒅𝒆𝒓 50 𝒎𝒊𝒏𝒖𝒕𝒆𝒔. 𝐐1,𝐐2 -> just maintain a Boolean DP array where DP[i] is true if nums[i]-nums[i-1] = 1, else DP[i] = 0; 𝐂𝐨𝐝𝐞 -> https://lnkd.in/gRENeEz7 Also, make a prefix_sum_array from this DP array, Now for any Subarray of size K, just check that all the DP values in this range are 1 or not (by the prefix_sum_array we are maintaining). 𝐐3, 𝐐4 -> Run A O(n^2) loop to pick two different rows, we will select the highest possible numbers from these rows, now to get the third number Maintain a List of structures that will contain 3 things: value, row, and column for a particular entry, and sort this list in decreasing order of value. Now for the third value, just iterate on this loop till you find an entry whose rows and columns differ from the previous 2 values selected, This loop in total will run very less which will effectively not affect the Time Complexity to make it O(n^3). 𝐂𝐨𝐝𝐞 -> https://lnkd.in/g6_FiYmz #leetcode #codeforces #codechef #google #amazon #uber #swiggy #zomato #meta #apple #tiktok #adobe #salesforce

🌟 Monday Motivation: Build the Future, One Line of Code at a Time! 🌟 New week, new opportunities! 🚀 Whether you’re debugging, designing front-end interfaces, or crafting seamless APIs, every step brings us closer to innovation. 💡 💻 Here’s to building smarter solutions, solving complex challenges, and achieving new milestones. Let’s make this week count! #MondayMotivation #CodingLife #SoftwareEngineering #FrontEndDevelopment #BackEndDevelopment #APIIntegrations