Aaron Binns

Worth reading: How the Apache Arrow Format Accelerates Query Result Transfer by Ian Cook One of the hidden problems of intersystem communication is transcription of data formats. Ian does a wonderful job of explaining the "why" of Apache Arrow as well as the "how" of its advantages. Read on...

8

Much has been made since last week's announcement of Apple Intelligence at Apple's WWDC. Many have been pontificating on what Apple can do from a training perspective with the vast amount of data being collected on iPhones, and this incredibly comprehensive view they have of an individual consumer. But it's very important to remember that *most* of the valuable data on an iPhone is in third-party apps, the most important of which are already owned by Google, Meta, and other giants. While Apple sanctions all apps that can be installed on iPhones through the App Store's monopoly, it does not have the right to use the application specific or consumer specific data being collected by those apps. Evidence of breaching this confidence would quickly become a massive problem. Much like Google's acquisition of Hubspot in order to easily incorporate all of an enterprises' historical content for each of its customers to build customer-specific AI models (eg think an Accenture-specific model trained on all historical hubspot-housed Accenture marketing data, and all other Google data Google about Accenture, available with the click of a button), it will be interesting to see if Apple begins to methodically buy up mobile app companies to acquire their underlying data sets for use in training. Given Apple's penchant for vertical integration, it would seem a more relevant approach than attempting to do licensing partnerships as OpenAI has to get at underlying training data. If Apple wants to own the consumer from a GenAI and personal assistant perspective, they'll need to get at the data inside the myriad apps on the iPhone. https://lnkd.in/eH-BcfNe

6

This week marked a watershed moment for Snowflake: We launched #SnowflakeArctic, a state-of-the-art large language model uniquely designed to be the most open, enterprise-grade LLM on the market. Arctic not only outperforms many leading open models, but we brought it to market at one-eighth of the cost and in less than three months compared to other models. Arctic will significantly enhance our ability to deliver reliable, efficient AI to our customers. By delivering industry-leading intelligence and efficiency in a truly open way to the AI community, we are furthering the frontiers of what open-source AI can do. https://lnkd.in/eueT5Wcs

78

If you are ingesting data from Kafka/Kinesis into Snowflake you should watch this short video by Charles Tan and Shawn Gordon on how you can use DeltaStream to shift left your workloads to reduce cost and latency! #Flink #Kafka #Snowflake #ShiftLeft https://lnkd.in/gnZ45MTa

14

Read this great story from hipages Group on how they overcame challenges including legacy data systems slowing productivity and insight, requiring days to transition data between multiple warehouses. Partnering with Databricks hipages overcame these hurdles by transforming their data architecture into a single lakehouse architecture, enhancing both speed and reliability.  “We were able to tackle all our platform, productivity and insights challenges by migrating to a single platform, Databricks. Our architecture is more streamlined and our data users are more autonomous.” — Minas Kamel, Senior Product Manager, hipages #data #ai #lakehouse https://lnkd.in/gfXSqNSy

80

2 Comments

Today, Snowflake launched #SnowflakeArctic, our first foundation #LLM built for the Enterprise use case. Snowflake eschewed the "bigger = better" mantra currently dominating frontier LLM research. Instead, it focused on techniques and approaches designed to bring the cost of training and inference down to an approachable level. Using a unique Dense+MoE Hybrid transformer model, which combined a "small" 10b dense transformer with 128x3.66b MoE for 480B total and 17B active parameters. This approach allowed Snowflake to train Arctic with far less cost and time than other models - with a budget of under $2m and less than 3K GPU weeks of compute capacity. That's 17x less compute budget than Llama 3 and 7x less than DBRX for comparable capability across metrics important to the Enterprise. The demonstrated cost efficiency of Arctic, opens the way for many more enterprises to train their own first-party model tailored to their own expert domains. This is an incredibly exciting time to be in this space! Play with it at https://lnkd.in/gneYWTav #AI #Snowflake #GenAI #Opensource #SnowflakeArctic

19

Exciting news! Snowflake has unveiled a cutting-edge system for large language models, featuring the powerful Llama 3.1 405B from Meta. This innovation brings a new level of efficiency and capability to AI applications. At 7Rivers, we're excited to harness this technology, enhancing our Data-Native™ approach to deliver unparalleled AI solutions for our clients. Our commitment at 7Rivers is to drive business value through use of intelligent data and advanced AI. Snowflake's latest offering, which allows real-time AI processing with minimal infrastructure, perfectly complements our mission. This advancement enables us to provide our clients with scalable, cost-effective AI applications, enhancing operational efficiency and driving impactful outcomes. The integration of Meta’s Llama 3.1 405B within Snowflake Cortex AI aligns with our Data Native™ Model, empowering us to create transformative solutions, optimizing processes, and creating a competitive advantage. Together with Snowflake, we're revolutionizing data usage to create intelligent, augmented business experiences. #AI #DataNative #7Rivers #Snowflake #GenerativeAI #BusinessTransformation

34

2 Comments

❄️ Come and hear how Snowflake uses DataOps.live for massive productivity gains for all its engineers. #data #dataops #analytics #dataproducts

10

Snowflake should at least tell us what bugs they are fixing and what features they are working on in snowflake managed Polaris. It will helps us to prepare .. For example: add_files, register_table procedure are not working. I know snowflake team is working on it but no info to community. Some of these features essential to move on from PoC to prod. James Malone Snowflake

8

Spend your Sunday with a deep dive with the top execs at Databricks:

14

1 Comment

On #AutomaticPerformance and always shipping.... You may have seen me post in the past about how we view Automatic Performance at Snowflake. In a nutshell, this "true north" encapsulates our continuous quest to accelerate workload performance for our customers, and to deliver these performance gains without requiring any user intervention or manual configuration. One thing that makes this possible (and hopefully magical for our customers) is our approach to continuously shipping updates to our core services with NO SCHEDULED DOWNTIME for our customers. No scheduled maintenance windows, no upgrading your service - these enhancements just show up one day, and things are better and faster. As you can read here https://lnkd.in/gwNdi4Rr : “We deploy new releases each week. This allows us to regularly deliver service improvements in the form of new features, enhancements, and fixes. The deployments happen transparently in the background; users experience no downtime or disruption of service, and are always assured of running on the most-recent release with access to the latest features.” As someone who has been in the software industry for… well… let’s just say a long time, I am consistently impressed by the technical craft, sophistication and customer-focus that goes into the systems that support this kind of continuous delivery. And while I (still) might not know all the details, I do know this - customers love it! Curious about all the automatic performance improvements that have just shown up for Snowflake customers? Check out the link to the performance release notes and more detailed performance blogs in the comments.

114

2 Comments

Same Chicken (Iceberg), different Curries (Engines) Cloudera: Read Snowflake Data from Polaris External Sync Catalog I am able to "read" and integrate Snowflake data through Polaris with other data in Cloudera Spark. Cloudera Snowflake

17

Last week, I was fortunate enough to be included in the first Databricks Startup Founders Dinner in NYC. Thank you to the Databricks team for inviting us. It was inspiring to meet and listen to Jonathan Frankle, who is one of the most genuine and humble AI scientists, who is helping to lead the Databricks team into the future with their release of DBRX in March. It was great to talk to other founders with unique businesses in the data science space. I had the opportunity to showcase how Liminal Data Inc. is leveraging tools made available by Databricks, Google, and Amazon Web Services (AWS) to innovate in the impact and sustainability space. Our platform helps reduce the complexity of the underlying data science needed to do carbon emissions calculations and quantify the complex impact metrics that are material to a business. I have many takeaways from the event that will shape our thinking as we continue to develop our solution. A few highlights include: 1. AI is no longer a 'nice to have' but will be necessary for innovation. There are countless ways in which we can incorporate the use of AI in our product and we are already developing some solid use cases. 2. Databricks is an innovative team and clearly motivated to move fast and push the envelope. Every Databricks employee I have met is excited and eager to be a part of a fast moving company. 3. Companies like Databricks are moving fast and breaking things, so that there are revolutionary products in the marketplace for companies like Liminal to leverage. We are already using many of their products to improve our customers' experiences and advance their own thought leadership. Special thanks to Arjun Rajput and Josh Lillie for the invite to the startup dinner. We're excited to continue building the future with you. *I somehow managed to not take any pictures of the event other than the entrance to Catch NYC restaurant. Including it in this post since I loved that they had an Underwood typewriter positioned at the entrance; typewriters were a revolutionary technology at one point in time and it seemed kismet that was the first thing I saw when I arrived at the dinner.

32

2 Comments

Using Kumo.AI in your Snowflake environment is like peanut butter and chocolate - better together! Check out this webinar to learn more.

10

1 Comment

Our commitment to AI-driven solutions and strong alliances with Snowflake and Databricks are enabling significant advancements in data management and analytics. Check out this SiliconANGLE & theCUBE article where Rob Strechay and George Gilbert discuss how Informatica remains at the forefront of data management, consistently driving innovation and customer success:

2

At Snowflake, we're committed to delivering ongoing Automatic Performance enhancements. And, instead of using synthetic benchmarks for performance comparisons, we measure the impact of these improvements by looking at the performance of real customer workloads. We call this measurement the “Snowflake Performance Index”, or, SPI. The SPI reflects genuine improvements that make a difference in your day-to-day operations. Since we launched the SPI in August 2022, the average query duration for stable workloads has now improved by 40%. In the last 12 months alone, the SPI has seen a 20% improvement! These improvements included: >> Improved Query Compilation: Enhancements to query compilation processes result in faster start times for certain query types. >> Optimized Table Operations: Performance improvements for DML (Data Manipulation Language) operations, especially for bulk updates and inserts. >> Caching Enhancements: Enhanced caching mechanisms for frequently accessed data to reduce latency. >> Adaptive Query Execution: Dynamically adjusted execution strategies for complex query patterns, improving runtime efficiency. Want to learn more about the SPI results? Check out Rudi Leibbrandt's blog post: https://lnkd.in/g3Q7iegg (If you want to know more about our SPI methodology, and all the improvements that have contributed, check out the links in the comments.)

361

3 Comments

If you look at areas where Microsoft and OpenAI are leading in AI, they’re very complementary. Microsoft has the hardware and OpenAI has the models and mainstream product adoption. If their partnership can hold, they’ll likely continue to be hard to beat.

3

🌟 From Raw Data to Behavioral Predictions—Effortlessly What if AI deployment didn’t require months of preparation? With BaseModel.ai now on the Snowflake Marketplace, you can transform behavioral data into predictions in just a few clicks. ✅ Rapid Deployment: Build and deploy models in a week.  ✅ Scalable Flexibility: Create multiple models for diverse use cases.  ✅ Privacy Guaranteed: Data never leaves the secure Snowflake environment. Say goodbye to complexity and hello to actionable insights. AI for behavioral data has never been this simple. 👉 Try BaseModel.ai Today  #AI #MachineLearning #BehavioralData #BaseModel #Snowflake #Synerise 

39

4 Comments

We’re introducing Snowflake Intelligence to make it easier for everyone to drive value with data. Imagine asking a data agent: “give me a summary of this Google Doc” or “tell me how many deals we had in North America last quarter,” and instantly following up with the next steps using that same agent. That’s exactly what Snowflake Intelligence will enable: a seamless way to leverage and take action on your data in one place. Excited to share this update with you all. This is what we mean by the Era of Enterprise AI! 🚀

1,468

31 Comments