Dirk-Jan van Veen

This article described AI as another "dot-com bubble" that is ready to bust anytime soon as soon as the "right time" comes and likened to a keg of gun powder waiting to explode at the next ignition opportunity. Hmmmmm ...... While some commercial entities might have portrayed AI as a magic wand that MUST work everywhere and take over ALL workflows, the reality is that it is a tool that has the capability to enhance existing workflows, create new opportunities to improve workflows, and support domain experts to do more and perform better at what they know to do best. Since AI, and especially the computational aspect of it, ML, relies on real data and the verifiable relationship between input features and the target properties, there is no opportunity to scam or spook anyone. The algorithms are well established in mathematics and statistics. There is a standard practice procedure (SPP) to machine learning applications that, if followed, would help to ensure the success in a modeling task or explain why it failed. It's not a magic wand. It's backed up by the physics, chemistry, and biology embedded in operational data. Success stories abound everywhere and, as a researcher, the reasons for the failed cases can be identified and mitigated. The article is here for those interested in more details: https://lnkd.in/dK9HYwn7 #artificialintelligence #ai #machinelearning #ml #dotcombubble

10

The future of QUANTUM machine learning could be closer than previously envisaged. Thanks to the on-going research and development efforts of the likes of Google, IBM, Cambridge, etc. For those handling big data and complex algorithms such as deep learning, machine learning is beginning to reach the limit of the traditional computing paradigm. Imagine training a deep learning model for days, weeks, and months. The training time of GPT-4 is around five to six months. So, imagine the huge energy consumption. Quantum computer, and quantum machine learning in particular, is poised to significantly reduce the training time to seconds or minutes as well as the associated energy consumption. Be ready! This article gives a succinct introduction. https://lnkd.in/dYfwX5Rb #quantumcomputing #quantummachinelearning #quantumml #machinelearning #ml #deeplearning #quantumcomputer #chatgpt4

24

6 Comments

🔺 A little gotcha in #python Or maybe an artifact of #pytorch (because the logic is same, outputs are differently shaped) Suppose you have a tensor of the shape T =: (Batch, X, Y, Channels) If you slice the last channel-layer, Approach 1 : T[ : , : , : , -1] Output Shape : (Batch, X, Y) Approach 2 : T[: , : , : , -1:] Output Shape : (Batch, X, Y, 1) While it is a common norm in python that -1 is the last index, it makes no logical sense to interpret -1: as anything but -1 itself. That additional dimension might break codes downstream. It might go down as a difficult bug to find in your deep-neural-architectures! Happy weekend. Happy Learning! (Edit, second print statement: Approach 2*)

18

2 Comments

I am not sure how many of you are paying attention, but this post is one you should read. The line between AI and AGI (artificial general intelligence) is blurring rapidly. Po-Shen Loh has laid out a major concern about employability of humans in a world where AI is simply better and faster at solving problems. Math Olympiad problems require creativity, not just regurgitation of facts. Read his full post, it’s thought provoking and urgent. #ai #articialintelligence #employment #walle

3

🚀 Exciting Progress in #AI for Mineral Analysis! 🚀 Last week, we explored #LLAVA, a multimodal model (text and image analysis) based on Meta's LLAMa3, applied to describing mineral thin sections. While the results initially seemed impressive, expert petrographers noted they were more on par with an undergrad just starting their petrology journey. Taking your valuable feedback, I set out to enhance the model's capabilities this week. Key improvements include: 🔍 Training the model to use a specific template for descriptions. 🖼️ Incorporating both PPL and XPL images for comprehensive analysis. 📏 Adding measurement scales for metrics like grain size and percentage distribution. The new results are promising, but there's still work to be done before integrating this into the RadiXplore platform. One current limitation is variability in results when running the model multiple times (check the comments for examples). Foreseeing significant applications, I see leveraging this technology for bulk quantification of extensive thin section datasets. This approach facilitates rapid validation and correction by petrographers through various statistical techniques, transforming archived data into instantly accessible, exploration-enabling repositories. 💡 What are your thoughts on this approach? ✔ I'm also attending the Australian Energy Producers Conference and Exhibition conference in Perth today if you would like to discuss! #AI #MineralAnalysis #Innovation #ExplorationTech #unstructureddata #LLM

65

16 Comments

A good post. Highlighting the "actual" areas LLMs are winning at.

4

LLMs have become more capable with better training and data. But they haven’t figured out how to “think” through problems at test-time. The latest research from Scale finds that simply scaling inference compute–meaning, giving models more time or attempts to solve a problem–is not effective because the attempts are not diverse enough from each other. 👉 Enter PlanSearch, a novel method for code generation that searches over high-level "plans" in natural language to encourage response diversity. PlanSearch enables the model to “think” through various strategies before generating code, making it more likely to solve the problem correctly. The Scale team tested PlanSearch on major coding benchmarks (HumanEval+, MBPP+, and LiveCodeBench) and found it consistently outperforms baselines, particularly in extended search scenarios. Overall performance improves by over 16% on LiveCodeBench from 60.6% to 77%. Here’s how it works: ✅ PlanSearch first generates high-level strategies, or "plans," in natural language before proceeding to code generation. ✅ These plans are then further broken down into structured observations and solution sketches, allowing for a wider exploration of possible solutions. This increases diversity, reducing the chance of the model recycling similar ideas. ✅ These plans are then combined before settling on the final idea and implementing the solution in code. Enabling LLMs to reason more deeply at inference time via search is one of the most exciting directions in AI right now. When PlanSearch is paired with filtering techniques—such as submitting only solutions that pass initial tests—we can get better results overall and achieve the top score of 77% with only 10 submission attempts. Big thanks to all collaborators on this paper including: Evan Wang, Hugh Zhang, Federico Cassano, Catherine Wu, Yunfeng Bai, William Song, Vaskar Nath, Ziwen H., Sean Hendryx, Summer Yue 👉 Read the full paper here: arxiv.org/abs/2409.03733

180

2 Comments

In this article, the authors show that #Reservoir #Computing (#RC) can universally approximate a general #linear #time-#invariant (#LTI) system. Specifically, they present a clear signal processing interpretation of RC and utilize this understanding in the problem of approximating a generic LTI system. Under this setup, they analytically characterize the optimum probability density function for configuring (instead of training and/or randomly generating) the recurrent weights of the underlying #recurrent #neural #network (#RNN) of the RC. ---- Shashank Jere, Lizhong Zheng, Karim Said, Lingjia Liu More details can be found at this link: https://lnkd.in/eVnTTxVH

14

1 Comment

This is really awesome. I used notebookLM to summarise a recent paper that shows how o1 like model can be reproduced. After providing the pdf from Arxiv, I noticed that the generated podcast was almost 49 minutes long. Very different from the earlier 10-12 minute version. And another impressive feature was the interactive mode where you can interject the two hosts and ask a question which they very graciously respond to. I wanted to share the audio summary but YT didn’t allow me to upload the WAV file. YT expects a video mp4 file. So I found free videos from Pexels and stitched them together with ffmpeg. The videos are meditative acquarium shots of fish swimming around. And with the audio discussing reproducing o1 as an engaging podcast episode, the video keeps the distraction to the minimum. The paper itself is a great read but if you have a 30-40 minute commute, or want to watch some cool fish swimming around and learn about o1, check out

12

Except for highly curated and artificially generated data, no real operational data will give you a 100% accuracy on blind validation from any model, however sophisticated. This is mainly because of the embedded and hidden bias in every data. Bias creeps into data in different and various ways. Each field of application needs to define its own error tolerance. Definitely, some fields are more sensitive to and accommodative of different degrees of errors than others. While the bias in models can be significantly reduced by employing the automatic hyperparameter optimization, the one in data is very challenging to handle. It requires a lot of work! I like the conclusions of this article: ensuring data quality and integrity, rigorous validation of trained models, and keeping the human in the loop. Read more: https://lnkd.in/dwecVaVy #data #operationaldata #dataquality #dataintegrity #errortolerance #bias #biasindata #hyperparameteroptimization #humanintheloop

16

3 Comments

Kolmogorov-Arnold Networks look interesting and explainable. But weren’t we doing this already in engineering school? I mean curve matching and fitting. The innovation here is that they are doing it on a interrelated multiple equation basis, where the “constants” (activation functions) are being changed dynamically, which is pretty neat. Definitely will be more explainable than multilayer perceptrons or simple neural networks. KANs “explainable” nature is equivalent to explain why the constants in the equations are what they are. In other words, this is not the path to AGI, or discovering news laws of physics either, although the method is elegant. In my opinion, where the “gold” resides in Kolmogorov-Arnold Networks is this: Being trained with reasonable good data and then use the trained network to (i) predict a more accurate output when any of inputs is out of whack; (ii) discard inputs that are out of range; (iii) qualify the inputs before a prediction is applied; (iv) get a notion of the extend at which the inputs are diverging from the ground truth One case that comes to my mind is Virtual Flow Meters where data (inputs) coming from well tests is very inconsistent due to uncertainties ocurring during measurement. In the abstract the authors got carried away a bit: " KANs are shown to be useful collaborators helping scientists re-discover mathematical and physical laws." It seems incorrect to me. You don’t rediscover the laws of physics with this ML algorithm. You get a hint of what external factors could be affecting the input data that make the output deviate from the laws of physics. #machineLearning #physics #artificialIntelligence #AI #AGI #spe #petroleumEngineering #VirtualFlowMeters #dataScience

47

10 Comments

💥💥💥 RieszBoost: Gradient Boosting for Riesz Regression Kaitlyn J. Lee, Alejandro Schuler Abstract Answering causal questions often involves estimating linear functionals of conditional expectations, such as the average treatment effect or the effect of a longitudinal modified treatment policy. By the Riesz representation theorem, these functionals can be expressed as the expected product of the conditional expectation of the outcome and the Riesz representer, a key component in doubly robust estimation methods. Traditionally, the Riesz representer is estimated indirectly by deriving its explicit analytical form, estimating its components, and substituting these estimates into the known form (e.g., the inverse propensity score). However, deriving or estimating the analytical form can be challenging, and substitution methods are often sensitive to practical positivity violations, leading to higher variance and wider confidence intervals. In this paper, we propose a novel gradient boosting algorithm to directly estimate the Riesz representer without requiring its explicit analytical form. This method is particularly suited for tabular data, offering a flexible, nonparametric, and computationally efficient alternative to existing methods for Riesz regression. Through simulation studies, we demonstrate that our algorithm performs on par with or better than indirect estimation techniques across a range of functionals, providing a user-friendly and robust solution for estimating causal quantities. 👉 https://lnkd.in/dCszhVaP #machinelearning

1

math performance of various AI models

2

Finetuning ColPali-3B model on Norwegian language with specific knowledge about oil and gas on my Colab Pro's NVIDIA GPU A100 machine

23

gguf format changes the bit size of the weights, that shrinks down the size of ai llm, allowing it to run on smaller hardware. I have tried few quantized ai llm. they were slow, some hang my system, most likely due to only 32g memory. however, for smaller devices like mobile phone, and trained for few simple use cases instead of generalized big ai llm, they offer a way to incorporate ai on small form hardware poor devices.

15