Chrisantha Fernando

I am pleased to share that our paper on Jaynes machine is now available online via Open Access: https://lnkd.in/e64_Jgja John Hopfield and Geoffrey Hinton, who are receiving the Physics Nobel next month, looked to physics for inspiration to model neural networks. They developed the Hopfield network and the Boltzmann machine by using the Ising model and energy minimization as their guiding principles. I went in a completely different direction, looked to game theory, and identified potential games as the starting point. In our framework, the neurons and the connections compete in an error-minimization game to increase their effective utilities during training and reach an arbitrage equilibrium. The outcome is a lognormal distribution of weights and neuronal iota. Our paper shows that this is empirically observed in real-life deep neural nets such as the LLMs. The Hopfield model and the Boltzmann machine cannot predict the distribution of connection weights as it is beyond their capabilities. We show that the Hopfield network and the Boltzmann machine are special cases of the Jaynes machine. One can think of the former two as the "ideal gas law" version of neural networks, whereas ours is the "van der Waals equation". Just like the real-world gases are not ideal and behave more like van der Waals gases, the real-world deep neural networks such as VGGnet, BERT, and LLAMA are not Boltzmann machines, but are more like Jaynes machines.

132

14 Comments

🌟 Exploring ViSNet in the Recently Published AI2BMD Neural Network Potential 🌟 I recently came across the new AI2BMD neural network potential just published in Nature, and its underlying architecture—ViSNet—really stood out. ViSNet offers a unique approach to molecular modeling by efficiently capturing molecular geometry—bond lengths, bond angles, and dihedral angles—using innovative, low-complexity vector operations. This design strikes a strong balance between detail and efficiency, making it ideal for applications in molecular dynamics. 🚀 Central to ViSNet’s design is the Runtime Geometry Calculation (RGC) module, which dynamically extracts high-order geometric features as needed, avoiding explicit tensor storage. The key idea here is to break down complex geometric features into manageable vector representations. For instance, bond lengths are derived directly from the distance between atoms, while bond angles and dihedral angles are calculated from vector inner products and rejections between neighboring atoms. By summing unit vectors between atoms, RGC efficiently captures angular and torsional information at linear complexity, avoiding the O(N³) cost of traditional tensor operations. This keeps computational demands low and allows ViSNet to scale effectively to larger molecular systems, even when full geometric detail is required. 🧬 Vector-scalar interactive message-passing (ViS-MP) mechanism is another key method. Instead of treating scalar and vector features independently, ViSNet integrates them within each layer, letting vector features (like directions) and scalar features (like bond distances) inform each other. This combined representation captures detailed spatial information with fewer samples, enhancing data efficiency without sacrificing accuracy. Benchmarking on datasets like QM9, MD17, and Molecule3D, ViSNet demonstrated strong performance in predicting molecular properties such as energy and atomic forces, achieving high-fidelity results with as few as 950 training samples. 👍 The AI2BMD paper shows how ViSNet’s careful balance of accuracy, computational efficiency, and scalability can support realistic simulations, driving forward practical applications in drug discovery and beyond. 🌱 👉 Check out the ViSNet paper: https://lnkd.in/ggUZAeVg 👉 Check out the AI2BMD paper: https://lnkd.in/gxR6wtZX #DrugDiscovery #MolecularModeling #MachineLearning #NeuralNetworkPotential #AI4Science

32

Our paper on quantum computed Green's functions has been published! 🎉 https://lnkd.in/e6aeasRZ

75

7 Comments

Check out the latest research on high-performance graph processing from a hardware/software co-design perspective! This paper dives deep into innovative strategies to optimize computational efficiency. https://lnkd.in/gUXumc5x 廖小飞 Hai Jin Long Zheng Zhiyuan Shao

Gravity might be a universal decoherence mechanism, underlying quantum-to-classical transitions and the emergence of classical behaviors from a fundamentally quantum universe. https://lnkd.in/dijVRFzV

2

Our #icra2024 paper is now available from #ieee dl. We proposed a new Inverse Reinforcement Learning based method for hand trajectory prediction. Mukund Mitra Gyanig K. Partha Pratim Chakrabarti

18

2 Comments

Proprietary datasets and the combination of AI and experimental data are the key to build successful companies.

1

That is impressive.

14

With the recent Nobel prizes in physics and chemistry going to AI and AI respectively, this is the time to talk about the opportunities (and risks) of using AI in all science and humanities research and knowledge production. Looking forward to discussing 'Science, Shadows & Simulacra' at this event today, and listening to the other interesting talks that are part of the program. Bonus points for anyone who guesses what shadows and simulacra I am referring to here ;).

19

I had a great time proving the XEB for approximate tensor representations and also working on the randomness extraction (q crypto aspect) of this Nature paper out of our group. https://lnkd.in/guPuxXph #quantum #publication #research #quantumsupremacy

63

2 Comments

In a remarkable stride towards revolutionizing computing, researchers have developed a new form of exceptionally pure silicon, poised to underpin the next generation of quantum computers. ⚛️ ➡️ This advancement, emerging from the University of Manchester, tackles the persistent challenge of 'fragile quantum coherence'—a vexing issue where quantum computers accumulate errors rapidly due to environmental disruptions like temperature fluctuations. ➡️ The innovation employs phosphorous atom-based qubits embedded in crystalline silicon, enhancing stability and robustness. Such a breakthrough could halve the timeline for practical quantum computing, shifting a potential ten-year journey to just five. ❓ Could this leap in materials science catapult us into a new era where quantum computing transforms AI, drug discovery, and energy usage, or are we witnessing an overpromise under the guise of progress? Read the full story on the Independent: https://lnkd.in/gBxvWnUn #QuantumComputing #Breakthrough #innovation #Computing #Silicon #Future ---- 💡 𝗜𝗳 𝘆𝗼𝘂 𝗲𝗻𝗷𝗼𝘆𝗲𝗱 𝘁𝗵𝗶𝘀 𝗰𝗼𝗻𝘁𝗲𝗻𝘁, 𝗯𝗲 𝘀𝘂𝗿𝗲 𝘁𝗼 𝗱𝗼𝘄𝗻𝗹𝗼𝗮𝗱 𝗺𝘆 𝗻𝗲𝘄 𝗮𝗽𝗽 𝗳𝗼𝗿 𝗮 𝘂𝗻𝗶𝗾𝘂𝗲 𝗲𝘅𝗽𝗲𝗿𝗶𝗲𝗻𝗰𝗲 𝗯𝗲𝘆𝗼𝗻𝗱 𝘆𝗼𝘂𝗿 𝘁𝗿𝗮𝗱𝗶𝘁𝗶𝗼𝗻𝗮𝗹 𝗻𝗲𝘄𝘀𝗹𝗲𝘁𝘁𝗲𝗿 - you can have real-time insights, recommendations (a lot more than I share here) and conversations with my digital twin via text, audio or video in 28 languages! Join >5000 users who went before and go to app.thedigitalspeaker.com to sign up and take our connection to the next level! 🚀

3

Confounding? Detect it! A brand new paper on detecting and measuring confounding using causal mechanism shifts (presented this week at NeurIPS) In their new NeurIPS 2024 paper, Abbavaram Gowtham Reddy and Vineeth N. Balasubramanian (IIT Hyderabad) propose a framework for detecting and measuring confounding in a number of scenarios, based on shifts in causal mechanisms. The authors propose three complementary measures of confounding (with conditional and multivariate variants), designed to assess relative strengths of confounding in a variety of settings. The proposed measures complement one another depending on the available contextual information and offer a unified way to study confounding across diverse scenarios where data from different contexts is available. The authors generously shared Python implementation of their solution on GitHub. 💡 Paper: https://lnkd.in/dGXbnqAy 💡 Code: https://lnkd.in/dpczZvcU

142

7 Comments

A few weeks ago, KyungHyun Cho's blog post on anxiety and frustration at #NeurIPS (https://buff.ly/4fN4Hah) has provided a lot of insight into the recent history of the AI boom in academia and the industry, as well as the angst many PhD students might be facing in today's job market. I'd like to echo two points I find myself agreeing a lot with Kyunghyun and expand a little. 1. Indeed, a lot of AI jobs in the industry will no longer require a PhD, because with scale and production comes processes and reliability requirements that do not require paradigm-shifting changes all the time (which PhD programs train people to aspire to). However, this does not mean a PhD training will make no difference -- critical thinking, independence in problem solving, domain expertise, and vision about strategic technical directions are all valuable traits that takes a serious amount of training to obtain, and something one should be proud of and play to their own strength. 2. Having more roles and players in the AI market does not mean fewer opportunities for PhDs, because the market is still in an upward trend. It does mean, however, that opportunities might be lying in places you hadn't thought about or anticipated when you entered the PhD or were writing the research papers you are the most proud of. While many academic institutions are growing fast in recent years, there is likely still a fierce competition in more popular directions. In the meantime, there is actually a growing demand in the industry of AI talent, including the usual suspects, as well as many non-large-tech companies, including large companies not traditionally known for Internet/software products, AI-adjacent scale-up companies, and AI-native startup companies. Many of these can provide opportunities for exciting exploratory work with reasonable freedom, depending on the scale/focus of the business. Case in point, we are looking to expand the research/foundation models team at Orby AI and are looking for highly motivated researchers and ML/Research engineers. Please reach out if you're interested in learning more!

43

1 Comment

In the war against AI plagiarism, authors are fighting back with sneaky, invisible traps. ➡️ In the era of generative AI, creators have struggled to protect their work from being scraped by AI models without consent. Enter "copyright traps," a clever innovation by Imperial College London researchers. ➡️ These traps, akin to fake locations on a map, allow writers to mark their work with hidden text, enabling detection if used in AI training. This breakthrough comes amidst high-profile lawsuits like The New York Times vs. OpenAI. ➡️ The traps are created using gibberish sentences injected repeatedly into a text, making them detectable by models trained on that data. ❓ While not foolproof, they present a formidable challenge to unauthorized use. Can such inventive measures ensure fair play in the digital age, or will it be an endless game of cat and mouse? Read the full story on MIT Technology Review: https://lnkd.in/gwJ7BVBU #Data #Copyright #Content #Creators #Ethics ---- 💡 𝗜𝗳 𝘆𝗼𝘂 𝗲𝗻𝗷𝗼𝘆𝗲𝗱 𝘁𝗵𝗶𝘀 𝗰𝗼𝗻𝘁𝗲𝗻𝘁, 𝗯𝗲 𝘀𝘂𝗿𝗲 𝘁𝗼 𝗱𝗼𝘄𝗻𝗹𝗼𝗮𝗱 𝗺𝘆 𝗻𝗲𝘄 𝗮𝗽𝗽 𝗳𝗼𝗿 𝗮 𝘂𝗻𝗶𝗾𝘂𝗲 𝗲𝘅𝗽𝗲𝗿𝗶𝗲𝗻𝗰𝗲 𝗯𝗲𝘆𝗼𝗻𝗱 𝘆𝗼𝘂𝗿 𝘁𝗿𝗮𝗱𝗶𝘁𝗶𝗼𝗻𝗮𝗹 𝗻𝗲𝘄𝘀𝗹𝗲𝘁𝘁𝗲𝗿 - you can have real-time insights, recommendations (a lot more than I share here) and conversations with my digital twin via text, audio or video in 28 languages! Join >5000 users who went before and go to app.thedigitalspeaker.com to sign up and take our connection to the next level! 🚀

4

Thrilled to share a groundbreaking paper published by colleagues, Henrik Dreyer and Etienne Granet here at Quantinuum, featured in Nature's NPJ Quantum Information. This innovative research presents a novel method for simulating Hamiltonian dynamics on digital #quantumcomputers, bypassing the need for Trotterization – a standard but computationally expensive technique. Why is this important? The ability to simulate quantum systems more efficiently opens up new possibilities for quantum research and practical applications, especially in fields like #materialsscience, #chemistry, and #drugdiscovery, where understanding complex systems at the quantum level can lead to breakthroughs. Link to the article: https://lnkd.in/etS52Mer

63

It was a pleasure to chair the AI Salon at St Edmunds College in Cambridge last night. Many thanks to Dr Mike Katell for a fantastic keynote on data justice and to our brilliant discussants Pamela Peter-Agbia and Francois Buet-Golfouse for their insights and comments. A few takeaways: 📢 Data justice is an important topic that fits well in a wider conversation about democracy, human rights and the rule of law 📢 Whilst the UK has a proud history of engaging with notions of justice, we need to reflect on how to apply this to data governance and digital infrastructures 📢 Developments in the creative industries due to Generative AI are an issue of labour rights and data justice #aiethics #datajustice #artificialintelligence #datagovernance #aigovernance

20

The idea of #AIDO (AI-driven Digital Organism) is put under close cross-examination and discussion in an intriguing article by Matteo Wong in The Atlantic today. As for any disruptive idea, there are both concurrence and skepticism of the AIDO vision, but I hereby want to share my optimism: no one would have imagined 10 years ago what we now see from #GPTs or #Geminis in terms of simulating human intelligence; and likewise, what we envision for AIDO to simulate biology at all levels is not a fantasy either thanks to even greater advancements in AI architecture, infrastructure, and data for the years to come. Coming along with #AIDO is a concrete and operationalizable engineering roadmap as outlined in https://lnkd.in/d8eZ7mJp, built on a multi-scale foundation model system which we will be implementing in our venture GenBio AI. I look forward to updating soon our next release of the AIDO models. Stay tuned! https://lnkd.in/dr8kxgVH

281

4 Comments

Anthropic has unveiled groundbreaking insights into the inner workings of Large Language Models (LLMs) in a recent paper. This pioneering research employs "dictionary learning" to map and manipulate the internal states of these models, offering unprecedented transparency and control. This breakthrough not only enhances our understanding of neural networks but also opens new avenues for ensuring the safety and reliability of AI systems. Read more: https://lnkd.in/eAzYgnKj IdentifAI - Find Origin - #artificialintelligence

235

3 Comments

NEW PAPER | The paper “On-device Online Learning and Semantic Management of TinyML Systems“ by Haoyu Ren, Darko Anicic, Xue Bo Li and #ThomasRunkler has been accepted in the ACM Transactions on Embedded Computing Systems journal, with acknowledgement to NEPHELE. It is available to download in open access from our website: ▶ https://lnkd.in/dpS8yNt4 Recent advances in Tiny Machine Learning (TinyML) empower low-footprint embedded devices for real-time on-device Machine Learning (ML). While many acknowledge the potential benefits of TinyML, its practical implementation presents unique challenges. This study aims to bridge the gap between prototyping single TinyML models and developing reliable TinyML systems in production: (1) Embedded devices operate in dynamically changing conditions. Existing TinyML solutions primarily focus on inference, with models trained offline on powerful machines and deployed as static objects. However, static models may underperform in the real world due to evolving input data distributions. We propose online learning to enable training on constrained devices, adapting local models toward the latest field conditions. (2) Nevertheless, current on-device learning methods struggle with heterogeneous deployment conditions and the scarcity of labeled data when applied across numerous devices. We introduce federated meta-learning incorporating online learning to enhance model generalization, facilitating rapid learning. This approach ensures optimal performance among distributed devices by knowledge sharing. (3) Moreover, TinyML’s pivotal advantage is widespread adoption. Embedded devices and TinyML models prioritize extreme efficiency, leading to diverse characteristics ranging from memory and sensors to model architectures. Given their diversity and non-standardized representations, managing these resources becomes challenging as TinyML systems scale up. We present semantic management for the joint management of models and devices at scale. We demonstrate our methods through a basic regression example and then assess them in three real-world TinyML applications: handwritten character image classification, keyword audio classification, and smart building presence detection. The results confirm the effectiveness of our approaches from various perspectives, such as accuracy improvement, resource savings, and engineering effort reduction. Siemens, SmartEdgeProject, EUCloudEdgeIoT, European Research Executive Agency (REA), #horizoneu, #TinyML, #machinelearning,  #online, #learning, #federated #metalearning, #edgecomputing, #semanticweb, #knowledgegraph, #industrial #InternetofThings, #iot

33