A NOTE FROM THE EDITOR

Hello hello and welcome back to Datakami’s Bot Bulletin! Last month, we celebrated AI’s win at the Nobel prize awards, and we considered some of the changes in the AI market: from decreasing GPU rental prices to the challenges AI founders face. Enjoy your read!

—Alexander

John Hopkins and Geoffrey Hinton.
Niklas Elmehed © Nobel Prize Outreach

Featured: Nobel Prizes for AI Researchers

This year, two Nobel prizes have been given to AI-related research! The Nobel prize in physics went to John Hopfield and Geoffrey Hinton for “foundational discoveries and inventions that enable machine learning with artificial neural networks”. Specifically, these researchers were given the prize for their work on Hopfield networks and Boltzmann machines, respectively. In chemistry, half of the prize was awarded to Demis Hassabis and John Jumper for their work on AlphaFold, an AI system that can predict the 3D shape of proteins.

The GPU Rental Bubble

Considering buying yourself some H100 GPUs? Don't, says the author of this post diving into the GPU rental market. In just 1.5 years, the price of renting H100 GPUs decreased from $8/hr to just $2/hr. This drastic price drop is traced to several changes in the market, such as:

• fewer companies training their own models from scratch,
• large companies building their own clusters instead of renting them,
• more GPUs becoming available for rent,
• cheaper alternatives becoming available for inference.

The conclusion: don’t buy new H100 GPUs.

The Changing Challenges of AI Founders

As Judith explained in this thread , she has noticed a shift in the types of challenges AI founders face. Over time, the focus has shifted from steering models and dealing with hallucinations to selecting which model to use, fine-tuning and finding data, and evaluating performance. Currently, many companies are concentrating on optimizing efficiency and managing their AI infrastructure.

Paper: Large Language Models Reflect the Ideology of their Creators

This paper analyses the differences in ideological stances between many different LLMs. Their results show that these language models respond quite differently depending on where they were developed and in which language they are prompted, reflecting the views of their creators as well as the data they were trained on. As such, the authors argue that the ideological stance of an LLM should be an important factor when selecting which model to use.

A nice analogy on LLM memory size

"Fitting an LLM on a GPU is a bit like photography. Model weights = film sensitivity, activation size = shutter speed, I/O tensors = aperture. These 3 dials control your model's memory footprint, just as they shape a photo's exposure.”

Source: Judith's original post on Mastodon

Book: 'Nexus: A Brief History of Information Networks from the Stone Age to AI’ by Yuval Noah Harari

I am almost done reading this new book by Yuval Noah Harari, who you might know from his previous books such as Sapiens or Homo Deus. In Nexus, Harari analyses the history of human society, culture, and politics through the lens of information networks. By looking at plenty of historical examples, from the invention of the printing press fueling medieval witch hunts to the role of Facebook's algorithms in fueling violence in Myanmar, Harari shows how information technology can be a force for good or evil and what this can tell us about the risks of AI. While the parts on AI often remain a bit shallow and do not offer many new technical insights for readers familiar with AI, Nexus does offer a rich historical context and a unique perspective on information technology. Overall, I enjoyed reading Nexus, and as long as you can appreciate a long historical detour, I highly recommend this book.

Datakami news

Judith recently appeared on the Dutch tech & science podcast Met Nerds om Tafel (“a sitdown with nerds”) to talk about Datakami, generative AI, text generation for games, why neural networks are not scary, and building software tools for investing. Listen to the episode here!

Judith and Yorick are joining the AI Summit Brainport in Eindhoven (NL) on November 7th. Judith will be part of the Matchmaking Track for "connecting AI innovators with industry challenges". Come talk to her if you have an interesting applied generative AI problem to discuss! ;)