This Week in AI: OpenAI moves away from safety | TechCrunch

Keeping up with a fast-moving industry aye A tall order. So until an AI can do this for you, here’s a handy roundup of recent stories in the world of machine learning, as well as notable research and experiments we haven’t covered ourselves.

By the way, TechCrunch is planning to launch an AI newsletter soon. stay tuned. In the meantime, we’re upping the rhythm of our semiregular AI column, which previously appeared twice a month (or thereabouts), to weekly – so be on the lookout for more editions.

This week in AI, OpenAI once again dominated the news cycle (despite Google’s best efforts) with a product launch, but also, with some palace intrigue. The company unveiled GPT-4o, its most capable generator model to date, and just days later effectively disbanded a team working on the problem of developing controls to prevent “superintelligent” AI systems from malfunctioning. done.

The team’s disbandment predictably generated a lot of headlines. Reporting– including us – suggests that OpenAI prioritized the team’s security research in favor of launching new products like the aforementioned GPT-4o, which ultimately led to resign Of the two co-heads of the team, Jan Leik and OpenAI co-founder Ilya Sutskever.

Superintelligent AI is more theoretical than real at this point; It’s unclear when — or if — the tech industry will make the breakthroughs needed to create AI capable of completing any task performed by a human. But this week’s coverage seems to confirm one thing: OpenAI’s leadership – specifically CEO Sam Altman – has chosen to prioritize products over security measures.

Altman reportedly “distressedSutskever was quick to launch the AI-powered features at OpenAI’s first dev conference last November. and he is Having said A paper co-authored by Helen Toner, director of Georgetown’s Center for Security and Emerging Technologies and former OpenAI board member, took a critical look at OpenAI’s approach to security – to the extent that Until he tried to push her. Board.

Over the past year or so, OpenAI let its chatbots store fill with spam and (allegedly) Data scraped from YouTube Expressing an ambition to let its AI generate illustrations against the platform’s terms of service obscene And blood. Certainly, it seems the company has left security behind – and a growing number of OpenAI security researchers have come to the conclusion that their work would be better supported elsewhere.

Here are some other AI stories worth noting from the past few days:

• OpenAI + Reddit: In more OpenAI news, the company inked a deal with Reddit to use the social site’s data for AI model training. Wall Street welcomed the deal with open arms — but Reddit users might not be so happy.
• Google’s AI: Google hosted its annual I/O developer conference this week, during which it debuted But of AI products. we surrounded them HereFrom video-generating VOs to AI-arranged results in Google Search and upgrades to Google’s Gemini chatbot apps.
• Anthropic hired Krieger: Mike Krieger, one of the co-founders of Instagram and, most recently, co-founder of the personalized news app Distortion proof (which was recently acquired by TechCrunch corporate parent Yahoo), is joining Anthropic as the company’s first chief product officer. He will oversee both the company’s consumer and enterprise efforts.
• AI for Kids: Anthropic announced last week that it will start allowing developers to create kid-focused apps and tools built on its AI models — as long as they follow certain rules. In particular, rivals like Google don’t allow their AI to be built into apps for younger users.
• AI Film Festival: AI startup Runway held its second AI Film Festival earlier this month. Takeaway? Some of the more powerful moments in the showcase came not from the AI, but from the more human elements.

More Machine Learning

AI security is clearly top of mind with the departure of OpenAI this week, but Google DeepMind is working on it With a new “Frontier Safety Framework”. Basically it’s the organization’s strategy to identify and stop any runaway capabilities – it doesn’t have to be AGI, it could be a malware generator gone mad or something like that.

The framework has three steps: 1. Identify potentially harmful capabilities in a model by simulating development paths. 2. Regularly evaluate models to determine when they have reached a known “critical capability level.” 3. Implement a mitigation plan to prevent intrusion (by others or yourself) or problematic deployment. There are more details here, This may seem like an obvious series of actions, but it’s important to formalize them or everyone is just winging it. That’s how you get bad AI.

A different risk has been identified by Cambridge researchers, who are rightly concerned over the proliferation of chatbots that train on data from a deceased person to provide a superficial simulacrum of that person. You (as I think) may find the whole concept somewhat disgusting, but if we are careful it can be used in grief management and other scenarios. The problem is that we are not being careful.

“This area of ​​AI is an ethical minefield,” Lead researcher Katarzyna Nowaczyk-Basińska said, “We need to start thinking now about how we mitigate the social and psychological risks of digital immortality, because the technology is already here.” The team identifies several scams, potential bad and good outcomes, and generally discusses the concept (including fake services) Paper published in Philosophy and Technology, Black Mirror predicts the future once again!

Among the less scary applications of AI, physicist at mit Are looking for a useful (to them) tool to predict the phase or state of a physical system, typically a statistical task that can be difficult with more complex systems. But train a machine learning model on the right data and ground it with some known content characteristics of the system and you have a significantly more efficient way to go about it. This is another example of how ML is finding a niche in advanced science as well.

At CU Boulder, they are talking about how AI can be used in disaster management. The technology can be useful for quickly predicting where resources will be needed, mapping damage, even helping to train responders, but people are (apparently) struggling to apply it to life-and-death scenarios. Hesitating.

Professor Amir Behzadan Trying to move the ball forward on that, it says, “Human-centered AI leads to more effective disaster response and recovery practices by fostering collaboration, understanding, and inclusivity among team members, survivors, and stakeholders. ” They’re still in the workshop stage, but it’s important to think deeply about this stuff before trying to automate aid delivery after a hurricane.

Finally some interesting work from Disney Research, which was looking at how to diversify the output of diffusion image generation models, which can produce similar results repeatedly for some signals. Their solution? “Our sampling strategy denoises the conditioning signal by adding scheduled, monotonically decreasing Gaussian noise to the conditioning vector during estimation to balance diversity and position alignment.” I couldn’t put it better myself.

As a result the image output has a very wide variety in angles, settings and general look. Sometimes you want it, sometimes you don’t, but it’s nice to have the option.