This Week in AI: Apple won’t say how the sausage gets made | TechCrunch

Hello friends, welcome to TechCrunch’s regular AI newsletter.

Apple made headlines this week in AI.

At the company’s Worldwide Developers Conference (WWDC) in Cupertino, Apple unveiled Apple Intelligence, its long-awaited, ecosystem-wide effort in generative AI. Apple Intelligence includes a variety of features ranging from an upgraded Siri AI-generated emojis From photo-editing tools to removing unwanted people and objects from photos.

The company promised that Apple Intelligence is being built with security as well as highly personalized experiences in mind.

“It has to understand you and be grounded in your personal context, such as your routine, your relationships, your communications, and more,” CEO Tim Cook said during a keynote address on Monday. “All of this goes beyond artificial intelligence. It’s personal intelligence, and it’s the next big step for Apple.”

Apple intelligence is classical Apple: It hides nuanced technology behind obvious, intuitively useful features. (Not once did Cook utter the phrase “big language models”.) But as someone who makes his living writing about the underbelly of AI, I wish Apple would be more transparent – just this once – about how the sausage was made.

Take Apple’s model training practices, for example. Apple revealed in a blog post that it trains the AI ​​models powering Apple Intelligence on a combination of licensed datasets and the public web. Publishers have the option to opt out of future training. But what if you’re an artist and are curious to know if your work was included in Apple’s initial training? Unfortunately — it’s better to stay quiet.

This secrecy may be for competitive reasons. But I suspect it is also to protect Apple from legal challenges – particularly those related to copyright. Courts have yet to decide whether vendors like Apple have the right to train on public data without compensating or crediting the creators of that data – in other words, whether the fair use doctrine applies to generative AI.

It’s a little disappointing to see Apple, which often touts itself as a champion of common-sense tech policy, so blatantly embrace the fair use argument. Behind the marketing curtain, Apple can claim it’s taking a responsible and balanced approach to AI, while training it on creators’ works without permission.

A little clarification would go a long way. It’s a shame we haven’t gotten any clarification yet – and I don’t expect we’ll get any soon, other than a lawsuit or two.

news

Apple’s top AI features: Here’s a list of the top AI features Apple announced during its WWDC keynote this week, from improved Siri to deep integration with OpenAI’s ChatGPT.

OpenAI appoints officers: OpenAI this week appointed Sarah Friar, the former CEO of hyperlocal social network Nextdoor, as its chief financial officer, and Kevin Weil, who previously led product development at Instagram and Twitter, as its chief product officer.

Mail, now with more AI: This week, Yahoo (TechCrunch’s parent company) updated Yahoo Mail with new AI capabilities, including AI-generated summaries of emails. Google recently introduced a similar generative summary feature — but it’s behind a paywall.

Controversial views: A recent study from Carnegie Mellon found that not all generative AI models are created equal — particularly when it comes to treating polarizing subject matter.

Sound generator: Stability AI, the startup behind AI-powered art generator Stable Diffusion, has released an open AI model for generating sounds and songs, which they claim was trained exclusively on royalty-free recordings.

Research Paper of the Week

Google believes it can create a generative AI model for personal health — or at least take initial steps in that direction.

in a new research paper Featured on the official Google AI blogResearchers at Google have unveiled the Personal Health Large Language Model, or PH-LLM for short – an advanced version. Google’s Gemini modelThe pH-LLM is designed to provide recommendations for improving sleep and fitness by reading heart and breathing rate data from wearable devices such as smartwatches.

To test the ability of pH-LLM to make useful health recommendations, the researchers created nearly 900 case studies of sleep and fitness involving people living in the United States. They found that pH-LLM made sleep recommendations that were highly useful. Close — but not as good as recommendations from human sleep experts.

The researchers say that PH-LLM could help contextualize physiological data for “personal health applications.” Google Fit comes to mind; I wouldn’t be surprised to see PH-LLM eventually power some new feature in a fitness-focused Google app, Fit or another.

Model of the Week

Apple has devoted quite a bit of blog copy to explaining in detail its new on-device and cloud-bound generative AI models that make up its Apple Intelligence Suite. Yet no matter how long the post is, it reveals very little about the models’ capabilities. Here’s our best attempt to understand it:

The nameless on-device model highlighted by Apple is small in size, no doubt so it can run offline on Apple devices like the iPhone 15 Pro and Pro Max. It includes 3 billion parameters — “parameters” are the parts of the model that essentially define its skill at a problem, like generating text — making it comparable to Google’s on-device Gemini model Gemini Nano, which comes in 1.8-billion-parameter and 3.25-billion-parameter sizes.

The server model, meanwhile, is bigger (Apple won’t say exactly how much bigger). to do Most importantly, it’s more capable than the on-device model. On benchmarks listed by Apple, the on-device model performs on par with models like Microsoft’s Phi-3-mini, Mistral’s Mistral 7B, and Google’s Gemma 7B, but the server model “compares favourably” to OpenAI’s older flagship model, GPT-3.5 Turbo, Apple claims.

Apple also says that both the on-device model and the server model are less likely to derail (i.e., cause poisoning) than similarly sized models. That may be so – but this writer is reserving judgement until we have a chance to test Apple Intelligence.

grab bag

This week marked the sixth anniversary of the release of GPT-1, the ancestor of OpenAI’s latest flagship generative AI model, GPT-4o. And while Deep learning may be hitting a wallIt’s incredible how far this field has come.

Let’s say it took a month to train GPT-1 on a dataset of 4.5 gigabytes of text (the BookCorpus, which contains ~7,000 unpublished fiction books). GPT-3, which is roughly 1,500 times the size of GPT-1 in parameter count and vastly more sophisticated in the prose it can generate and analyze, took 34 days to train. How’s that for scaling?

What made GPT-1 revolutionary was its approach to training. Previous techniques relied on vast amounts of manually labeled data, which limited their usefulness. (Manually labeling data is time-consuming and laborious.) But GPT-1 was not like that; it trained primarily on data. Not Labeled Learn how to perform a variety of tasks using the data (e.g., writing an essay).

Many experts believe that we won’t see a meaningful change like GPT-1 in the near future. But then again, the world didn’t expect GPT-1 to arrive either.