Become a MacRumors Supporter for $50/year with no ads, ability to filter front page stories, and private forums.

MacRumors

macrumors bot
Original poster
Apr 12, 2001
62,932
29,557


Apple researchers have released a new open-source AI model that is capable of editing images based on a user's natural language instructions (via VentureBeat).

DALL·E-apple-logo-image-editing-ai.jpg
MacRumors image made with DALL·E

Called "MGIE," which stands for MLLM-Guided Image Editing, it uses multimodal large language models (MLLMs) to interpret user requests and perform pixel-level manipulations.

The model is capable of editing various aspects of images. Global photo enhancements can include brightness, contrast, or sharpness, or the application of artistic effects like sketching. Local editing can modify the shape, size, color, or texture of specific regions or objects in an image, while Photoshop-style modifications can include cropping, resizing, rotating, and adding filters, or even changing backgrounds and blending images.

A user input for a photo of a pizza could be to "make it look more healthy." Using common sense reasoning, the model can add vegetable toppings, such as tomatoes and herbs. A global optimization input request might take the form of "add contrast to simulate more light," while a Photoshop-style modification could be made by asking the model to remove people from the background of a photo, shifting the focus of the image to the subject's facial expression.

Apple collaborated with University of California researchers to create MGIE, which was presented in a paper at the International Conference on Learning Representations (ICLR) 2024. The model is available on GitHub, and includes the code, data, and pre-trained models.

MGIE-apple-AI-model-image-editing.jpg

This is Apple's second breakthrough in AI research in as many months. In late December, Apple revealed that it had made strides in deploying large language models (LLMs) on iPhones and other Apple devices with limited memory by inventing an innovative flash memory utilization technique.

For the last several months, Apple has been testing an "Apple GPT" rival that could compete with ChatGPT. According to Bloomberg's Mark Gurman, work on AI is a priority for Apple, with the company designing an "Ajax" framework for large language models.

Both The Information and analyst Jeff Pu claim that Apple will have some kind of generative AI feature available on the ‌iPhone‌ and iPad around late 2024, which is when iOS 18 will be coming out. iOS 18 is said to include an enhanced version of Siri with ChatGPT-like generative AI functionality, and has the potential to be the "biggest" software update in the iPhone's history, according to Gurman.

Article Link: New Apple AI Model Edits Images Based on Natural Language Input
 
  • Like
Reactions: KeithBN

AlmightyKang

macrumors regular
Nov 20, 2023
226
470
Apple as always do something because it has a benefit to the user, not just for the sake of the technology. This is the ML approach we need.

Most of the VC and AI industry are just grifting and trying to sell it as an application for everything and trying to create new markets for it.
 

jimbobb24

macrumors 68040
Jun 6, 2005
3,330
5,349
There’s AI that can do this type of stuff today
Now there is one more. Download it and compare.

Seems a little bit like using control net in stable diffusion and doing a style change. Now sure if a whole model is needed for this. Also many corporate models (like MS Bing creator/Dallle3) are so “sanitized” that they cannot function normally. I wonder where Apple is going to fall on that spectrum.
 

Pezimak

macrumors 68030
May 1, 2021
2,713
2,833
What gets me with all this AI talk, is how much of it Apple will lock behind new devices, even though it'll mostly all be software and cloud based.
 
  • Like
Reactions: Crow_Servo

CJM

macrumors 68000
May 7, 2005
1,531
1,051
U.K.
Pretty cool. The paper describes limitations as being things like compound instructions in the same prompt and it also struggles with numbers, so saying 'change one cake in the picture' doesn't get picked up on.
 
Register on MacRumors! This sidebar will go away, and you'll see fewer ads.