AR, meet ML: IKEA app lets you erase and replace your furniture

You might have seen augmented reality (AR) mobile apps that allow you to place 3D models of furniture in a camera view of your home, but a new app from IKEA will take that idea to a new level with the help of machine learning (ML).

Called IKEA Kreativ, the app allows you to take an accurate 3D scan of your room, then remove existing furniture items or clutter by replacing them with IKEA products you want to see in the space.

The delete-a-piece-of-furniture capability is reminiscent of the Google Pixel 6's Magic Eraser and iOS 16's upcoming "lift subject from background" features for smartphone photos. Those features are driven by similar AI/ML technology.

For now, the app will only be available on iOS devices (and the web). But IKEA plans to release an Android version in the next couple of months. iPhones and iPads with lidar sensors will work best, but other iOS devices are also supported.

IKEA Kreativ is built on machine learning and spatial imaging technologies developed by Geomagical Labs, an AI company acquired by IKEA's holding company, Ingka Group.

It's the sort of thing that may show an early hint of what future AR products can do with the help of machine learning and artificial intelligence. But as with most early attempts, there are some limitations.

First, the app has to start with a somewhat laborious, multi-step room-scanning process that involves pointing your phone around the room and moving it in a figure-eight motion. The process can take minutes rather than seconds.

With lidar and recent developments in ARKit, Apple has greatly reduced this scanning time on some devices for most basic and common AR applications, but the process is more involved for this app.

There's another notable limitation, too. You might already be familiar with IKEA Place, an existing app that lets you place furniture in your home using your phone's camera view. That app essentially allowed you to walk around the placed furniture and see it as if it were right there in front of you. IKEA Kreativ can't do that. Rather, it puts the furniture inside static images. Since the functionality is a bit different, the IKEA Place app will remain available.

IKEA is far from the only retail company offering AR furniture placement apps. Amazon, Target, and others have released AR apps, and on Thursday, Walmart announced an app release that provides the feature.

While Google and Apple have put a lot of work into building hardware and software features, as well as SDKs for app-makers, to facilitate mobile AR experiences, not that many killer apps have emerged for smartphone AR.

That might be because many high-potential ideas would be more appealing with comfortable AR glasses rather than a smartphone screen, and the market hasn't seen mainstream, high-quality AR glasses just yet. Apple, Google, Amazon, and other big tech companies are all developing mixed reality headsets that could take these sorts of AR apps to new heights of popularity, but none seem close to release.

Nonetheless, these shopping apps have emerged as one of the more popular uses of smartphone AR.