Watch a smartphone camera app take objects from the real world and paste them into Photoshop

An impressive, futuristic demonstration was posted on Twitter, showing how one can snap a photo of an object with their smartphone camera, and the subject can be automatically cut from the background, and placed inside a Photoshop project, all thanks to machine learning and other software working in tandem.

The demonstration was showcased and developed by artist and programmer Cyril Diagne, who made the software in question, named "AR Cut & Paste."

In order for the software to work, a mobile app, a local server connecting it with Photoshop on a computer, and a service for detecting objects and removing backgrounds from photos are used. In addition, an algorithm for detecting where the smartphone camera is pointing at, in relation to the Photoshop project on the computer screen, helps paste the object in the correct spot.

This is an impressive look into a more streamlined future of design and image editing, and it's not a stretch to imagine Photoshop and other flagship software supporting a feature like this eventually.

Especially now that Apple added a LiDAR sensor to its iPad, its main purpose being accurately detecting objects in the world surrounding us, and supporting AR (Augmented Reality) applications, first-party support for features like this are a no-brainer, and would be a game changer.


The aforementioned AR cut and paste software by Diagne may support more than only Photoshop in the future. Unfortunately, his software showcased here is a research prototype and not a consumer user tool. And until large software companies start implementing AR like this, making it accessible to all users, we can rely on individual programmers to provide such tools for only the more tech savvy among us, if for nothing else, just for impressive proof of concept.