A few years ago, Apple shocked the mobile market by shipping its latest iPhones and iPads with LIDAR sensors. As of now, they are the only ones making this feature available in-game for mobile phones. However, this technology has a long history, and many modern mobile phones include features based on the same principle. So what is LIDAR and what can you do with it?
What is a rider?
Lidar is an acronym for LIght Detection And Ranging and works on the same principle as radar. In both systems, a transmitter emits electromagnetic waves (laser light in the case of lidar, radio waves in the case of radar), and when the electromagnetic waves hit an object, a sensor listens for (detects) the reflection. Based on the time between transmission and reception, the distance (range) between the reflecting object and the wave source is determined.
The first LIDAR systems appeared shortly after the invention of lasers in the early 1960s. By the 1970s, NASA was using his LIDAR to measure the moon’s topography. Today, everyone uses lidar, from archaeologists discovering lost cities in the jungle to self-driving car manufacturers that allow their vehicles to sense objects in front of them.
Oh, and it’s even included in the latest iPhone Pro.
mobile phone with laser beam
Laser-equipped phones started appearing in 2014 with LG’s G3, used as a time-of-flight sensor to assist with autofocus. This is essentially his most basic use of LIDAR, where a single infrared laser is emitted to measure the distance to the subject of the photo. That information is used to automatically adjust the camera’s focus.
Source: Wikimedia Commons/RCraig09 CC BY-SA 4.0
Three years later, Apple launched the iPhone X with the TrueDepth camera system that enabled revolutionary Face ID functionality. While Apple wasn’t the pioneer in using facial recognition to unlock phones (that honor should go to his Android 4.0 in 2011), the company’s The TrueDepth system has completely changed the game. While Android’s first attempts at this technology, such as being able to unlock your phone using a photo, were notoriously insecure, Apple’s foray into this space has been surprisingly strong. You can tell twins apart while wearing sunglasses and hats.
How did they achieve this amazing technology? Lasers. The TrueDepth system shines a patterned array of 30,000 infrared lasers onto your face. The laser is detected by an infrared camera and analyzed by AI software, producing a detailed 3D scan of your face in seconds. They differ from time-of-flight sensors because they do not measure the time it takes for the laser light to return to the camera. He measures how the laser’s pattern (structured light) is deformed by objects in the scene, and based on that information he infers 3D data.
In 2020, three years after dropping the TrueDepth bomb on the market, Apple introduced its own LIDAR system. Like the TrueDepth system, it emits an array of dozens of lasers (also in a known pattern), but instead of measuring how the light transforms, a dedicated sensor detects how each beam hits the sensor. Measure the time it takes for the reflection to occur. These time-of-flight measurements are combined with data from the phone’s motion sensors and camera to generate a 3D scan.
What good is it, Rider?
Apple’s LIDAR technology is great, but what can you do with it? After all, while there’s a lot of it, it’s a bit niche, and none of the use cases are similar to how most of us interact with our phones. It’s not like I’m going to change my method. More importantly (especially for manufacturers), it is less likely to influence the decision-making process when buying a new phone.
The most common use of Apple’s LIDAR system is as a simple time-of-flight sensor to speed up rear camera autofocus. Most cell phone cameras in recent years have used contrast-based or phase-detection autofocus to keep images sharp, but these techniques rely on sufficient light to work. Time-of-flight sensors have no such limitations, so phones equipped with these sensors also have an advantage in low-light photography.
Apple also uses its LIDAR in two first-party apps. The Measure app uses LIDAR data, augmented reality, and simple trigonometry to take precise measurements of people and objects.
The Magnifier app also has a detection mode that uses AI and LIDAR to help visually impaired people detect the presence and distance of doors and people.
Most third-party apps that use Apple’s LIDAR fall into one of three categories: interior design, 3D scanning, and augmented reality games. Ikea Place is the current king of interior design apps. We use augmented reality to visualize how IKEA products will look in situ, taking much of the guesswork out of knowing whether that rug will bring a room together. If you’re interested in measuring indoor spaces, choose Canvas, which allows you to scan a room and export the data to CAD software.
For 3D printing enthusiasts and game makers looking for 3D models, there are Scaniverse and Polycam. By scanning objects from multiple angles, you can build 3D models that can be exported to popular design software. Scaniverse is free and Polycam has a free tier, but some of its features are locked with a subscription, so which one you choose comes down to personal preference and wallet size.
Source: Scaniverse.com
If you want to play games using the latest technology, you’re in luck. One of the best LIDAR-enabled games on the App Store is RC Club. It allows you to drive a virtual RC car in your living room. When using a LIDAR scanner, you may run into walls or drive over obstacles. The virtual RC car also reacts to the road it is driving on, slowing down when it hits the carpet.
where can i get the rider
Apple’s LIDAR system is only available on iPhone Pro and iPad devices released after 2020, so it’s not widely available. Nothing similar exists for Android smartphones, and standalone systems cost hundreds of dollars. While this exclusivity is a clear win for Apple, it could also negatively impact software-based innovation and prevent LIDAR from expanding beyond the niche position it currently occupies. On the other hand, the work Apple has done to make these systems small and robust enough to carry in your pocket means it may not be long before every new phone is equipped with his LIDAR system. It means that you cannot do it.