How Delta Air Lines Trials "Parallel Reality" by Misapplied Sciences
The same digital board at Detroit Airport can display personalized flight information to different passengers simultaneously
Parallel Reality Displays by Misapplied Sciences is an innovative tech development enabling Personalized Information Boards in public spaces! Delta is trialing the first public application of this new tech at its Detroit Airport. They are using it for flight and baggage claim information. Read on to understand more about this Delta trial and what Parallel Reality is all about!
"Any sufficiently advanced technology is indistinguishable from magic" - Arthur C. Clarke
What is the Parallel Reality trial at the Detroit Metropolitan Airport?
Delta Air Lines is trialing the Parallel Reality Experience since June 29, 2022. It is currently available at the Concourse A of McNamara Terminal at the Delta Airport (DTW). Delta customers need to opt-in to this trial. They can use their Fly Delta app for facial recognition. Or they can scan their boarding pass when they visit the airport. Then they must proceed towards a digital board capable of simultaneously displaying unique travel info for up to 100 travelers. The trial is beginning with a single screen, but Delta will add more screens as they study the results from this trial.
The digital display consists of multi-view pixels that can simultaneously project multiple images in different directions. A person need not stand still. They can move around and continue to see only their own information. At the same time, other people will continue to see their own info on the same display. This is because the system creates "viewing zones" for every specific person, and tracks that person's movements in space. It continues to beam their info towards their viewing zone as they move.
A California-based startup called Misapplied Sciences has developed this multi-view display tech. Misapplied Sciences has operated in stealth mode since 2014. It debuted for public demo at CES 2020. First-hand experiencers have described it as a "mind bending innovation". It seems to be straight out of science fiction. A similar concept was imagined with the personalized advertisements in public places as depicted in the movies Minority Report and Blade Runner 2049.
How did Delta achieve the Parallel Reality Experience?
The Detroit Metro Airport Trial is a partnership between Delta Air Lines and Misapplied Sciences. Delta's systems store the personal information of their customers. And Misapplied Sciences' multi-view display tech achieves the simultaneous beaming of different information to different people.
The multi-view effect is achieved with the help of multi-view pixels that comprise the screen. And by using ceiling-mounted cameras to track the positions of individual "viewing zones". We can also call the multi-view pixel as an "MV pixel", "magic pixel" or "Parallel Reality" pixel. It is a new kind of pixel. It can "simultaneously output up to millions of controllable rays of different colors and brightness". Misapplied Sciences currently has 20 patents to its name, and several more are pending.
Misapplied Sciences itself does not rely on Facial Recognition technology or store any other personal data. It simply relies on its partner Delta to temporarily pull the personal information from their systems. Then it associates this information with its own "viewing zones" for delivering the personalized information board experience. Is there a way to see what information is beamed to another person? Yes, it is to invade their personal space by standing very close to them, or right behind them.
Does the "multi-view" display technology require any specialized headsets or goggles?
We do not require any specialized headsets or goggles to view the "Parallel Reality" displays. Multiple displays are visible from multiple angles at the same time to the plain eye! The technology "magic" behind this feature is a "multi-view pixel". Any single MV pixel can project different colors in different angles at the same time.
Generally, in other screens, a single pixel projects only a single color in all the directions. This helps provide a wider viewing angle for the same image. A "multi-view display pixel" restricts the viewing angle for a given image to a specific viewing position. But the advantage is that multiple images can be simultaneously projected in different directions.
How can we be sure that the "multi-view" displays are real, and not just some photographic trick?
Misapplied Sciences knows how to demonstrate that their system is displaying multiple images simultaneously on the same screen. One method is to record the same demonstration from multiple cameras simultaneously. You can see this lab test with multiple laptops recording the same scene. You can clearly see that different numbers are visible on the same screen from different positions!
They also do this by placing a grid of 12 mirrors in front of the screen. Each mirror reflects a unique image at the same time. It does this naturally by the virtue of its unique position in space in relation to the screen. This proves that the system is beaming different sets of light rays to the different "viewing zones".
For such demonstrations, Misapplied Sciences also places a model in the foreground of the screen. This can be a person, animal, or object, often making some unique motion. This sufficiently proves to the viewer that the multiple displays are real. They are actually seeing them simultaneously on the same screen as part of the physical reality. It is not some photographic or editing trick!
What are the limitations of the "multi-view display" pixels?
Every pixel is projecting multiple colors in different directions. What happens if you go too close to the display? All the light rays from every pixel overlap and enter your vision at once. As a result, you will see a very blurry and mixed-up final image. So you need to be standing at a sufficient distance away from the screen.
The second limitation is that the individual "multi-view display" pixels are still quite large. This limits the maximum number of pixels that we can place on one screen. As a result, we can display only up to a certain maximum resolution of the images. The Misapplied Sciences team will continue to improve their systems. They will reduce the pixel size, and pack more pixels per square unit of the screen area. With that, we can display crisper and realistic images with natural colors in the subsequent generations of "multi-view" displays.
How is the Parallel Reality Experience at the Detroit Metro Airport different from the regular airport experience?
Unnecessary information about many flights and terminals clutters the regular information boards today. The displayed information can also cycle between different languages to serve international travelers. People have to look through long cluttered lists and spot the information specific to them. With Parallel Reality Display, every passenger is show only the relevant, personalized, contextual information in their preferred lanugage. This means they no longer need to work through heaps of irrelevant information about their flight schedule and baggage status. This greatly simplifies the wayfinding experience at the airport and reduces the cognitive load on travelers.
Gradually, Parallel Reality will move out of beta. Delta will apply it across the Detroit Metropolitan airport. Once that happens, travelers will no longer need to look at their smartphones to receive personalized flight information. They will be able to see this required information in their external physical environment. This is a much more natural and straightforward airport experience. This technology is accessible to ordinary people who don't need to be tech-savvy to use it.
People don't have to depend on their smartphones for personalized info. They can give better attention to their surroundings. This will facilitate ease of movement throughout the airport. This will greatly reduce accidents due to lack of attention and stress. The Parallel Reality will thus create an enjoyable and relaxed "walking speed" airport experience for everyone.
What are the limitations and anticipated roadblocks in the Parallel Reality trial at the Delta Metro Airport?
Delta is trialing only one information board right now. Delta is limiting this personalized flight information experience to only up to 100 passengers in a batch. Only up to 10 passengers can simultaneously experience this feature. They can be standing still or moving in front of the Parallel Reality board. This is an opt in experience requiring an extra step of signing up for it in two ways. Customers can choose digital identification via facial recognition technology using the Fly Delta app. Or they can get their boarding pass scanned at a kiosk to experience this experimental feature.
Delta anticipates that the novel nature of Parallel Reality will cause some trouble at first. The customers will play around with it rather than using it naturally or seriously. This can cause unintended delays and excitements at the airport. But the intended end-goal is a swift, frictionless experience. This is part of the reason why Delta is limiting to 100 people in one batch.
Most people will familiarize with this technology and start taking it for granted gradually. It will be similar to how smartphones are ubiquitous today! Ubiquity and naturalization of Parallel Reality Displays will realize the design-intent of Misapplied Sciences. Hopefully, this trial will be a success and provide valuable feedback from customers. If this trial is successful, Delta Air Lines would like to gradually expand the Parallel Reality Experience. They will add many information boards at the Delta Metro Airport, and then across their other airports.
Where does the company Misapplied Sciences get its name from?
The co-founders of Misapplied Sciences are Paul Dietz and Albert Ng. They both formerly worked at the Applied Sciences hardware research group at Microsoft (MS). People also labelled this department as "MS Applied Sciences". Misapplied Sciences was earlier based in Redmond, Washington where Microsoft is also headquartered. Today, Misapplied Sciences is located in Pasadena, California.
You can see short video-clips of the early innovations at MS Applied Sciences on the little known Youtube Channel MSAppliedSciences: https://www.youtube.com/user/MSAppliedSciences/videos
What are the various technological building blocks of Parallel Reality?
Misapplied Sciences uses advanced optical design, precision spatial callibration, high performance parallel computation to build their Parallel Reality displays. Also needed to channel these technological powers are skills and tools. These include experiential design, simplified programming interfaces, and real-time environmental modeling tools.
What are the various potential applications of Parallel Reality Tech?
Parallel Reality Displays unlock a world of endless possibilities. In such a future, we will deliver the right information to the right person, wherever they are, even if it is a crowded public place! Parallel Reality will thus usher in an age of narrowcasting, the opposite of broadcasting. Some of the potential applications include areas like: internationalization & hospitality, marketing, entertainment, wayfinding, traffic signs & signals, legibility, people flow management, architectural lighting & effects.
Traffic Signs & Signals
We can apply Parallel Reality Display for real-world GPS navigation on the outdoor highway signs. Depending on your route, you will be getting the accurate directions specific to your needs. At the same time, other cars will get their own navigational directions. We can personalize traffic signals depending on the speed of a car. A fast car closer to the intersection will continue to see a green light. A slower car further from the intersection will see a red light. This will make the roads safer by cutting off any ambiguities in traffic signals. Today, an orange light can cause drivers to speed up unwisely and end up in accidents.
People Flow Management
Parallel Reality Display will benefit any context where we need to co-ordinate a group of people. Whether it is group-based games or crowd-control through a public place like a museum or bus station. Parallel Reality displays will serve a great way of beaming appropriate information to every person. This will help to co-ordinate their movements and decisions.
Internationalization & Accessibility
In a movie theatre, you will see closed captions in your preferred language. In fact, you will see all the information in all public spaces in your language. Such a world will be a friendlier and more accessible place to navigate and understand. Imagine that you are a Colombian tourist in China. You will see all the information in Spanish as you travel throughout China! Wouldn't that make your life easier? We can personalize even the font size of the information for your vision. That way, you can always read everything comfortably.
Real-world 3D Graphics
The current focus of Misapplied Sciences is on directing individualized information to a specific person in a crowd. So the team has engineered and optimized all of their interlocking systems to achieve this purpose. In the future, Misapplied Sciences team can optimize the same system for transmitting 3D graphics by re-engineering their algorithms for stereoscopic vision. In that case, slightly different images of the same object will enter each eye of the viewer. And thus, you will experience 3D without any special glasses, lenses, or headsets, simply with your naked eyes!
What are some of the ethical concerns about multi-view technology?
Personalized advertisements and propaganda that follows you is a reality today because of trackers and cookies on websites. But at least we can turn off our devices. But imagine if such negative or manipulative information follows you in the physical world wherever you go. Now that can be a scary prospect!
Imagine a future world where everyone looks at the same thing but sees different information. Agreeing on anything could become impossible! A world of Parallel Reality Displays will feature a widespread usage of personalized information. In such a world, malicious entities will find more points to breach privacy and security.
But we can counter every scary imagination regarding ethics and privacy with good intent and design. It's an exciting and challenging future for sure!
Parallel Reality Videos Playlist by Blogsaays
We have compiled a Youtube Playlist about Parallel Reality, Misapplied Sciences, and Delta trial. You can watch and learn more about Parallel Reality Displays here: