Ideas for the future of Vision Pro

Continuing the discussion from Impressions and Thoughts from Early Vision Pro Reviews:

This got me thinking about what kind of AVP-unique apps/features might make it a better product for the future.

Immediately coming to my mind is a way for two or more people with AVPs to share content.

Imagine, if you will. You’re wearing your AVP. Someone else enters the room with one on. Your AVP detects the other one (maybe highlight his goggles in a color to call it out). You click on his unit and select “request sharing”. If he accepts, you can now start sharing content.

For instance, tag a window/screen/app as “shared”. So now it appears on both your devices. Maybe provide the option so each person can place it in his own spatial location, or put it in a fixed location for you both, so you can both walk around it and point to things, much like you would on a markerboard.

Maybe extend the concept to allow group-sharing. So any number of people can share content. Or a “classroom” mode where one person controls a single shared screen that everybody else joining the classroom can see (and manipulate when given access by the owner).

Another related idea. I’ve seen several videos about people using an AVP while walking down the street. Not approved usage, but people are still doing it.

Imagine now a future situation where AVPs or other related devices are common. You look at a billboard (or sign on a bus shelter, or subway station, or whatever). The AVP detects that there is content available to take home (maybe detecting a QR code or a Bluetooth/NFC beacon) and presents a virtual “flyer” document, which you will see when you look at the billboard. You can grab the flyer, look at it, interact with it, and if you like it, save a copy to view later.

Maybe something similar for exchanging contacts. So someone advertising their contact info (e.g. a virtual nametag at an event) would present as an object on their person that you can view and take a copy of (which would probably be internally represented as a VCard object to be added to your Contacts app).

You could then amp it up by integrating with a retail store. Maybe you’re in the grocery looking at items on the shelf. You want more information about something, so you look at it and make a gesture to get more information. The AVP identifies it (visual detection, NFC, bar code) and lets you pull up a web page of information about it (from the manufacturer or from the retailer or maybe third-party review sites).

Lots of interesting possibilities for this tech as it evolves from today’s “version 1.0” product.

1 Like

The problem, of course, is spam and content overload. I imagine that today’s Vision Pro environment probably has a lot in common with the early, pre-spam days of email or pre-AOL USENET. You can be sure that digital scammers and marketers are hard at work imagining how to use and abuse AVP tech.

I remember a video from a few years ago imagining what a walk down the street or a visit to the supermarket might become. I’m not sure if this is the video I am thinking of, but if not, it comes close:

Dystopia.

2 Likes

Wow. I hope it never comes to that, but it does remind me of what web browsing was like before ad blockers were invented, or computers run by people who click to install everything they see - creating the need for anti-malware software.

Yes, the dystopia presented (except for getting stabbed in the hand as a part of stealing points) could come about as a result of spammers abusing the ideas I mentioned (which are far from unique). But I’d like to think that before any situation would get that bad, you would find appropriate security software and protections built-in to operating systems (like you see today), to make sure you have to actually authorize installation of such a mess.

1 Like

Is there a way to stave off such a future for AR? I want to say that Apple would never allow such pollution of the real world with digital overlays, but it’s hard to imagine how it could be effectively regulated. Could government do it? Seems unlikely in the US, at least. Could the industry self-regulate? No chance. Could Apple keep such things out of the App Store? To an extent, but there are a lot of trashy apps in the App Store now, and it’s easy to see marketers paying people (in some way or another) to allow that kind of advertising. (“Free Vision Air if you allow MegaAds!”)

1 Like

I don’t think this can be prevented, except by the users themselves.

Note that many things depicted in that video are features we may actually want. Like giving walking directions by animating the ground. Or alerting you when it’s time to get off the bus. Or telling you to get out of a crosswalk when the light is about to change. Or letting you know that some items on a store shelf are on sale.

The dystopia happens when everything, plus a lot of spam, is all turned on at once.

But we’ve already experienced this in the realm of web browsing. There was a time when people were installing plugins for all kinds of things. And some users were installing everything some random banner ad told them to install. But then the world wised up. People started installing ad blockers. Malware scanners became popular. Browser makers eliminated plugins and added extra security controls for their various “add on” mechanisms. And people without technical savvy asked their children, friends and co-workers for help deleting the junk.

My prediction is that we’ll probably see the same in the AR/VR space. But I think users and manufacturers will get on top of it much faster than they did for web browsers. Partly because everybody is expecting abuse and the big players are going to make preemptive plans in order to avoid destroying their brands. And partly because the hardware is too expensive - by the time it becomes affordable, the early adopts will have been abused and will have responded.

But ultimately, it is going to come down to each individual user to decide whether or not to install junk apps. Just like it is today on phones, tablets, laptops and everything else.

2 Likes

I still worry because even with all the ad blockers and whatnot, it’s still common to be forced to turn those things off to read an article. My concern is that there will be a lot of slippery slopes, where if you want something helpful, like a pop-up over a shelf of apples in a store display that tells you where they come from or how fresh they are, you’ll end up having to put up with ads throughout the store. At first, there will only be a couple of ads so the company can pay for collecting data about where the produce was grown and how fresh it is. And maybe the company will even be forward-thinking enough to offer subscriptions, but then it will find that it can make more money selling the ads so there will be more and more ads. And then they’ll get an offer they can’t refuse to sell location and other user data to a data broker.

And if every app does that, it will require constant effort to keep the ads to a dull roar. Sigh…

3 Likes

This may be an “it can already do that” suggestion, but the thought occurred to me that if a drone app (e.g., DJI Fly) could run on, and was optimized for Vision Pro, that could be a killer app. One of the things that keeps me from flying mine is that using that app on my phone outdoors is almost impossible. (Must be just my eyes, because others seem to manage.)

1 Like

I’d prefer if it could give first person view from the drone and allow the vision pass through of AVP to see the controller (although I suppose AVP could do both on separate ‘screens’). It wouldn’t comply with VFR (visual flight rules) but I suspect the authorities will need to change as the tech develops.

I have a few drones but I use either iPad or Pro controller as I never liked using a phone.

1 Like

The Vision Pro can load iPhone and iPad apps if the developer agrees. It’s not clear if this agreement is opt-in or opt-out). Several DJI apps show up in the Vision Pro App Store. Obviously, there may be issues that make these less than satisfactory, and the compatible apps do not optimally utilize the Vision Pro’s features. But that may be good enough.

To a certain extent, that ship has sailed:

Obviously, this is an extreme example, but the physical world is pretty overwhelmed with ads already and we’ve normalized it enough not to be bothered.

It’d be interesting if the future Vision Pros could actually help with the physical world by “ad-blocking” on a street like this.

1 Like

For those of us with glasses, a version that automatically shifts correction depending on how far you’re looking.

1 Like

I wouldn’t cite Times Square as an example. It’s only a few blocks of the city. Most of Manhattan is nothing like that.

Every big city has a small region of information overload, usually in a big tourist district. Like Toyko’s Ginza or “the strip” in Las Vegas. But these are not typical of the rest of the city.

I would be far more afraid of an AR application designed to selectively delete real-world content. I can’t imagine any such app where third parties would be able to withstand the temptation to abuse that kind of power.

Would such a thing be necessary? You’re not viewing the real world through a lens. You’re seeing an image displayed on screens in the device. The distance to those screens through the device’s optics is going to be the same no matter what you’re looking at. So I don’t see why your eyes would need to adjust the correction based on the real-world distance to what you’re looking at.

Now, if this was a different kind of AR, where you have an optical path to reality, with the device projecting overlays to your eyes, that would be different. Your eyes would be refocusing on different real-world objects, and the overlays would have to adjust. But that’s not what the AVP is doing.

As I said, it’s an extreme example. Nonetheless, every time you walk down a street of shop fronts or drive down a road with stores on either side, you’re seeing advertising of a sort, if not at that level. It’s just that it’s so normal we don’t recognize it.

The iPhone didn’t have third party apps when it started, either. Things evolve.

We’re talking about the future of the VP, so this would fit with a version that lets you see the real world but adjusted in the way I mentioned.

Perhaps he is talking about a pair of glasses with autofocus, something I would have liked when I was in the throes of presbyopia (essentially fixed focus eye lenses).

Well, I’m not sure I’d say not bothered—I find Times Square to be sensory overload at the best of times, and there’s a lot of legislation limiting outdoor advertising and concern about the topic.

The Times Square ads presumably make economic sense for the companies buying them because of the massive foot traffic and impressions. And it seems likely that if there’s a business model for companies to display ads in AR space, they’ll do everything they can to make that happen.

1 Like

A post was split to a new topic: What is the Vision Pro doing optically?

And it looks like content will be able to be created and edited directly in Final Cut Pro and Logic Pro on Vision Pro:

Isn’t there some sort of “watch a video together” feature in iOS? I’ve never used it, but that was one of the first things I thought of. If my friend and I both have AVPs and are in the same room together, or even if we weren’t, it seems like Apple already has the infrastructure to do some sort of “watch a virtual 100” tv show/movie" together feature.