Apple Working On Automated Desktop Pictures

[MacRumors]

http://www.macrumors.com/images/macrumorsthreadlogo.gif (http://www.macrumors.com)

Various sites are reporting on a patent application released today (number 20070085854 (http://appft1.uspto.gov/netacgi/nph-Parser?Sect1=PTO2&Sect2=HITOFF&p=1&u=%2Fnetahtml%2FPTO%2Fsearch-bool.html&r=1&f=G&l=50&co1=AND&d=PG01&s1=20070085854&OS=20070085854&RS=20070085854)) entitled "System and method for computing a desktop picture." The abstract reads:

Disclosed are a system and method for computing a desktop picture. Instead of loading a file that contains the desktop image from memory, the present invention provides for a system and method for opening and retaining a procedural recipe and a small set of instructions that can be executed to compute a desktop picture. The desktop picture can be computed very quickly using a GPU (graphics processing unit), and can be made to move on demand. When a part of the desktop image is needed to composite, that part is computed using a fragment program on the GPU using the procedural recipe and a specially written fragment program into a temporary VRAM buffer. After it is computed and composited, the buffer containing the result of the fragment program may be discarded.

In summary, Apple seems to be leveraging the programmability of today's modern GPUs to be able to create computed desktop pictures that can be constantly updated to changing situations on the user's desktop.

Apple suggests that situations such as logging in or out, waking from sleep, or coming out from a screen-saver are currently too jarring, and the invention hopes to make this experience more seamless. Also mentioned is a tool that would allow designers to edit and specify the user experience.

Apple also seems to be conscious about system resources, mentioning several times about opportunities to free resources for use by other applications.

Patent applications can provide interesting insight into what concepts or technologies Apple is working on in their labs, however readers are reminded that only a subset of these applications will make it into final shipping products.

[Xavier]

Awesome idea, another good reason to wait for Leopard, that is if this will even be used there

[Maccus Aurelius]

I don't find coming out of sleep, booting up or coming out of the screen saver too jarring. Oh well, whatever works.

[c.joe.go]

really impressive technology. seems rather obvious for evolution of the os, but to read more on it and the possible uses of this feature. i would be hopeful to see this feature on a beta @ wwdc07. in that same statement, i would have to include disappointment if this was one of the top secret features.

can't wait to see it. :)

[KingofAwesome]

I'd be interested to see what factors could be used to influence that background image. Are we talking about a visualization like what we see in iTunes (although probably slower) that takes certain events into account (like the cpu utilization, or sound output), or could it be the desktop interacting with windows on top of it (similar to the ripple effect when adding a widget to the dashboard)?

Considering Apple's love of fading reflections in things like Coverflow, I can definitely picture something similar with the desktop. I hope that if they implement it, they'll put enough effort into design so that the options we have will all be usable and really improve the experience. Apple's generally been good at giving us only a few very good choices, as opposed to MS giving users the ability to tweak everything and leaving it up to users to make sane choices.

[Mac-Addict]

What so you wouldnt need to wake from sleep or turn the computer on? Just press a button or move the mouse and its there? I dont really understand xD

[liv4Mac]

Does that mean there is a posibility of having animated icons aswell?

[ziggyonice]

anybody remember Atmosphere (http://mydreamapp.com/contestants/view/cameronwestland/) from the My Dream App contest?

sounds a lot like it.

[Counter]

Does that mean there is a posibility of having animated icons aswell?

If so I hope they're optional. :eek:

[MacVault]

I guess I don't get it. What the heck is so "jarring" when logging in/out of an account in Mac OS X??? I hope this is not the kind of stuff Steve meant when he said "Top Secret Features". I hope this is not what is delaying Leopard til October. I mean, DAMN! Instead of this crap why doesn't Apple do some things that matter - like make it easy to administer a network of 10,000 Macs! Like a real competitor to Active Directory. Like really pursue the enterprise. Like work on making generic PC drivers work with OS X so I can buy a mini tower from Dell for ~$600 and run a supported version of OS X.

:mad:

[MongoTheGeek]

Its like the cube transition from fast user switching.

In fact that is probably what this refers to.

[Macula]

I don't mean to curb our enthusiasm (the idea sounds terrific) but isn't this a feature that "sort of" exists in Vista?

[MacVault]

Its like the cube transition from fast user switching.

In fact that is probably what this refers to.

Ok, so we have that already. What's this new thing about? I'm all for non-"jarring" user interfaces. I guess I just don't get what this is about.

[pdpfilms]

anybody remember Atmosphere (http://mydreamapp.com/contestants/view/cameronwestland/) from the My Dream App contest?

sounds a lot like it.
Was this a real app, or simply just a concept? And if it's real, where does one find it?

[TheSlush]

What so you wouldnt need to wake from sleep or turn the computer on? Just press a button or move the mouse and its there? I dont really understand xD

Probably this wouldn't address actually turning the computer on and off. The screen will still have to go from black to your desktop image at some point.

[TheSlush]

I guess I don't get it. What the heck is so "jarring" when logging in/out of an account in Mac OS X???

My guess is that this refers to, for instance, a smoother transition between screensaver and desktop. Currently with most screensavers, it's often a quick-cut from one to the other (although I don't know if that's what one would necessarily call "jarring" per se). This new technology might allow for subtler, or more entertaining, transitions that may incorporate desktop elements.

Additionally, it could refer to a desktop that is in constant motion, or constantly reacting to user input, or constantly able to display updating information.

[MacVault]

...This new technology might allow for subtler, or more entertaining, transitions that may incorporate desktop elements.

Additionally, it could refer to a desktop that is in constant motion, or constantly reacting to user input, or constantly able to display updating information.

Now that sounds like something Mac OS X needs. This would be very cool!

[hotdamn]

My guess is that this refers to, for instance, a smoother transition between screensaver and desktop. Currently with most screensavers, it's often a quick-cut from one to the other (although I don't know if that's what one would necessarily call "jarring" per se). This new technology might allow for subtler, or more entertaining, transitions that may incorporate desktop elements.

do yourself a favor and google fenetres volantes.
welcome to a year ago.

[mainstreetmark]

To quote that caveman in the commercials,

"Uh....... what?"

Maybe everything's gonna do stuff like the transition between normal mode and Frontrow.

[Analog Kid]

Talk of picture "recipies" brought to mind Core Image-- it's Applespeak for how CI calculates pictures on the fly by passing it through a bunch of filters. It's how Aperture gets away with only storing master images and computing modified images on the fly. The patent does reference CI in the detailed description. Core Animation might figure into this too, but the patent was filed 10/05 which I think was before Core Animation was really in development, no?

A lot of talk about not having to put the background image in RAM since it can be computed on the fly. Is that really such a big deal?

[mobopro]

One other possibility for this patent... perhaps it's to enable remote logins. If the desktop was proceduralized enough, it'd take less bandwidth. With powerful clients taking on the burden of display, it'd mean you could have your Mac wherever you had a network connection to it.

Just a thought.

[Project]

http://creativebits.org/files/time_machine.jpg

An example.

[longofest]

One other possibility for this patent... perhaps it's to enable remote logins. If the desktop was proceduralized enough, it'd take less bandwidth. With powerful clients taking on the burden of display, it'd mean you could have your Mac wherever you had a network connection to it.

Just a thought.

The patent doesn't explicitly say it, but that is a good thought!

[Peace]

This would work great for spaces/windows virtualization ;)

Each space has it's own desktop picture.Transitioning from one space to another space that has a different desktop picture.

[ready2switch]

This may be either a really dumb question or a brilliant one. :p

So, when they say the transitions are too "jarring", are they referring to the user experience or to the hardware/system?

From my limited knowledge and perception, it almost sounds to me like it's something that would make these transitions smoother for the computer, with a sort of side-effect of making the screen transition more smoothly for the user.

Of course, I don't know what I'm talking about. It just seems like a waste of resources to say "the cut between desktop and screen saver just doesn't LOOK smooth enough. Let's fix it!" when there are so many other things waiting to be worked on. :p

[aristobrat]

I don't mean to curb our enthusiasm (the idea sounds terrific) but isn't this a feature that "sort of" exists in Vista?
That's what I thought of too -- Microsoft's DreamScene.

Doesn't sound exactly the same, but IMHO, it's close enough that some people are going to say that Apple copied the idea from Microsoft.


http://www.microsoft.com/windows/products/windowsvista/editions/ultimate/default.mspx
More Windows Ultimate Extras coming soon

* Windows DreamScene, an Extra that enables you to use looped, full-motion video as your desktop wallpaper instead of a static image

[johnee]

since it sends commands to the GPU, would this require specific GPU processors? I know nothing about graphics processors/graphics programming.

[MongoTheGeek]

That's what I thought of too -- Microsoft's DreamScene.

Doesn't sound exactly the same, but IMHO, it's close enough that some people are going to say that Apple copied the idea from Microsoft.


http://www.microsoft.com/windows/products/windowsvista/editions/ultimate/default.mspx

Never mind that my Amiga did that back in 89.

[Object-X]

Welcome to the new OS X UI. Apple is about to completely change the paradigm through which we interact with a computer. :apple:

[chabig]

I don't mean to curb our enthusiasm (the idea sounds terrific) but isn't this a feature that "sort of" exists in Vista?
Not to my knowledge.

[Alfawan]

Sounds like a type of fractal compression. Might be useful for a device with a decent GPU, but low memory count...hmmm

[chabig]

That's what I thought of too -- Microsoft's DreamScene.

Doesn't sound exactly the same, but IMHO, it's close enough that some people are going to say that Apple copied the idea from Microsoft.


http://www.microsoft.com/windows/products/windowsvista/editions/ultimate/default.mspx

Not the same...if you're going to hold up DreamScene as an example, OS X has had the ability to do video desktops since Panther, maybe as far back as Jaguar. It's just not available by default for good reason.

[Objectivist-C]

Was this a real app, or simply just a concept? And if it's real, where does one find it?

You went to the application's homepage and you couldn't figure this out?

[nagromme]

I guess I don't get it. What the heck is so "jarring" when logging in/out of an account in Mac OS X??? I hope this is not the kind of stuff Steve meant when he said "Top Secret Features". I hope this is not what is delaying Leopard til October. I mean, DAMN! Instead of this crap why doesn't Apple do some things that matter - like make it easy to administer a network of 10,000 Macs! Like a real competitor to Active Directory. Like really pursue the enterprise. Like work on making generic PC drivers work with OS X so I can buy a mini tower from Dell for ~$600 and run a supported version of OS X.

:mad:

This is a patent, among many others (most of which never become products), from some time ago. There is no indication that the concept is occupying Apple's time or killing anyone's puppy :)

That's what I thought of too -- Microsoft's DreamScene.

Doesn't sound exactly the same, but IMHO, it's close enough that some people are going to say that Apple copied the idea from Microsoft.


http://www.microsoft.com/windows/products/windowsvista/editions/ultimate/default.mspx

This isn't about video, but about procedural generation.

As for accusations of copying, head them off as follows: OS X has had dynamic desktop pictures in a general sense for years: you choose a folder and the OS will smoothly fade from one image to another at the speed you choose. At the fastest speed it's kind of a cool effect with Apple's "abstract" wallpaper category.

[KingofAwesome]

I wouldn't compare it to Vista's Dreamscene for two reasons - first, that seems to be looping a video clip, and second, it's just "coming soon" for those who bought the most expensive version. Granted, this idea of Apple's is currently not available either, but I wouldn't say Vista has it already.

[iZac]

Anyone remember a patent application a few years back which suggested a dynamic lit skin? (basically the hardware skin was covered in RGB LEDs or some such thing) It would allow the computer to react visually to user inputs / outputs etc. It sounds pretty similar, but this time in software as opposed to hardware

EDIT:

*boom*

link (http://www.macrumors.com/2002/12/27/apple-patent-dynamic-ornamental-appearance/)

[ortuno2k]

Ummm...I didn't really understand any of that stuff, but whatever I think it means sounds cool.:o

[Analog Kid]

since it sends commands to the GPU, would this require specific GPU processors? I know nothing about graphics processors/graphics programming.
No, just a common interface. Core Image piggybacks on Open GL which is a standard 3D graphics interface. If suitable graphics hardware isn't available, Core Image falls back on the CPU to do the heavy lifting.

[aristobrat]

I wouldn't compare it to Vista's Dreamscene for two reasons
Oh, I wouldn't either.

After seeing how some folks screamed that CoverFlow was a blatant ripoff of how Windows Media Player displays album art (which it most definitely isn't), I was just making the point that I think it's likely the same group of folks will probably scream about this vs. Dreamscene.

[debrey]

I can't believe I read a website that, in all sincerity, used the word "leveraging." I'm going to go wash myself.

[puuukeey]

http://equarium.tripod.com/tpot/Clouds.jpg
/ducks
//actually sounds pretty cool... hope the forground changes though

[twoodcc]

sounds nice! looking forward to Leopard......but it's so far in front of me, i can't even see it :rolleyes:

[Konradx]

Sounds interesting. Curious how many of the patents Apple signs they actually use

[whooleytoo]

Like most patents these days, it sounds vague enough to cover almost anything.. :rolleyes:

However, it could be interesting. As suggested above, using this to reduce desktop size when logging in remotely is a very clever idea.

Perhaps subtle Core-Animations on the desktop, a nice subtle water ripple across the desktop would be relaxing rather than distracting.

How about incorporating information into the desktop? Your desktop photo could be a landscape photo, which changes depending on the weather forecast? Details could be overlaid onto it.

However, based on the description it sounds most like your screensaver morphs slowly into your desktop when your Mac wakes.

[Analog Kid]

Duh. Just hit me: resolution independent UI.

Edit: Doh! Just hit me: this is about the background image, not the whole display...

[Dagless]

Was this a real app, or simply just a concept? And if it's real, where does one find it?

Me too. Looked around the site and couldn't find a link anywhere.

[ckinyc]

I would rather see improvements like:

- easier fonts management (for designer like myslef)
- bring back telephony...so we can use our Mac as an automated receptionist
- automatic file syncing betwen multiple Macs
etc...

[notjustjay]

I can't believe nobody has said it yet...

Multi Touch???

[noaccess]

This is the image included in the patent.

http://i13.tinypic.com/40647tj.jpg

See the "Top Secret Project" folder on the desktop? What's described in this patent could be some of the Top Secret stuff... or the guys at Apple are just trying to stir us up, then at WWDC Steve's gonna show everyone the new GarageBand and add another entry for the 'Super Secret Apple Rumors' podcast. If anyone remembers that one. Funny guy :D

[MattInOz]

Probably this wouldn't address actually turning the computer on and off. The screen will still have to go from black to your desktop image at some point.

There are a couple of stages during the boot process where desktop changes completely from one image to another. As i understand it this happens as different parts take command EFI thru to User Space.

I'd imagine in an example of this patent that we would still get the grey screen with Apple Logo. from there each of the transistions from there could evolve out of the last. Say the shapes of the System Desktop could float in to place across the grey then the colour could then fade in. As a replacement to the little bar scrolling across the screen before the login window appears.

[macwatcher]

Apple has always been at the forefront of smooth animation. My work PC still has trouble making stars go by smoothly on that old "flying through space" screensaver.

This patent is a one in a long line of patents related to smooth visuals. My favorite smooth visual transition is when the dock sucks in the app.

SLUUURRRPPP!!!!:D

[zkarj]

Being lazy and only reading the excerpt in this posting it sounds to me just like they are switching to vector-based wallpaper rather than a raster image. After all, procedural graphics from a recipe is just something like SVG.

The 'fragment' bit is then about 'clipping' the image to "composite" - which I assume means calculating the visual effect of transparency without needing to load a bitmap at the start.

The only missing part then is the movement, but that's no big deal and perhaps even distracting - just like a lot of flash animations. Gee, could it just be you can put a .swf file as your wallpaper?

[phillipjfry]

This is the image included in the patent.

http://i13.tinypic.com/40647tj.jpg

See the "Top Secret Project" folder on the desktop? What's described in this patent could be some of the Top Secret stuff... or the guys at Apple are just trying to stir us up, then at WWDC Steve's gonna show everyone the new GarageBand and add another entry for the 'Super Secret Apple Rumors' podcast. If anyone remembers that one. Funny guy :D

But in the window that is open in the picture, it also says "holocaust.gif"
so we have a "Top Secret" folder and pictures with "holocaust" as the name...
Startin to really creep me out...
:)

[ChrisA]

How many of you are old enough to remember computer graphics BEFORE raster displays were common?

It takes at least about a MB of VRAM to hold a raster image for display. You only have to go back to the 1970's to find a time when 1MB was more RAM then even a big university computer would have. Clearly graphic displays back then did not contain 1MB of VRAM. No, they had only enough memory to hold what was called a "display list". The list contained commands like "draw a line" "fill a box with color" or "put this text at location x,y". The display device could hold at most a few thousand commands and would loop over then, over and over and "compute" the image for each refresh cycle. The only image "memory" was the persistence of the phosphor. The main computer would change the list of commands if it wanted to change the display. Rather then working like a TV set as current monitors do the old display list devices would paint the electron beam around on the phosphor randomly as it it were a pen plotter drawing with ink. Shorter display lists had faster refresh times.

Sounds like the new patent doesn't it?

Over time I think ALL of the old mainframe techniques from the 1960's will creep up onto our desktops.

[AvSRoCkCO1067]

This is the image included in the patent.

http://i13.tinypic.com/40647tj.jpg

See the "Top Secret Project" folder on the desktop? What's described in this patent could be some of the Top Secret stuff... or the guys at Apple are just trying to stir us up, then at WWDC Steve's gonna show everyone the new GarageBand and add another entry for the 'Super Secret Apple Rumors' podcast. If anyone remembers that one. Funny guy :D

They are totally messing with us...but seriously that is weird...

[SMM]

I don't mean to curb our enthusiasm (the idea sounds terrific) but isn't this a feature that "sort of" exists in Vista?

No, not really.

[swingerofbirch]

I want to make a couple of points.

1. Emergence of Alternative UIs

2. Desktop as Possible Alternative UI, so that Dock and Traditional OS environment is only left for apps

3. Need for Such a Situation

1. Apple has introduced two shipping alternative UIs: Front Row and Dashboard, where the user leaves all interactivity with the normal OS environment. And it has introduced Time Machine where the user in a way leaves the traditional OS environment (although is still working with files etc).

2. The current method for dealing with files is the Finder, Desktop, Open and Save dialogue boxes, and the trash can in the dock. It is confusing for people like my grandmother who I help on the phone with her iMac quite often. Maybe the desktop will become an "alternative UI" like the ones I previously mentioned. It might be a three dimensional wooden table top, upon which you interact with files, see your disks, and so forth. It would be invoked in some similar manner to Dashboard and would eliminate the use of the Finder. The traditional OS would become just another UI, one that has a dock and applications, but no finder or trash on the dock. That would be handled by the Desktop UI.

3. This would greatly simplify the use of a computer I believe. You have a desktop where all your stuff is, then you have Dashboard where your widgets are, and you have the traditional OS environment where you use your Apps, and finally Front Row for using your media.


Don't be too harsh with this wild idea, it's 2 AM and I'm a bit punchy.

[TheSlush]

do yourself a favor and google fenetres volantes.
welcome to a year ago.

Um... okay... if that is all Apple is going to do with this patent, then the company will have greatly disappointed me.

[badcrumble]

Er, doesn't this just refer to stuff like Front Row and Time Machine?

[kalisphoenix]

do yourself a favor and google fenetres volantes.
welcome to a year ago.

Wow, that's totally what we were all talking about! You're amazing! Thank you, hotdamn, for telling us to look a screensaver up on Google! Oh, thank you!

[Bengt77]

I want to make a couple of points.

1. Emergence of Alternative UIs

2. Desktop as Possible Alternative UI, so that Dock and Traditional OS environment is only left for apps

3. Need for Such a Situation

1. Apple has introduced two shipping alternative UIs: Front Row and Dashboard, where the user leaves all interactivity with the normal OS environment. And it has introduced Time Machine where the user in a way leaves the traditional OS environment (although is still working with files etc).

2. The current method for dealing with files is the Finder, Desktop, Open and Save dialogue boxes, and the trash can in the dock. It is confusing for people like my grandmother who I help on the phone with her iMac quite often. Maybe the desktop will become an "alternative UI" like the ones I previously mentioned. It might be a three dimensional wooden table top, upon which you interact with files, see your disks, and so forth. It would be invoked in some similar manner to Dashboard and would eliminate the use of the Finder. The traditional OS would become just another UI, one that has a dock and applications, but no finder or trash on the dock. That would be handled by the Desktop UI.

3. This would greatly simplify the use of a computer I believe. You have a desktop where all your stuff is, then you have Dashboard where your widgets are, and you have the traditional OS environment where you use your Apps, and finally Front Row for using your media.


Don't be too harsh with this wild idea, it's 2 AM and I'm a bit punchy.
No Trash in the Dock? No Dock at all? Disks only available on the Desktop? No file management in Open and Save dialogs? Some environment with special little tools (or a certain folder containing those)? A Desktop you have to activate (or dig your way towards, or reach for, or whatever you'd like to call it) when you're busy working in any other application?

Sounds a lot like Mac OS 9.

[swingerofbirch]

No Trash in the Dock? No Dock at all? Disks only available on the Desktop? No file management in Open and Save dialogs? Some environment with special little tools (or a certain folder containing those)? A Desktop you have to activate (or dig your way towards, or reach for, or whatever you'd like to call it) when you're busy working in any other application?

Sounds a lot like Mac OS 9.
What I was envisioning for save/open dialogues would be some sort of creative 3d experience whereby when you go to save something, you save it literally to your desktop and that 3d environment appears (a wooden desk) and allows you to bring your document physically to where you want it saved, be it a pile of papers about literature, etc, or just an empty spot on the desk under your coffee cup, whatever. It would somehow integrate both the window from the OS environment and then as it is saved shrink the window to an icon that is on this dekstop, much like Time Machine uses an alternative 3d environmental with the traditional OS windows.

Concerning the dock, etc: what I meant was the environment you are likely in now, using Safari etc, would stay similar as it is now and be just another UI, as is Time Machine, Dashboard (or Desktop in my imagination). So yes, you would still have your dock. All your apps and aliases would still be in it. But, the finder and the trash would not be. The finder and the desktop are one now and they are the Desktop (the 3d environment). As per your comment about having to wade to it, I guess in what I was conceiving you would get there through a mouse click, like designating a mouse button, a key or something for access, or it would appear automatically when you save a document, or open one. The trash would of course be somewhere cute, like underneath the 3d desk..... So basically you have different environments, Desktop environment is for managing files, Time Machine is invoked by a big red emergency button on the Desktop wooden desk, Front Row is for viewing media, Dashboard is for get fast info, and traditional OS is for running all your full-fledged apps...

OK, so I KNOW it's a weird idea, I was just thinking!

[iBookG4user]

This seems like a cool idea, and it increases performance to boot :).

[Bengt77]

OK, so I KNOW it's a weird idea, I was just thinking!
Okay, I see what you mean now. Sounds very interesting. A real digital analogy, this would be. Well, it would be so analogous, that it woulnd't even [/i]be[/i] an analogy anymore. The more I think about it, the more I like it. Don't see how saving or opening files would send you to this desk area, though. Well, in real life, yes, but digitally? Weird, yes, but a very cool idea, nonetheless.

[bobber205]

Here's what I hope comes out of this.

When you trash an item from your desktop, it "falls" with a cool animation through the back of your desktop and disappears!

Wouldn't that be cool?

[Andrew D.]

Here's what I hope comes out of this.

When you trash an item from your desktop, it "falls" with a cool animation through the back of your desktop and disappears!

Wouldn't that be cool?

Actually yeah, that would be pretty cool. Haha.