Sidebar Sidebar
Become a MacRumors Supporter for $50/year with no ads, ability to filter front page stories, and private forums.

Future iPhones Could Capture 'Super-Resolution' Photos Using Optical Image Stabilization

MacRumors

MacRumors

macrumors bot
Original poster
Apr 12, 2001
63,509
30,784



Today, the US Patent & Trademark Office published Apple patent application number 20140125825 that describes a method of producing super-resolution images using optical image stabilization (OIS) and burst-mode photos (via AppleInsider). This may allow future models of the iPhone to generate images that exceed the megapixel resolution limits of the camera hardware.

ois-patent-super-resolution.jpg
The invention uses a camera with an optical image stabilization module that can take multiple images with each subsequent capture offset by a small amount. These lower resolution images are then intelligently stitched together by a dedicated image processor to create a resulting high-resolution photograph.
A system and method for creating a super-resolution image using an image capturing device. In one embodiment, an electronic image sensor captures a reference optical sample through an optical path. Thereafter, an optical image stabilization (OIS) processor to adjusts the optical path to the electronic image sensor by a known amount. A second optical sample is then captured along the adjusted optical path, such that the second optical sample is offset from the first optical sample by no more than a sub-pixel offset. The OIS processor may reiterate this process to capture a plurality of optical samples at a plurality of offsets. The optical samples may be combined to create a super-resolution image.
Click to expand...
Image stabilization allows for sharper photos and videos by utilizing motion tracking and other technology to remove the effects of device shakiness during capture. Apple currently offers software-based image stabilization in which the device takes four photos in quick succession and the combines them to reduce blurring., but the company has been rumored to be looking to improve this system. While OIS would embed the technology directly into the camera hardware, improved electronic image stabilization would allow for the iPhone to have a slimmer camera that does not protrude outside the device.

Rumors from earlier this year suggested Apple could incorporate Optical Image Stabilization (OIS) technology into its upcoming iPhone 6, possibly contracting with InvenSense to supply the necessary hardware. The company's work on the technology has also surfaced in other patent filings, leveraging the expertise of Richard Topliss an engineer recruited from Cambridge Mechatronics in early 2012. The most recent rumor from ESM China analyst Sun Chang Xu claims, however, that Apple may equip the iPhone 6 camera with an improved sensor module and an electronic image stabilization system.

Article Link: Future iPhones Could Capture 'Super-Resolution' Photos Using Optical Image Stabilization
 
Article Link
https://www.macrumors.com/2014/05/08/super-resolution-ois/
constantsnags!

constantsnags!

macrumors regular
Apr 16, 2008
163
161
Aotearoa
This is really interesting - basically what a lot of very high end medium format backs do. You have a 20MP sensor, but a 60MP image built with 3 separately processed scans of the sensor output. If it can be done on the hoof, it should be remarkably good.
 
gnasher729

gnasher729

Suspended
Nov 25, 2005
17,980
5,565
santydolby said:
"My Olympus had this way back in..."

3... 2... 1...
Click to expand...

Unlikely. Your Olympus has most likely superior optical components. The iPhone most definitely has a vastly superior processor, both CPU and GPU. Anything using data from multiple shots will need tons of processing power.
 
Rocketman

Rocketman

macrumors 603
Dec 2, 2001
6,191
1,186
Claremont, The intellectual capitol of the world.
There was a fractal demonstration app on the Mac IIci I used at the university which had images you could zoom into by over 500x and maintain visually full resolution. That was not a bitmap storage method, but a version of vector called fractal.

I am wondering if the multi-shot pixelized images could be translated to a multi-resolution fractal file rather than, or in addition to, a single better bitmapped file. It sounds like the proposed computation platform is in the handtop not in the cloud. That's impressive.

Rocketman
 
2

2457282

Suspended
Dec 6, 2012
3,327
3,015
The quality improvements in the 5s from both the bigger pixels and the software to combine 4 photos has been a real improvement. This seems to take that even further from a software perspective while adding OIS. I had thought that Apple could only do minor incremental improvements without significantly increasing the size of the sensor and therefore the thickness of the phone. Well, this approach is going to make me eat my words. That is a nice technological leap to solve the size issue.
 
VulchR

VulchR

macrumors 68040
Jun 8, 2009
3,380
14,252
Scotland
Four oF NINE said:
Sorry, but you can only get so good without a big lens...
Click to expand...

Perhaps, but I think the point is for the software to do more of the information processing of light that optical lenses traditionally do (e.g., Lytro - which is incidentally one of the companies that I am surprised that Apple has not snatched up). The software approach has the advantage that it has the potential to simplify lens design, potentially reducing costs and potential optical aberrations.

... I don't suppose sapphire has any special optical qualities like fluorite ....
 
F

foobarbaz

macrumors 6502a
Nov 29, 2007
873
1,953
Rocketman said:
There was a fractal demonstration app on the Mac IIci I used at the university which had images you could zoom into by over 500x and maintain visually full resolution. That was not a bitmap storage method, but a version of vector called fractal.
Click to expand...

That was not a storage method at all. The picture you saw was generated, so when zooming, it could just be re-generated at a different resolution.

It does not translate to storing anything from the physical world.
 
P

prowlmedia

Suspended
Jan 26, 2010
1,589
813
London
Rocketman said:
There was a fractal demonstration app on the Mac IIci I used at the university which had images you could zoom into by over 500x and maintain visually full resolution. That was not a bitmap storage method, but a version of vector called fractal.

I am wondering if the multi-shot pixelized images could be translated to a multi-resolution fractal file rather than, or in addition to, a single better bitmapped file. It sounds like the proposed computation platform is in the handtop not in the cloud. That's impressive.

Rocketman
Click to expand...

Not sure how a fractal relates to this. It's just a mathematical equation relating to repeating trees. That's why the zoom works. It's the same equation repeated infinitely. It's not an image.

However a image is captured there is a finite amount of zoom. This is down to the pixel density on the CCD. The more pixels the more you can zoom - so long as there is enough light per pixel. The problem with this is to maintain light you need bigger lenses.
 
Orlandoech

Orlandoech

macrumors 68040
Jun 2, 2011
3,341
887
As they do this, the standard storage capacity will go from 16GB to 8GB to drive margins back up.


Jk... I hope...
 
macduke

macduke

macrumors G5
Jun 27, 2007
13,136
19,665
If brought to market, this is true innovation. With the slowing of the megapixel race, I've been wondering what the next breakthrough would be. I thought that maybe foveon type sensors would be the answer. Canon is rumored to have a foveon like sensor coming out later this year or early next year. But that technology combined with this would be even more incredible. For those who don't know, foveon basically stacks the typical RGB filters in the Bayer array so that each pixel occupies a smaller footprint, therefore increasing density on the sensor. The light is measured by each color's penetration into the sensor, rather than averaging three RGB receptors.

I could see Apple implementing this technology by having a toggle, much like HDR, to enable the higher resolution capture mode so that it doesn't eat through your storage space. This way, for instance, when taking a shot of a landscape while on vacation you can crank up the resolution. Neat. This is the upside to having super fast image processors. You can capture multiple frames so quickly that they can be stitched together without blur from motion.
 
Chupa Chupa

Chupa Chupa

macrumors G5
Jul 16, 2002
14,835
7,396
So the rumor mill is back to OIS this week. Good to know. But what I really need to know is what does Ming Chi Kuo think... and Digitimes :D /sarc
 
Marrakas

Marrakas

macrumors 6502
May 23, 2012
420
122
Isn't this what the new Oppo Find already does? creating 50mp photos with a 13mp camera.
 
Chupa Chupa

Chupa Chupa

macrumors G5
Jul 16, 2002
14,835
7,396
Four oF NINE said:
Sorry, but you can only get so good without a big lens.

That's why I'll keep my Nikon D3.
Click to expand...

People use to say that with car engines but some of todays 4cyl are faster than yesteryears 6cyl.

Truth: D3 will crush phone cameras for a while but who carries a D3 or any (d)SLR with them 24/7? The best camera one has is the camera they take with them. A camera phone isn't meant to replace a (d)SLR so silly comment. No need to be a camera snob.
 
AppleScruff1

AppleScruff1

macrumors G4
Feb 10, 2011
10,026
2,949
Chupa Chupa said:
People use to say that with car engines but some of todays 4cyl are faster than yesteryears 6cyl.

Truth: D3 will crush phone cameras for a while but who carries a D3 or any (d)SLR with them 24/7? The best camera one has is the camera they take with them. A camera phone isn't meant to replace a (d)SLR so silly comment. No need to be a camera snob.
Click to expand...

But today's performance 6 cylinders are faster than today's 4 cylinders. My guess is DSLR technology will continue to advance too.
 
aliensporebomb

aliensporebomb

macrumors 68000
Jun 19, 2005
1,907
332
Minneapolis, MN, USA, Urth
Ah

The convenience and portability of smartphones produces credible images on the fly but I still prefer my Nikon D3200 with multiple lenses if I'm going to be producing super high quality "super resolution" images. Yes, 24 megapixels is probably overkill.

Honestly, what makes a great image is great glass (lens) combined with a photographer with some concept of his things work to get great images.
 
D

deconstruct60

macrumors G5
Mar 10, 2009
12,268
3,867
gnasher729 said:
Unlikely. Your Olympus has most likely superior optical components.
Click to expand...

Which make diddly squat difference in a patent. Everything thing here is stuff that some cameras have had for a long while prior 2012 ( when this was filed). The components in the vast majority of cameras are better than they were 3-4 years ago. That isn't particularly evident of a singular patentable process either.

it is nice that Apple is adding a better camera infrastructure to the iPhone over time. That notion that they are making huge, unique leaps in the camera technology and knowledge is a bit questionable.



The iPhone most definitely has a vastly superior processor, both CPU and GPU. Anything using data from multiple shots will need tons of processing power.
Click to expand...

Don't need tons of generalized processing power. Indeed, that is exactly why there is a OIS stablization processor in the diagram. A fixed function circuit can effectively do the processing ( exactly why cameras 5-6 years ago where doing this with the current mobile CPUs and GPUs no where in sight.)
 
You must log in or register to reply here.
Register on MacRumors! This sidebar will go away, and you'll see fewer ads.
Top Bottom