PDA

View Full Version : Testing: do you test on ALL iphone/itouch models in-house?




speedracer3000
Nov 16, 2009, 04:07 AM
I am new to iphone/itouch development and I am wondering if everyone keeps an inventory of all the hardware models (iphone 2G, 3G, 3GS, iTouch 1st Gen,2nd Gen etc.) in-house for testing purposes? Or is it reasonable to just test on whatever you have and solicit beta testers from the public? Ideally it makes sense to test on all the models, but since the iphone/itouch is a fairly closed system (unlike Android for example), there really isn't a big difference between say for example iphone 2G and 3G to require keeping both (or is there?)



Noisemaker
Nov 16, 2009, 10:46 AM
I am new to iphone/itouch development and I am wondering if everyone keeps an inventory of all the hardware models (iphone 2G, 3G, 3GS, iTouch 1st Gen,2nd Gen etc.) in-house for testing purposes? Or is it reasonable to just test on whatever you have and solicit beta testers from the public? Ideally it makes sense to test on all the models, but since the iphone/itouch is a fairly closed system (unlike Android for example), there really isn't a big difference between say for example iphone 2G and 3G to require keeping both (or is there?)

Obviously, it's better to test on every device that you can. However, with all the devices running the same OS, and being extremely similar to one another, the odds of an app not working across the board are very, very slim. Unless of course it's a more hardware-intensive game, in which case the first generation devices may run it poorly. A prime example of that is Dungeon Hunter on the 1st Gen iPhone and iPod Touch.

neoserver
Nov 16, 2009, 04:47 PM
I test with what I have: an iPhone 3G and a 2nd Gen iPod Touch. I probably don't even need to have the touch, but I think it's good to have a device from each family, but I think having one of each device ever made is going to get pretty expensive...

xsmasher
Nov 16, 2009, 05:32 PM
I've seen more differences between OS versions than I have between devices - more aggressive memory reclamation on 3.0, plus changes in audio handling.

I use a 1st gen iPod touch with 2.2.1 and two 3G iPhones with 3.0 and 3.x.latestVersion; plus I send a build to friends with 3Gs'es.

PhoneyDeveloper
Nov 16, 2009, 06:50 PM
It depends a little on the needs of your app. In my case it's the difference between OS versions that's important, more so than the hardware. I do the majority of my development on the Sim. I do move around between OS versions while developing, 3.0 one day 3.1 the next, etc. For testing I have one iPod touch 2 gen and I change the OS from 2.2.1 to whatever's latest.

There are of course major differences between 2.2.1 and 3.x but only minor differences between 3.0 and 3.1, mainly just bug fixes. I can't remove the workarounds for the bugs in many cases because I'm still supporting older versions.

I've never received a bug report from a user that was related to their hardware, although I have received some related to the OS version.

Niiro13
Nov 16, 2009, 08:00 PM
If you could only choose a single device, the iPhone 3G would probably be the best as I don't think the iPod touch offers anything that is of concern whilst the iPhone 3G has interruption by calls, SMS, etc. And it's not fast enough so that you miss the optimization.

firewood
Nov 17, 2009, 12:35 AM
I would try to at least test on the slowest device with the oldest OS revision that your app supports, as well as the fastest device with the newest OS.

I keep an old 1st gen iPod Touch running 2.2.1 for the first.

mpramodjain
Nov 19, 2009, 12:01 AM
I would try to at least test on the slowest device with the oldest OS revision that your app supports, as well as the fastest device with the newest OS.

I keep an old 1st gen iPod Touch running 2.2.1 for the first.

if we need to test the sdk app on the downgraded os(say for 2.2.1) , now , where can we find the OS. As the latest OS is available on the apple sites, I would like to know two quests...

1) Where can we find the downgraded OS , mean OS with lesser version than the latest one.
2) Any link on the web that show the process of the installing (either upgrading or downgrading) the OS version on the iPhone.

firewood
Nov 19, 2009, 12:21 AM
You can't always downgrade an iPhone (especially the baseband). If you want to up/downgrade for testing, an old early model iPod Touch is better.

PhoneyDeveloper
Nov 19, 2009, 09:06 AM
1) Where can we find the downgraded OS , mean OS with lesser version than the latest one.

http://www.google.com/search?hl=en&as_q=iphone+os+download+free+2.2.1&as_epq=&as_oq=&as_eq=&num=100&lr=&as_filetype=&ft=i&as_sitesearch=&as_qdr=all&as_rights=&as_occt=any&cr=&as_nlo=&as_nhi=&safe=off

2) Any link on the web that show the process of the installing (either upgrading or downgrading) the OS version on the iPhone.
Nov 17, 2009 01:35 AM


See above and also this

https://devforums.apple.com/message/95395#95395

lasiksurgery
Nov 19, 2009, 10:04 AM
I do the majority of my development on the Sim. I do move around between OS versions while developing, 3.0 one day 3.1 the next, etc.

firewood
Nov 19, 2009, 02:37 PM
Doing development on the Simulator is fine if you are not stressing any of the performance limits or require any device specific features.

Testing on the Simulator is not fine, since it won't catch any of the differences in behavior or performance between it and actual devices. The Simulator doesn't even run the same code, so even subtle compiler differences could potentially kill an app.

And the performance differences are obvious. An app can run super smooth on the Simulator, and be a big jerky unresponsive mess on an older device.

djsiek
Nov 24, 2009, 06:16 PM
I tested thoroughly on my iphone 3Gs and on my gf's iphone 3g. The 3G is a lot slower as my app uses 3g.

I didn't test on ipod touch but should have. cuase if something goes wrong when the apple tester is using it... it wastes two weeks minimum these days.

And it does say that Apple test on both iPhone and iPod... (depending on ur app of course).

Cheers

tutukun
Nov 24, 2009, 09:47 PM
right now im just testing it on my 3gs. i will probably get a 2nd gen ipod touch to test whenever my game goes into the main testing stage.

firewood
Nov 25, 2009, 01:23 AM
right now im just testing it on my 3gs. i will probably get a 2nd gen ipod touch to test whenever my game goes into the main testing stage.

Note that a 2nd gen Touch is about 25% faster than a 1st gen iPhone, iPhone 3G, and 1st gen Touch.