macOS http URL load problem

larswik · May 13, 2013

I found the answer to my problem but I don't know why it was a problem? I am wondering if someone can shed light on it so I understand it better.

This is the new functioning code to get the contents of a URL, I changed the URL for this sample.

Code:

 NSURL *urlToFile = [NSURL URLWithString:@"http://www.mySite.com/clients/clients_wine.txt"];
 remotelist = [NSString stringWithContentsOfURL:urlToFile encoding:NSUTF8StringEncoding error:&error];
 if (error) {
     NSLog(@"Error: %@", error);
}

It seems straight forward, but its not. To make sure I get the URL correct I have been copying the URL from Firefox and pasting it into my xcode project. But when I copy / paste the FireFox URL looks like this

mySite.com/clients/clients_wine.txt

When I paste it into Xcode it adds the http: before it and it looks like this

http://mySite.com/clients/clients_wine.txt

Now, it loads just fine, no problems. But when I open the txt file with a program like Transmit, edit the data and re save the file, it will not load the updated content? It always refers to the old data?

But if the "www" is in the string, ie "http://www." it reloads the data just fine? I don't understand why it will load it once ok, it finds the file just fine without www, but it won't reload the updated content, it just refers to the old file some how?

Last thing, I know it was refering to the old file because I added code to NSLog the modification date of the file and even though I made changes to the remote txt file it still displayed the same date.

Why?

robbieduncan · May 15, 2013

Most likely there is some sort of caching happening. If you want control over caching use NSURLConnection...

larswik · May 15, 2013

That is what I thought too.

When I found that problem that day I was able to reproduce that issue by adding and deleting the "www" which SEEMED to solve the problem. So I posted here to see why it was doing that and the next day the problems resumed again and it was not loading correctly.

I could copy the path to the remote link from my xcode project into Firefox and it would correctly display the information in the txt file in Fire Fox. But 10 seconds later try and run it with Xcode and have it fetch that link it would be the old file, again.

In the end I solved this problem by not using Objective C to get the contents of the txt file. I use Objective C to call a php script to get and return the data from the text file.

chown33 · May 15, 2013

larswik said:
That is what I thought too.

When I found that problem that day I was able to reproduce that issue by adding and deleting the "www" which SEEMED to solve the problem. So I posted here to see why it was doing that and the next day the problems resumed again and it was not loading correctly.

I could copy the path to the remote link from my xcode project into Firefox and it would correctly display the information in the txt file in Fire Fox. But 10 seconds later try and run it with Xcode and have it fetch that link it would be the old file, again.

In the end I solved this problem by not using Objective C to get the contents of the txt file. I use Objective C to call a php script to get and return the data from the text file.

You should probably read this article:
http://blackpixel.com/blog/2012/05/caching-and-nsurlconnection.html

Found by googling NSURLConnection caching.

The reason adding or deleting a "www" from a URL changed the behavior is because the cache contains URLs as keys. That is, a request that refers to the same URL will see the same resource (asset) already in the cache. A request with a different URL won't see the resource, even if both URLs ultimately reach the same server on the remote endpoint.

Firefox probably doesn't use the NSURLConnection cache, or it disables it (see above article link).

A possible solution is to make sure the HTTP headers in the response are correctly telling the client to NOT cache the content reply. You may need to look at the traffic on your network in order to see this.

Or use the 'curl' command with the -D option to dump headers when making the request. I'm almost certain that 'curl' doesn't use NSURLConnection, so the presence or absence of a cached resource in the xx cache would make no difference in what 'curl' does.

Example command-line:

Code:

curl -D /dev/stderr http://example.com/resource

Another possible solution is to make sure your HTTP server is supplying a response header for a short cache lifetime, assuming the resource is actually cachable. If the resource isn't cachable, then the server should say that.

You need to solve two problems:
1. Development. The best solution here is to have the HTTP server give a response header that disables all caching. I can't think of a reason for caching during development, so best to just turn it off.

2. Deployment. If some resources are actually cachable, you need a sane policy on the server for giving cache expiries. And you need to test your client to make sure it behaves sensibly with that policy. This is the one time when you should turn on client-side caching (to test the cache mgmt), but only after you have code for implementing good cache mgmt.

Ap0ks · May 15, 2013

larswik said:
Now, it loads just fine, no problems. But when I open the txt file with a program like Transmit, edit the data and re save the file, it will not load the updated content? It always refers to the old data?

But if the "www" is in the string, ie "http://www." it reloads the data just fine? I don't understand why it will load it once ok, it finds the file just fine without www, but it won't reload the updated content, it just refers to the old file some how?

Last thing, I know it was refering to the old file because I added code to NSLog the modification date of the file and even though I made changes to the remote txt file it still displayed the same date.

Why?

As the others have said it seems like the server is responding with a 304 code and so using the locally cached copy rather than the updated version. You could confirm whether this is the case by using wireshark or similar to check the HTTP headers returned.

larswik · May 15, 2013

Thanks for explaining it and posting the link, that is the problem I was having from reading the link. I can now see why it seemed to work when I changed my link by adding www as you pointed out.

I was under the impression when a method exited all local objects created in that method were destroyed. When the method was called again it would create a new objects that requested and return the remote data. Even when I quit out of xcode and re launched it I thought that would have cleared any cache retaining issues, but I did not.

When I Googled the problem I was searching for the wrong things and found nothing. But when I tied in "NSURLRequestCachePolicy" that you pointed out I found lots of info and other having this issue.

But now I know that it is an issue and will be aware of it for the future. I resolved this problem with a PHP script since I have been learning that too. The code below returns the data to program and seems to ignore any caching issues, but how frustrating the last few days have been.

Thanks Chown33!

Code:

<?PHP

$catagory = $_POST['catagory'];

fetchData($catagory);

function fetchData($name){
	$file;

	if ($name == 'food') {
		$file = file_get_contents('./thc/clients_dining.txt', true);
		echo $file;
	}
	if ($name == 'shop') {
		$file = file_get_contents('./thc/clients_shopping.txt', true);
		echo $file;
	}
	if ($name == 'wine') {
		$file = file_get_contents('./thc/clients_wine.txt', true);
		echo $file;
	}
	if ($name == 'acti') {
		$file = file_get_contents('./thc/clients_activities.txt', true);
		echo $file;
	}
}

?>

gnasher729 · May 16, 2013

larswik said:
I was under the impression when a method exited all local objects created in that method were destroyed. When the method was called again it would create a new objects that requested and return the remote data. Even when I quit out of xcode and re launched it I thought that would have cleared any cache retaining issues, but I did not.

URL cache is system wide. If Safari downloaded the site, it would be in the cache now, and your app would load the URL from the cache. That's why responses from the server contain an expiry date. Or not, for example a site returning current shareprices would probably return pages that expire immediately.

larswik · May 16, 2013

Thanks gnasher729. I started to read up on it a little more last night.

Search

Search

macOS http URL load problem

larswik

macrumors 68000

robbieduncan

Moderator emeritus

larswik

macrumors 68000

chown33

Moderator

Ap0ks

macrumors 6502

larswik

macrumors 68000

gnasher729

Suspended

larswik

macrumors 68000

Our Staff