Become a MacRumors Supporter for $50/year with no ads, ability to filter front page stories, and private forums.

John Baughman

macrumors regular
Original poster
Oct 27, 2003
100
0
OK, I give up. After reading the Amazon Web Services docs and trying some code snippets I found on line, I am stumped as to how to get a signed request to work.

Does anyone have a working example of how to properly prepare a signed request for use in the chapter 28 Web Service lesson?

No matter what I do I get the following returned error...

"The request signature we calculated does not match the signature you provided."

I am using 2 methods I found on line...

Code:
- (NSData *)HMACSHA1withKey:(NSString *)key forString:(NSString *)string

- (NSString *)base64forData:(NSData *)data
;

I am calling them with...

Code:
NSString *signature = [self base64forData:[self HMACSHA1withKey:secretAccessKey forString:stringToSign]];

I think that either I am constructing the request incorrectly, or the calculation code is not calculating the signature correctly.

I notice that the calculated signature I am getting does not appear to be encoded as I have read it should be in that it does not percentage plus and equal signs. I tried doing this with stringByReplacingOccurrencesOfString, but it still did not work.

I really would like to get this to work so I can complete this chapter and do the challenge.

thanks,

John
 
Hi John,
You are still a head of me, I planned to look at it tonight but I am very... very tierd after all day at work and some work in the garden. I will probably start to read the chapter 28 tomorrow.

I just made a fast search. I do not know if you tried this page:

http://apisigning.com/service.html

It seems like there is a chance to go via Amazon API signing service.

The old way stopped to work 15 of August, bad luck.
Let me know if you find something.

/petron
 
Hi Petron,

Thanks for the link. I had run across that site but did not pay attention to it. After your post I took a closer look and after questioning why they are asking for my "secret key" after amazon saying I should not give to anyone, I went ahead and signed up for their free account.

Bottom line it worked great. Now I can finish the chapter. I would still like to learn how to calculate the signature myself. If you figure out how, let me know.

Thanks,

John
 
Hello John,
The access to the Amazon worked for me as well, but I stopped at the same problem you described in another thread.

The tableView is not called, but the numberOfRowsInTableView method has been called twice. I check if the datasource for the tableView is APPController. It seems to be correct but I probably need to check all the spelling once again. Started to debug it at the midnight so I gave up pretty fast. One thing is clear, I have to make a closer look at the code and relations... I even cleand the project and build it again.

I did not looked at the signing much but it seems to work when using command line (terminal) openssl with x509 signing utility and using digest SHA1 , I think I can incorporate it into the xcode via the same way as I did for the sqlite interfaces...But it has to wait for a while.
/petron
 
John,

I think you are on the right track. I was able to get my Cocoa application to sign its own requests and successfully retrieve results from Amazon. It was a bit tricky, and took some trial and error. A few pointers:

1. Sort the fields in the query string before signing them. That is, the field "AWSAccessKeyId=xxxxxx" must be first.

2. % escape all characters in the query string except = and &. I use the following:
Code:
(NSString*)CFURLCreateStringByAddingPercentEscapes(
               NULL,
               (CFStringRef)stringToEscape,
               NULL,
               (CFStringRef)@"!*'();:@+$,/?%#[]",
               kCFStringEncodingUTF8);

3. Construct your string to sign by using this format:
Code:
[NSString stringWithFormat:@"GET\necs.amazonaws.com\n/onca/xml\n%@",
          canonicalQuery]
where canonicalQuery is constructed as above in steps 1 and 2.

4. Be sure to use SHA256, not SHA1 when calculating the HMAC checksum.

5. % escape the checksum before adding it to your query string:
Code:
(NSString*)CFURLCreateStringByAddingPercentEscapes(
                          NULL,
                          (CFStringRef)signature,
                          NULL,
                          (CFStringRef)@"+/=",
                          kCFStringEncodingUTF8);

6. You must have a "Timestamp=..." in the query string. The time must be close to the current time, or Amazon says it is expired. It must also be in GMT, so I used:

Code:
NSDateFormatter *dateFormatter = [[NSDateFormatter alloc] init];
[dateFormatter setDateFormat:@"yyyy-MM-dd'T'HH:mm:ss'Z'"];
[dateFormatter setTimeZone:[NSTimeZone timeZoneWithAbbreviation:@"GMT"]];
NSString *date = [dateFormatter stringFromDate:[NSDate date]];

The rest should be as per the code in the book.

I created a helper class, much like Amazon's Java example, to do the signing. I actually compared the url to the result of their Java example until it matched, but I had to strip a trailing CRLF from their signature to make it work. Also, fix the timestamp to a constant string close to the current time for testing. This allows you to compare the signature exactly.

I used an array to hold my query fields to make them easier to sort:

Code:
NSArray *urlFields = [[NSArray alloc] initWithObjects:
                      @"Service=AWSECommerceService",
                      @"Operation=ItemSearch",
                      @"SearchIndex=Books",
                      [NSString stringWithFormat:@"Keywords=%@", searchString],
                      nil];

I sorted and joined them all at once with:

Code:
[[request sortedArrayUsingSelector:@selector(compare:)]
                    componentsJoinedByString:@"&"];

Finally, I had to modify sample code I found to calculate the HMAC. Here is what I ended up with:

Code:
#import <CommonCrypto/CommonHMAC.h>

- (NSData *)HMACforString:(NSString *)string
{
	NSData *clearTextData = [string dataUsingEncoding:NSUTF8StringEncoding];
	
	uint8_t digest[CC_SHA256_DIGEST_LENGTH] = {0};
	
	CCHmacContext hmacContext;
	CCHmacInit(&hmacContext, kCCHmacAlgSHA256, secretKeyData.bytes, secretKeyData.length);
	CCHmacUpdate(&hmacContext, clearTextData.bytes, clearTextData.length);
	CCHmacFinal(&hmacContext, digest);
	
	return [NSData dataWithBytes:digest length:CC_SHA256_DIGEST_LENGTH];
}

secretKeyData is an instance variable in my helper class which was created from my AWS secret key by:

Code:
[secretKey dataUsingEncoding:NSUTF8StringEncoding]
 
My helper class which sign my request

Hello, here 's my way to complete the "AmaZon" program of the chapter 28.

Like Andy, i 've decide to create an helper class, named "HMACSHA256", but i 've gave myself one condition:
- i 've decided that HMACSHA256 would work as a black box with an input and an output; i mean by that i 've decided to corrupt the less possible the code from the book (adding only a method call which calculate the signature in the guenine code), so then HMACSHA256 have a public method which take my non-signed urlString and return a signed and well-formed request.

So then here 's the modifications ( in bold) i did to the book code:

Code:
[I]- (IBAction)fetchBooks:(id)sender
{
        // the first part of the code, same as the book code.
        ...

	NSString *urlString = [NSString stringWithFormat:
						   @"http://ecs.amazonaws.com/onca/xml?"
						   @"Service=AWSECommerceService&"
						   @"AWSAccessKeyId=%@&"
						   @"Operation=ItemSearch&"
						   @"SearchIndex=Books&"
						   @"Keywords=%@&"
						   @"Version=2009-01-01",
						   AWS_ID, searchString];
	
[/I][B]	NSString *signedUrlString = [HMACSHA256 getSignedRequest:urlString];

	NSURL *url = [NSURL URLWithString:signedUrlString];
[/B][I]        // The rest of the code unmodified.
        ...

}
[/I]

Now here 's my HMACSHA256 helper class code:

HMACSHA256.h

Code:
[B]#import <Cocoa/Cocoa.h>


@interface HMACSHA256 : NSObject {
}

+ (NSData *)HMACforString:(NSString *)string;
+ (NSString *)base64forData:(NSData *)data;

+ (NSString *)appendTimestamp:(NSString *)undatedUrlString;
+ (NSString *)sortUrlString:(NSString *)unsortedUrlString;
+ (NSString *)transformServiceAddress:(NSString *)serviceAddress;
+ (NSString *)signUrlString:(NSString *)unsignedUrlString;

+ (NSString *)getSignedRequest:(NSString *)datedUnsortedUnsignedUrlString;
+ (NSString *)getSignedRequestTest:(NSString *)datedUnsortedUnsignedUrlString;

@end[/B]

and now the HMACSHA256.m code (most of it has been inspired from Andy's code)

Code:
#import "HMACSHA256.h"
#import <CommonCrypto/CommonHMAC.h>

#define SECRET_KEY @"[I]put your secret key here"[/I]

@implementation HMACSHA256

#pragma mark HMACSHA256 et base64Encodage methodes

// The base64Encodage method has been found online, like Petron.
// I use the HMACforString method of Andy.

+ (NSData *)HMACforString:(NSString *)string
{
	NSString *secretKey = SECRET_KEY;
	NSData *clearTextData = [string dataUsingEncoding:NSUTF8StringEncoding];
	NSData *secretKeyData = [secretKey dataUsingEncoding:NSUTF8StringEncoding];
	[secretKey release];
	
	uint8_t digest[CC_SHA256_DIGEST_LENGTH] = {0};
	
	CCHmacContext hmacContext;
	CCHmacInit(&hmacContext, kCCHmacAlgSHA256, secretKeyData.bytes, secretKeyData.length);
	CCHmacUpdate(&hmacContext, clearTextData.bytes, clearTextData.length);
	CCHmacFinal(&hmacContext, digest);
	
	NSData *result = [NSData dataWithBytes:digest length:CC_SHA256_DIGEST_LENGTH];
	
	return result;
}

+ (NSString *)base64forData:(NSData *)data
{
    static const char encodingTable[] = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/";
	
    if ([data length] == 0)
        return @"";
	
    char *characters = malloc((([data length] + 2) / 3) * 4);
    if (characters == NULL)
        return nil;
    NSUInteger length = 0;
	
    NSUInteger i = 0;
    while (i < [data length])
    {
        char buffer[3] = {0,0,0};
        short bufferLength = 0;
        while (bufferLength < 3 && i < [data length])
			buffer[bufferLength++] = ((char *)[data bytes])[i++];
		
        //  Encode the bytes in the buffer to four characters, including padding "=" characters if necessary.
        characters[length++] = encodingTable[(buffer[0] & 0xFC) >> 2];
        characters[length++] = encodingTable[((buffer[0] & 0x03) << 4) | ((buffer[1] & 0xF0) >> 4)];
        if (bufferLength > 1)
			characters[length++] = encodingTable[((buffer[1] & 0x0F) << 2) | ((buffer[2] & 0xC0) >> 6)];
        else characters[length++] = '=';
        if (bufferLength > 2)
			characters[length++] = encodingTable[buffer[2] & 0x3F];
        else characters[length++] = '=';        
    }
	
	NSString *encodedString = [[[NSString alloc] initWithBytesNoCopy:characters
															  length:length
															encoding:NSASCIIStringEncoding
														freeWhenDone:YES] autorelease];
	
    return encodedString;
}

#pragma mark Arrange the request methods

+ (NSString *)appendTimestamp:(NSString *)undatedUrlString
{
	NSDateFormatter *dateFormatter = [[NSDateFormatter alloc] init];
	[dateFormatter setDateFormat:@"yyyy-MM-dd'T'HH:mm:ss'Z'"];
	[dateFormatter setTimeZone:[NSTimeZone timeZoneWithAbbreviation:@"GMT"]];
	NSString *date = [dateFormatter stringFromDate:[NSDate date]];
	NSString *datedUrlString = [undatedUrlString stringByAppendingFormat:@"&Timestamp=%@",date];

	[dateFormatter release];
	dateFormatter = nil;

	return datedUrlString;
}

+ (NSString *)sortUrlString:(NSString *)unsortedUrlString
{
	// Split and conserve the service address from the request.
	NSArray *unsortedUrlStringArrayWithHTTPAddress = [unsortedUrlString componentsSeparatedByString:@"?"];
	
	NSString *serviceAddress =
		[NSString stringWithFormat:@"%@", [unsortedUrlStringArrayWithHTTPAddress objectAtIndex:0]];
	
	NSString *unsortedUrlStringWithoutHTTPAddress =
		[NSString stringWithFormat:@"%@", [unsortedUrlStringArrayWithHTTPAddress objectAtIndex:1]];

	// Free the unsortedUrlStringArrayWithHTTPAddress array.
	[unsortedUrlStringArrayWithHTTPAddress release];
	
	// % Escape the request string.
	NSString *escapedUnsortedUrlString = 
		(NSString *) CFURLCreateStringByAddingPercentEscapes(
														NULL, (CFStringRef)unsortedUrlStringWithoutHTTPAddress,
														NULL, (CFStringRef)@"!*'();:@+$,/?%#[]",
														kCFStringEncodingUTF8); 
	
	// Split all items from the requestin an array and sort the array insensitive to the case.
	NSArray *arrayUrlString = [escapedUnsortedUrlString componentsSeparatedByString:@"&"];
	arrayUrlString = [arrayUrlString sortedArrayUsingSelector:@selector(caseInsensitiveCompare:)];
	
	// Join all items.
	int i,itemCount = [arrayUrlString count];
	
	NSString *enumString;
	NSString *appendStringArray;
	
	// Find and place the AWSAccessKeyId=XXXX field first.
	for (enumString in arrayUrlString) {
		if ([enumString rangeOfString:@"AWSAccessKeyId="].location != NSNotFound) {
			appendStringArray = [NSString stringWithFormat:@"%@",enumString];
		}
	}
	
	// Append the other fields to the first field, voiding recopying it a second times.
	for (i = 0; i < itemCount;i++) {
		if([[arrayUrlString objectAtIndex:i] rangeOfString:@"AWSAccessKeyId="].location == NSNotFound) {
			appendStringArray = [appendStringArray stringByAppendingFormat:
								  @"&%@",[arrayUrlString objectAtIndex:i]];
		}
	}
	
	// Join the serviceAddress string to the request.
	appendStringArray = [NSString stringWithFormat:@"%@?%@", serviceAddress, appendStringArray];
	
	return appendStringArray;
}

+ (NSString *)transformServiceAddress:(NSString *)serviceAddress
{
	// Transform the serviceAddress string to fit for the signature calculation.
	NSString *tSA = [serviceAddress stringByReplacingOccurrencesOfString:@"http://" withString:@"GET\n"];
	tSA = [tSA stringByReplacingOccurrencesOfString:@"/onca/xml" withString:@"\n/onca/xml\n"];
	return tSA;
}

+ (NSString *)signUrlString:(NSString *)unsignedUrlString
{
	// Split and conserve the service address from the request.
	NSArray *splitRequest = [unsignedUrlString componentsSeparatedByString:@"?"];
	
	NSString *serviceAddress =
		[NSString stringWithFormat:@"%@", [splitRequest objectAtIndex:0]];
	
	NSString *canonicalRequestForm =
		[NSString stringWithFormat:@"%@", [splitRequest objectAtIndex:1]];
	
	// Free the array.
	// [splitRequest release];
	
	// Getting the canonical form of the service address.
	NSString *sACanonicalForm = [HMACSHA256 transformServiceAddress:serviceAddress];
	
	NSString *fullCanonicalFormRequest = [NSString 
									 stringWithFormat:@"%@%@", sACanonicalForm, canonicalRequestForm];
	
	// Calcul the signature from the canonical request and % escape it.
	NSString *signatureUncoded = [HMACSHA256 base64forData:[HMACSHA256 HMACforString:fullCanonicalFormRequest]];
	NSString *signature = (NSString *)CFURLCreateStringByAddingPercentEscapes(
																			  NULL,(CFStringRef) signatureUncoded,
																			  NULL, (CFStringRef)@"+/=",
																			  kCFStringEncodingUTF8);
	[signature autorelease];
	
	// Rejoin service address string, canonical request string and the signature field.
	NSString *signedUrlString = [NSString stringWithFormat:
								 @"%@?%@&Signature=%@", serviceAddress, canonicalRequestForm,signature];

	return signedUrlString;
}

+ (NSString *)getSignedRequest:(NSString *)undatedUnsortedUnsignedUrlString
{
	// Append "&Timestamp=XXXX" field.
	NSString *datedUnsortedUnsignedUrlString = [HMACSHA256 appendTimestamp:undatedUnsortedUnsignedUrlString];
	
	// Sort and % escape the request fields.
	NSString *datedSortedUnsignedUrlString = [HMACSHA256 sortUrlString:datedUnsortedUnsignedUrlString];
	
	// Calcul and append the "&Signature=XXXX" field to the rest of the request.
	NSString *datedSortedSignedUrlString = [HMACSHA256 signUrlString:datedSortedUnsignedUrlString];
	
	return datedSortedSignedUrlString;
}

@end

I 've tested this code and it works, but i assum that it 's far from being well optimized and i'm sure that there is lot of memory leaks.
If any one wish to correct my code, feel free, it 'll help me by the way to improve my knowledge of Cocoa.

Pommade.
 
this is mentioned on books product page (bignerdranch.com under Errata)

Errata
Chapter 28: As of Aug 15, 2009, Amazon requires that all requests be signed. Thus, this exercise doesn't work any more. The workaround is pretty extreme: You need to create your own AWS account to get a private key and then sign your request as explained on this page.
 
You are right Darkroom...
If you looked at the beginning of this thread we, mentioned that this has been changed in 15th of August. and there is a link where you can get your own account at AWS.

/petron
 
I paraphrase petron and i write that you 're in the truth Darkroom.
But, like John Baughman, our thread creator, i was frustraded when i tried to implement this exercise and was'nt able to make it work. It literally ate me the fact i 've paid 40€ a book where one whole chapter was no more practicable.

When i posted a version of an helper class code, my goal was and stil being to prevent avarage joe programmer like me to have headache trying to make this exercise work.

So, sure this workabout is extrem, but with all the brains here, we can provide a way to go through this chapter at it was originally writen for purpose, and not looking here or there bits of an answer (we hardly understand when we begin programmation on Cocoa).

Andy Jensen showed us the way, i only proposed a made solution which works on my mini-mac, but which unfortunately is not well optimized and not well memory-leak proof.

I hardly recommand you (again) to try this piece of code, and i invite you (if you can) to improve it; so that people who learn and practice Cocoa with the Aaron Hillgass book could focus on what really matters in chapter 28, and not be stopped by an unpleasant difficulty.

Sincerely, me and the other.
 
Hi

Thanks for posting this code I have been able to use it to complete the chapter.

One point which is not immediately clear,

You need to use two amazon access keys in the sample code
1. Access Key ID - Used in the request
2. Secret Access Key - Used to sign the request and is passed to the helper class provided by John Baug..
 
Bad code generation in latest compiler?

The following line of code fails to find the string for the access key even though my debugging code to display the strings shows the string exists:
Code:
		if ([enumString rangeOfString:@"AWSAccessKeyId="].location != NSNotFound) {
(The .location is -1, which is also the value of NSNotFound.)

As a result, the code posted on this thread doesn't work as advertised.
 
The following line of code fails to find the string for the access key even though my debugging code to display the strings shows the string exists:
Code:
		if ([enumString rangeOfString:@"AWSAccessKeyId="].location != NSNotFound) {
(The .location is -1, which is also the value of NSNotFound.)

As a result, the code posted on this thread doesn't work as advertised.

Post compilable runnable example code that demonstrates the problem.

Also post the exact input data you're using.

Use a fake access-key and secret-key. If you don't know the exact form, then copy and paste the example access-key and secret-key from Amazon's S3 documentation examples.

Assuming for the moment that other people have used the code successfully, it's up to you to provide the example that shows a problem. I'm not saying you're wrong, just that without seeing exactly what your code and data actually is, there's no way for anyone else to replicate the problem, much less figure out a solution.


EDIT:
The following example code works, demonstrating that the problem does not lie in the code for HMACSHA256.m.

If I had to guess, I'd guess that you either haven't included an "AWSAccessKeyId" parameter at all, or it has different letter-case, or it wasn't preceded by "&" to delimit it from a prior parameter.

mainTest.m
Code:
#import <Foundation/Foundation.h>

#import "HMACSHA256.h"

// The key values are fake.  
// Copied from AWS's documentation examples.
#define AWS_ACCESS_KEY  @"0PN5J17HBGZHT7JJ3X82"
#define AWS_SECRET	@"/Ml61L9VxlzloZ091/lkqVV5X1/YvaJtI9hW4Wr9"

int main ( int argc, const char * argv[]) 
{
    NSAutoreleasePool * pool = [[NSAutoreleasePool alloc] init];

	NSString * accessKeyID = AWS_ACCESS_KEY;
	NSString * sdbAction = @"explodensparken";

	// The urlString MUST be in the form:  serviceAddress?p1=v1&p2=v2&..etc..
	// In particular, there MUST be a single ? delimiter, and one or more & delimiters.
	NSString * urlString = [NSString stringWithFormat:
			@"http://example.com/servicepath/?"
			@"&Version=2009-04-15"
			@"&SignatureMethod=HmacSHA256&SignatureVersion=2" 
			@"&AWSAccessKeyId=%@&Action=%@",   // intentionally NOT sorted
			accessKeyID, sdbAction ];

	NSString *signedURLString = [HMACSHA256 getSignedRequest:urlString withSecret:AWS_SECRET];

	NSLog(@"urlString = %@", urlString);
	NSLog(@"signed = %@", signedURLString);
	
    [pool drain];
    return 0;
}

Compiled as:
Code:
gcc -framework Foundation -std=c99 mainTest.m  HMACSHA256.m

Run as:
Code:
./a.out
Output:
Code:
2010-09-25 15:40:36.677 a.out[7022:903]  sortUrlString: http://example.com/servicepath/?&Version=2009-04-15&SignatureMethod=HmacSHA256&SignatureVersion=2&AWSAccessKeyId=0PN5J17HBGZHT7JJ3X82&Action=spitzensparken&Timestamp=2010-09-25T22:40:36Z
2010-09-25 15:40:36.679 a.out[7022:903] urlString = http://example.com/servicepath/?&Version=2009-04-15&SignatureMethod=HmacSHA256&SignatureVersion=2&AWSAccessKeyId=0PN5J17HBGZHT7JJ3X82&Action=spitzensparken
2010-09-25 15:40:36.679 a.out[7022:903] signed = http://example.com/servicepath/?AWSAccessKeyId=0PN5J17HBGZHT7JJ3X82&&Action=spitzensparken&SignatureMethod=HmacSHA256&SignatureVersion=2&Timestamp=2010-09-25T22%3A40%3A36Z&Version=2009-04-15&Signature=jyXM7GkudtjVUdIiFT1T1grtWrdl6U7c%2FXtPMTVm9hw%3D
 
Thanks, chown, for taking the time to look at the problem I was having. The cause of the problem was that I had spelled AWSAccessKeyId with a capital D. It's amazing how one can stare at the problem so long that one becomes blind to the obvious. :)
 
Register on MacRumors! This sidebar will go away, and you'll see fewer ads.