objective c - NSURLCache: inconsistent behaviour

Question

I was observirng some strange behaviour of my app sometime caching responses and sometime not caching them (all the responses have Cache-Control: max-age=600).

The test is simple: I did a test.php script that was just setting the headers and returning a simple JSON:

<?php
        header('Content-Type: application/json');
header('Cache-Control: max-age=600');
?>
{
    "result": {
        "employeeId": "<?php echo $_GET['eId']; ?>",
                "dateTime": "<?php echo date('Y-m-d H:i:s'); ?>'" }
}

This is the response I get from the PHP page:

HTTP/1.1 200 OK
Date: Thu, 28 Nov 2013 11:41:55 GMT
Server: Apache
X-Powered-By: PHP/5.3.17
Cache-Control: max-age=600
Keep-Alive: timeout=5, max=100
Connection: Keep-Alive
Transfer-Encoding: chunked
Content-Type: application/json

{
    "result": {
        "employeeId": "",
        "dateTime": "2013-11-28 11:41:55'" 
    }
}

Then I've created a simple app and added AFNetworking library.

When I call the script with few parameters, the cache works properly:

AFHTTPRequestOperationManager *manager = [AFHTTPRequestOperationManager manager];

NSDictionary *params = @{ 
                         @"oId": @"4011",
                         @"eId": self.firstTest ? @"1" : @"0",
                         @"status": @"2031",
                         };
[manager GET:@"http://www.mydomain.co.uk/test.php" parameters:params success:^(AFHTTPRequestOperation *operation, id responseObject) {
    NSLog(@"JSON: %@", responseObject);

    NSLog(@"Cache current memory usage (after call): %d", [cache currentMemoryUsage]);
} failure:^(AFHTTPRequestOperation *operation, NSError *error) {
    NSLog(@"Error: %@", error);
}];

But when I increase the number of parameters, like:

NSDictionary *params = @{
                         @"organizationId": @"4011",
                         @"organizationId2": @"4012",
                         @"organizationId3": @"4013",
                         @"organizationId4": @"4014",
                         @"organizationId5": @"4015",
                         @"organizationId6": @"4016",
                         @"eId": self.firstTest ? @"1" : @"0",
                         @"status": @"2031",
                         };

it doesn't work anymore and it execute a new request each time it is called.

I've done many tests and it seems to me that it is related to the length of the URL, because if I includes this set of params:

NSDictionary *params = @{
                         @"oId": @"4011",
                         @"oId2": @"4012",
                         @"oId3": @"4013",
                         @"oId4": @"4014",
                         @"oId5": @"4015",
                         @"oId6": @"4016",
                         @"eId": self.firstTest ? @"1" : @"0",
                         @"status": @"2031",
                         };

It works!!

I've done many tests and that's the only pattern I've found...

To exclude AFNetworking from the equation, I've created another test program that uses NSURLConnection only and I can see the same behaviour so it's not AFNetworking and definitely NSURLCache. This is the other test:

NSURL *url = [NSURL URLWithString:[NSString stringWithFormat:@"http://www.mydomain.co.uk/test.php?eId=%@&organizationId=4011&organizationId2=4012&organizationId3=4013&organizationId4=4014&organizationId5=4015&organizationId6=4016", self.firstTest ? @"1" : @"0"]];  // doesn't work
//NSURL *url = [NSURL URLWithString:[NSString stringWithFormat:@"http://www.mydomain.co.uk/test.php?eId=%@&oId=4011&oId2=4012&oId3=4013&oId4=4014&oId5=4015&oId6=4016", self.firstTest ? @"1" : @"0"]];  // work
//NSURL *url = [NSURL URLWithString:[NSString stringWithFormat:@"http://www.mydomain.co.uk/test.php?eId=%@", self.firstTest ? @"1" : @"0"]];  // work

NSURLRequest *request = [NSURLRequest requestWithURL:url];
NSURLResponse *response = nil;
NSError *error = nil;
NSData *data = [NSURLConnection sendSynchronousRequest:request
                                     returningResponse:&response
                                                 error:&error];

if (error == nil) {
    // Parse data here
    NSString *responseDataStr = [NSString stringWithUTF8String:[data bytes]];
    NSLog(@"Response data: %@", responseDataStr);
}

I've also tried to establish how many characters in the URL will trigger the problem but even in this case I've got strange results:

This one is 112 characters long and it doesn't work:

http://www.mydomain.co.uk/test.php?eId=1&organizationId=4011&organizationId2=4012&organizationId3=4013&orgaId4=4

This one is 111 characters long and it works:

http://www.mydomain.co.uk/test.php?eId=1&organizationId=4011&organizationId2=4012&organizationId3=4013&orgId4=4

Ive renamed the PHP script to see if the first part of the URL would matter and I've got a strange behaviour again:

This one is 106 characters long and it doesn't work:

http://www.mydomain.co.uk/t.php?eId=1&organizationId=4011&organizationId2=4012&organizationId3=4013&org=40

This one is 105 characters long and it works:

http://www.mydomain.co.uk/t.php?eId=1&organizationId=4011&organizationId2=4012&organizationId3=4013&org=4

So I've removed 3 characters from the page name and I've got a working threshold 6 characters lower.

Any suggestion?

Thanks, Dem

Answer 1

I am witnessing something similar with certain responses not being cached by NSURLCache and I have come up with another possible reason:

In my case I have been able to ascertain that the responses not being cached are the ones that are returned using Chunked transfer-encoding. I've read elsewhere that NSURLCache should cache those after iOS 6 but for some reason it doesn't in my case (iOS 7.1 and 8.1).

I see that your example response shown here, also has the Transfer-Encoding: chunked header.

Could it be that some of your responses are returned with chunked encoding (those that are not cached) and some are not (those that are cached)?

My back-end is also running PHP on Apache and I still can't figure out why it does that... Probably some Apache extension...

Anyway, I think it sounds more plausible than the request URL length scenario.

---

EDIT:

It's been a while, but I can finally confirm that in our case, it is the chunked transfer encoding that causes the response not to be cached. I have tested that with iOS 7.1, 8.1, 8.3 and 8.4.

Since I understand that it is not always easy to change that setting on your server, I have a solution to suggest, for people who are using AFNetworking 2 and subclassing AFHTTPSessionManager.

You could add your sub-class as an observer for AFNetworking's AFNetworkingTaskDidCompleteNotification, which contains all the things you will need to cache the responses yourself. That means: the session data task, the response object and the response data before it has been processed by the response serializer.

If your server uses chunked encoding for only a few of its responses, you could add code in -(void)didCompleteTask: to only cache responses selectively. So for example you could check for the transfer-encoding response header, or cache the response based on other criteria.

The example HTTPSessionManager sub-class below caches all responses that return any data:

MyHTTPSessionManager.h

@interface MyHTTPSessionManager : AFHTTPSessionManager


@end

MyHTTPSessionManager.m

#import "MyHTTPSessionManager.h"

@implementation MyHTTPSessionManager

+ (instancetype)sharedClient {
    static MyHTTPClient *_sharedClient = nil;
    static dispatch_once_t onceToken;
    dispatch_once(&onceToken, ^{
        [[NSNotificationCenter defaultCenter] addObserver:_sharedClient selector:@selector(didCompleteTask:) name:AFNetworkingTaskDidCompleteNotification object:nil];
    });

    return _sharedClient;
}

- (void)didCompleteTask:(NSNotification *)notification {
    NSURLSessionDataTask *task = notification.object;
    NSHTTPURLResponse *response = (NSHTTPURLResponse *)task.response;

    NSData *responseData = notification.userInfo[AFNetworkingTaskDidCompleteResponseDataKey];
    if (!responseData.length) {
        // Do not cache empty responses.
        // You could place additional checks above to cache responses selectively.
        return;
    }

    NSCachedURLResponse *cachedResponse = [[NSCachedURLResponse alloc] initWithResponse:response data:responseData];
    [[NSURLCache sharedURLCache] storeCachedResponse:cachedResponse forRequest:task.currentRequest];
}

I tried to come up with some sort of cleaner solution, but it seems that AFNetworking does not provide a callback or a delegate method that returns everything we need early enough - that is, before it has been serialized by the response serializer.

Hope people will find this helpful :)