c - Why does mmap() fail with ENOMEM on a 1TB sparse file?

Question

I've been working with large sparse files on openSUSE 11.2 x86_64. When I try to mmap() a 1TB sparse file, it fails with ENOMEM. I would have thought that the 64 bit address space would be adequate to map in a terabyte, but it seems not. Experimenting further, a 1GB file works fine, but a 2GB file (and anything bigger) fails. I'm guessing there might be a setting somewhere to tweak, but an extensive search turns up nothing.

Here's some sample code that shows the problem - any clues?

#include <errno.h>
#include <fcntl.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/mman.h>
#include <sys/types.h>
#include <unistd.h>

int main(int argc, char *argv[]) {
    char * filename = argv[1];
    int fd;
    off_t size = 1UL << 40; // 30 == 1GB, 40 == 1TB

    fd = open(filename, O_RDWR | O_CREAT | O_TRUNC, 0666);
    ftruncate(fd, size);
    printf("Created %ld byte sparse file
", size);

    char * buffer = (char *)mmap(NULL, (size_t)size, PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0);
    if ( buffer == MAP_FAILED ) {
        perror("mmap");
        exit(1);
    }
    printf("Done mmap - returned 0x0%lx
", (unsigned long)buffer);

    strcpy( buffer, "cafebabe" );
    printf("Wrote to start
");

    strcpy( buffer + (size - 9), "deadbeef" );
    printf("Wrote to end
");

    if ( munmap(buffer, (size_t)size) < 0 ) {
        perror("munmap");
        exit(1);
    }
    close(fd);

    return 0;
}

Answer 1

The problem was that the per-process virtual memory limit was set to only 1.7GB. ulimit -v 1610612736 set it to 1.5TB and my mmap() call succeeded. Thanks, bmargulies, for the hint to try ulimit -a!