c++ - Is it possible to initialize std::vector over already allocated memory?

Question

My question is fairly simple and I am quite surprised I can't find anything related. Probably it is easy or totally stupid (or I can't search).

As the title says, is it possible to use std::vector on already allocated memory, so it does not allocate new elements from start but uses what is given. I would imagine it as something like:

T1 *buffer = new T1[some_size];
std::vector<T2> v(buffer, some_size); // <- ofc doesn't work

The opposite is quite simple and (maybe not pretty but) works:

std::vector<T2> v(some_size);
T1 *buffer = &v[0];

The storage is guaranteed to be continuous, so it is as safe as an iterator.

My motivation is quite simple. I pass around some raw memory data, i.e. bytes, and since I know their interpretation at some other places I would like to convert them back to something meaningful. I could do a reinterpret_cast and use normal c-style array, but I prefer c++ facilities.

I get the feeling this should be safe given that we give up ownership of buffer to vector, because it needs to be able to reallocate.

Answer 1

Like this.. Containers in the standard usually take an allocator. Using c++11's allocator traits, it is very easy to create an allocator as you don't have to have all the members in the allocator. However if using an older version of C++, you will need to implement each member and do the rebinding as well!

For Pre-C++11, you can use the following:

#include <iterator>
#include <vector>
#include <iostream>

template<typename T>
class PreAllocator
{
    private:
        T* memory_ptr;
        std::size_t memory_size;

    public:
        typedef std::size_t size_type;
        typedef ptrdiff_t difference_type;
        typedef T* pointer;
        typedef const T* const_pointer;
        typedef T& reference;
        typedef const T& const_reference;
        typedef T value_type;


        PreAllocator(T* memory_ptr, std::size_t memory_size) throw() : memory_ptr(memory_ptr), memory_size(memory_size) {};
        PreAllocator (const PreAllocator& other) throw() : memory_ptr(other.memory_ptr), memory_size(other.memory_size) {};

        template<typename U>
        PreAllocator (const PreAllocator<U>& other) throw() : memory_ptr(other.memory_ptr), memory_size(other.memory_size) {};

        template<typename U>
        PreAllocator& operator = (const PreAllocator<U>& other) {return *this;}
        PreAllocator<T>& operator = (const PreAllocator& other) {return *this;}
        ~PreAllocator() {}

        pointer address (reference value) const {return &value;}
        const_pointer address (const_reference value) const {return &value;}

        pointer allocate (size_type n, const void* hint = 0) {return memory_ptr;}
        void deallocate (T* ptr, size_type n) {}

        void construct (pointer ptr, const T& val) {new (ptr) T (val);}

        template<typename U>
        void destroy (U* ptr) {ptr->~U();}
        void destroy (pointer ptr) {ptr->~T();}

        size_type max_size() const {return memory_size;}

        template<typename U>
        struct rebind
        {
            typedef PreAllocator<U> other;
        };
};

int main()
{
    int my_arr[100] = {0};
    std::vector<int, PreAllocator<int> > my_vec(PreAllocator<int>(&my_arr[0], 100));
    my_vec.push_back(1024);
    std::cout<<"My_Vec[0]: "<<my_vec[0]<<"
";
    std::cout<<"My_Arr[0]: "<<my_arr[0]<<"
";

    int* my_heap_ptr = new int[100]();
    std::vector<int, PreAllocator<int> > my_heap_vec(PreAllocator<int>(&my_heap_ptr[0], 100));
    my_heap_vec.push_back(1024);
    std::cout<<"My_Heap_Vec[0]: "<<my_heap_vec[0]<<"
";
    std::cout<<"My_Heap_Ptr[0]: "<<my_heap_ptr[0]<<"
";

    delete[] my_heap_ptr;
    my_heap_ptr = NULL;
}

For C++11, you can use the following:

#include <cstdint>
#include <iterator>
#include <vector>
#include <iostream>

template <typename T>
class PreAllocator
{
    private:
        T* memory_ptr;
        std::size_t memory_size;

    public:
        typedef std::size_t     size_type;
        typedef T*              pointer;
        typedef T               value_type;

        PreAllocator(T* memory_ptr, std::size_t memory_size) : memory_ptr(memory_ptr), memory_size(memory_size) {}

        PreAllocator(const PreAllocator& other) throw() : memory_ptr(other.memory_ptr), memory_size(other.memory_size) {};

        template<typename U>
        PreAllocator(const PreAllocator<U>& other) throw() : memory_ptr(other.memory_ptr), memory_size(other.memory_size) {};

        template<typename U>
        PreAllocator& operator = (const PreAllocator<U>& other) { return *this; }
        PreAllocator<T>& operator = (const PreAllocator& other) { return *this; }
        ~PreAllocator() {}


        pointer allocate(size_type n, const void* hint = 0) {return memory_ptr;}
        void deallocate(T* ptr, size_type n) {}

        size_type max_size() const {return memory_size;}
};

int main()
{
    int my_arr[100] = {0};
    std::vector<int, PreAllocator<int>> my_vec(0, PreAllocator<int>(&my_arr[0], 100));
    my_vec.push_back(1024);
    std::cout<<"My_Vec[0]: "<<my_vec[0]<<"
";
    std::cout<<"My_Arr[0]: "<<my_arr[0]<<"
";

    int* my_heap_ptr = new int[100]();
    std::vector<int, PreAllocator<int>> my_heap_vec(0, PreAllocator<int>(&my_heap_ptr[0], 100));
    my_heap_vec.push_back(1024);
    std::cout<<"My_Heap_Vec[0]: "<<my_heap_vec[0]<<"
";
    std::cout<<"My_Heap_Ptr[0]: "<<my_heap_ptr[0]<<"
";

    delete[] my_heap_ptr;
    my_heap_ptr = nullptr;
}

Notice the difference between the two allocators! This will work with both heap buffers/arrays and stack buffer/arrays. It will also work with most containers. It is safer to use the Pre-C++11 version because it will be backwards compatible and work with more containers (ie: std::List).

You can just place the allocator in a header and use it as much as you want in any projects. It is good if you want to use SharedMemory or any buffer that is already allocated.

WARNING: DO NOT use the same buffer for multiple containers at the same time! A buffer can be reused but just make sure no two containers use it at the same time.

Example:

int my_arr[100] = {0};
std::vector<int, PreAllocator<int> > my_vec(PreAllocator<int>(&my_arr[0], 100));
std::vector<int, PreAllocator<int> > my_vec2(PreAllocator<int>(&my_arr[0], 100));

my_vec.push_back(1024);
my_vec2.push_back(2048);

std::cout<<"My_Vec[0]: "<<my_vec[0]<<"
";
std::cout<<"My_Arr[0]: "<<my_arr[0]<<"
";

The output of the above is 2048! Why? Because the last vector overwrote the values of the first vector since they share the same buffer.