There are many high-level libraries dedicated to GPGPU programming. Since they rely on CUDA and/or OpenCL, they have to be chosen wisely (a CUDA-based program will not run on AMD's GPUs, unless it goes through a pre-processing step with projects such as gpuocelot).
CUDA
You can find some examples of CUDA libraries on the NVIDIA website.
- Thrust: the official description speaks for itself
Thrust is a parallel algorithms library which resembles the C++ Standard
Template Library (STL). Thrust's high-level interface greatly enhances
programmer productivity while enabling performance portability between
GPUs and multicore CPUs. Interoperability with established
technologies (such as CUDA, TBB, and OpenMP) facilitates integration
with existing software.
As @Ashwin pointed out, the STL-like syntax of Thrust makes it a widely chosen library when developing CUDA programs. A quick look at the examples shows the kind of code you will be writing if you decide to use this library. NVIDIA's website presents the key features of this library. A video presentation (from GTC 2012) is also available.
- CUB: the official description tells us:
CUB provides state-of-the-art, reusable software components for every layer of the CUDA programming mode. It is a flexible library of cooperative threadblock primitives and other utilities for CUDA kernel programming.
It provides device-wide, block-wide and warp-wide parallel primitives such as parallel sort, prefix scan, reduction, histogram etc.
It is open-source and available on GitHub. It is not high-level from an implementation point of view (you develop in CUDA kernels), but provides high-level algorithms and routines.
- mshadow: lightweight CPU/GPU matrix/tensor template library in C++/CUDA.
This library is mostly used for machine learning, and relies on expression templates.
Starting from Eigen 3.3, it is now possible to use Eigen's objects and algorithms within CUDA kernels. However, only a subset of features are supported to make sure that no dynamic allocation is triggered within a CUDA kernel.
OpenCL
Note that OpenCL does more than GPGPU computing, since it supports heterogeneous platforms (multi-core CPUs, GPUs etc.).
- OpenACC: this project provides OpenMP-like support for GPGPU. A large part of the programming is done implicitly by the compiler and the run-time API. You can find a sample code on their website.
The OpenACC Application Program Interface describes a collection of
compiler directives to specify loops and regions of code in standard
C, C++ and Fortran to be offloaded from a host CPU to an attached
accelerator, providing portability across operating systems, host CPUs
and accelerators.
- Bolt: open-source library with STL-like interface.
Bolt is a C++ template library optimized for heterogeneous computing.
Bolt is designed to provide high-performance library implementations
for common algorithms such as scan, reduce, transform, and sort. The
Bolt interface was modeled on the C++ Standard Template Library (STL).
Developers familiar with the STL will recognize many of the Bolt APIs
and customization techniques.
Boost.Compute: as @Kyle Lutz said, Boost.Compute provides a STL-like interface for OpenCL. Note that this is not an official Boost library (yet).
SkelCL "is a library providing high-level abstractions for alleviated programming of modern parallel heterogeneous systems". This library relies on skeleton programming, and you can find more information in their research papers.
CUDA + OpenCL
- ArrayFire is an open-source (used to be proprietary) GPGPU programming library. They first targeted CUDA, but now support OpenCL as well. You can check the examples available online. NVIDIA's website provides a good summary of its key features.
Complementary information
Although this is not really in the scope of this question, there is also the same kind of support for other programming languages:
If you need to do linear algebra (for instance) or other specific operations, dedicated math libraries are also available for CUDA and OpenCL (e.g. ViennaCL, CUBLAS, MAGMA etc.).
Also note that using these libraries does not prevent you from doing some low-level operations if you need to do some very specific computation.
Finally, we can mention the future of the C++ standard library. There has been extensive work to add parallelism support. This is still a technical specification, and GPUs are not explicitely mentioned AFAIK (although NVIDIA's Jared Hoberock, developer of Thrust, is directly involved), but the will to make this a reality is definitely there.
