OpenCL is really a kind of "computational shader" which runs on the hardware of your choice. It abstracts many parts of the concrete hardware to be portable (computing units, local - global memory etc). The most important decision the implementer has to make is: data parallel or task parallel. Most GPUs are inefficient for task parallelism and CPUs don't like data parallel problems.
Good point about task vs data parallelism... CPU / GPU distinction becomes increasingly blurred, but this will indeed be the fundamental nature of each for the time being.
higher level as those which are already available?
Well, to be fair I still haven't really properly cut my teeth with OpenCL (or CUDA or whatever), but as a general rule things can always be profitably driven to a higher level IMO. The work of making computer languages more natural and expressive should never be given up. Ideally, people shouldn't need to be engineers in order to tell their computers what to do... they should be able to get in just as deep as they need to to express what they want to express efficiently, no deeper.
For now, I don't know if the core hardware abstractions in OpenCL itself need to be higher level, I just mean it will take time for a range of useful higher level libraries to be built... for example, OpenMM is a promising library for molecular mechanics simulation; if my physics engine was replaced with that I might have GPU acceleration which would work on CUDA or OpenCL platforms without needing to actually get my hands dirty with either. As well as libraries such as yours, we need more of those kinds of things - I think that's all I'm saying.
