The Kepler architecture, building block of the US fastest supercomputer, was originally a GPU (graphics), but was adapted to general scientific problems via a process GPGPU - general programming on graphics processing units.
Nvidia makes the IC with 192 cores on board - 3 of them can be easily strung together for 576 cores. That is currently $576 retail, coincidentally. (or not)
The key to making this work is the c compiler which automagically does the parallelism. This then has been used to build a Python interface as well. These language tools were developed as part of the supercomputing effort.
These will run neural networks very well, as the nets require dot products, which the cores feast on. Graphics pixel processing, machine learning, and data exploration are being pursued energetically.
It turns out the key to making this technology work is to train programmers and scientists how to use it. The course ay udacity, given by experts, fills that need. This is free, and a Stanford Prof. plus Nvidia developer give the course.
I believe there are 60,000 people taking it at any one time.
SO that's it. My interest is to see what I/O capabilities there are. Nvidia claims high speed camera interfaces.