The rapid development of Internet and Internet of Things opens the era of big data. Currently, heterogeneous architectures are being widely adopted in large-scale datacenters, for the sake of performance improvement and reduction of energy consumption. This paper presents the design and implementation of Hadoop+, a programming framework that implements MapReduce and enables invocation of parallelized CUDA/OpenCL within a map/reduce task, and helps the user by taking advantage of a heterogeneous task model. Experimental result shows that Hadoop+ attains 1.4× to 16.1× speedups over Hadoop for five commonly used machine learning algorithms. Coupled with a heterogeneous task model that helps allocate computing resouces, Hadoop+ brings a 36.0% improvement in data processing speed for single-application workloads, and for mixed workloads of multiple applications, the execution time is reduced by up to 36.9% with an average 17.6%.