Friday, October 5, 2012

MapReduce Implementations

Hi everyone, here are all MapReduce implementations I found online, all of which are either an independent Mapreduce Framework or coupled to a database or file system, a single programming language or a single domain, or made as a library or implement only a subset of mature MapReduce features. Hope this list might be helpful. 


Hadoop


Hadoop mapreduce
The most commonly used open source Mapreduce implementation by Apache
Other Mapreduce Implementations


Disco
Mapreduce implementation in Erlang developed by Nokia

Cloud MapReduce
Mincemeat
Python implementation of the MapReduce distributed computing framework
Mapredus
simple mapreduce framework using redis and resque
Qizmt
Peregrine
Fast Mapreduce framework for running iterative jobs across partitions of data. 
HTTPMR
R3
Mapreduce engine written in python using a redis backend.
Octopy
Fast-n-easy MapReduce implementation for Python
Implementation of MapReduce in the bash shell
Starfish
Open source ruby implimentatoin of mapreduce
Skynet
open source Ruby implementation mapreduce created at Geni

FileMap
system for applying Unix-style file processing tool and provides full map-reduce functionality

Meguro
simple javascript Map/Reduce implemention
Mapreduce-lite
lightweight C++ implementation of the MapReduce
Misco
Mobile Mapreduce framework

Galago's TupleFlow
search engine toolkit which distribute execution across many processors 
GPMR
Multi-core GPU mapreduce on GPU clusters
Hadoop Pipes
Hadoop C++ API for mapreduce
Haloop
Mapreduce for storage and file system


Sphere
distributed data storage use Mapreduce

MongoDB
Map/reduce database for batch processing of data and aggregation operations
Greenplum
Next-gen data warehousing and large-scale analytics processing database Supporting SQL and MapReduce parallel processing
http://www.greenplum.com/        

Plasma MapReduce
Distributed filesystem, key/value db, and map/reduce system
Riak
Riak
open-source, distributed database using mapreduce for Searching and Aggregating Data
Grid Grain
map reduce open source data grid

InfiniDB 
open-source free analytical (columnar) DB  implements MPP in a MapReduce-like fashion
CouchDB 
Database use Json for ducuments and javascirpt for mapreduce queries
Mapreduce librarires


Boost
Boost C++ library for mapreduce
MapReduce library for  Haskell

MPI-MR
Mapreduce-MPI library







1 comment: