dispy favicon

dispy

dispy is a Python framework for parallel execution of computations by distributing them across multiple processors on a single machine (SMP), among many machines in a cluster or grid. dispy is well suited for data parallell (SIMD) paradigm where a computation is evaluated with different (large) datasets independently (similar to Hadoop, MapReduce). dispy features include automatic distribution of computations and their dependencies (files, Python functions, classes, modules), security, scheduling computations to specific nodes if necessary, client-side and server-side fault recovery, sharing of compute nodes if desired, etc.