Rayon - A data parallelism library for Rust

  •        2353

Rayon is a data-parallelism library for Rust. It is extremely lightweight and makes it easy to convert a sequential computation into a parallel one. It also guarantees data-race freedom. Rayon makes it drop-dead simple to convert sequential iterators into parallel ones: usually, you just change your foo.iter() call into foo.par_iter(), and Rayon does the rest.

Parallel iterators take care of deciding how to divide your data into tasks; it will dynamically adapt for maximum performance. If you need more flexibility than that, Rayon also offers the join and scope functions, which let you create parallel tasks on your own. For even more control, you can create custom threadpools rather than using Rayon's default, global threadpool.

https://github.com/rayon-rs/rayon

Tags
Implementation
License
Platform

   




Related Projects

Vc - SIMD Vector Classes for C++

  •    C++

Recent generations of CPUs, and GPUs in particular, require data-parallel codes for full efficiency. Data parallelism requires that the same sequence of operations is applied to different input data. CPUs and GPUs can thus reduce the necessary hardware for instruction decoding and scheduling in favor of more arithmetic and logic units, which execute the same instructions synchronously. On CPU architectures this is implemented via SIMD registers and instructions. A single SIMD register can store N values and a single SIMD instruction can execute N operations on those values. On GPU architectures N threads run in perfect sync, fed by a single instruction decoder/scheduler. Each thread has local memory and a given index to calculate the offsets in memory for loads and stores. Current C++ compilers can do automatic transformation of scalar codes to SIMD instructions (auto-vectorization). However, the compiler must reconstruct an intrinsic property of the algorithm that was lost when the developer wrote a purely scalar implementation in C++. Consequently, C++ compilers cannot vectorize any given code to its most efficient data-parallel variant. Especially larger data-parallel loops, spanning over multiple functions or even translation units, will often not be transformed into efficient SIMD code.

FastFlow: programming multi-core

  •    C

Pattern-based multi/many-core parallel programming framework

Parallel Runtime Library

  •    

Parallel Runtime Library is optimized library that provide Easy to use and High Performance Parallelism Computing. Parallel Runtime Library provide: Effective Parallel Runtime, Concurrent Data Structure, Task and Data Parallel, Producer and Consumer and Agent Model.

bild - A collection of parallel image processing algorithms in pure Go

  •    Go

A collection of parallel image processing algorithms in pure Go.The aim of this project is simplicity in use and development over high performance, but most algorithms are designed to be efficient and make use of parallelism when available. It is based on standard Go packages to reduce dependency use and development abstractions.

Parallel Java Course

  •    Java

A course with labs to help you explore parallelism in Java : from low level to the cloud, including threads, JSR166, J2EE, hadoop and Google App Engine.


parallel - Ruby: parallel processing made simple and fast

  •    Ruby

Run any code in parallel Processes(> use all CPUs) or Threads(> speedup blocking operations). Best suited for map-reduce or e.g. parallel downloads/uploads. Produce one item at a time with lambda (anything that responds to .call) or Queue.

pigz - A parallel implementation of gzip for modern multi-processor, multi-core machines

  •    C

pigz, which stands for parallel implementation of gzip, it compresses using threads to make use of multiple processors and cores. The input is broken up in to 128 KB chunks with each compressed in parallel. The individual check value for each chunk is also calculated in parallel. The compressed data is written in order to the output, and a combined check value is calculated from the individual check values.

karate - Test Automation Made Simple

  •    Java

Karate is the only open-source tool to combine API test-automation, mocks, performance-testing and even UI automation into a single, unified framework. The BDD syntax popularized by Cucumber is language-neutral, and easy for even non-programmers. Assertions and HTML reports are built-in, and you can run tests in parallel for speed.

concurrencpp - Modern concurrency for C++

  •    C++

concurrencpp is a tasking library for C++ allowing developers to write highly concurrent applications easily and safely by using tasks, executors and coroutines. By using concurrencpp applications can break down big procedures that need to be processed asynchronously into smaller tasks that run concurrently and work in a co-operative manner to achieve the wanted result. concurrencpp also allows applications to write parallel algorithms easily by using parallel coroutines. concurrencpp is a task-centric library. A task is an asynchronous operation. Tasks offer a higher level of abstraction for concurrent code than traditional thread-centric approaches. Tasks can be chained together, meaning that tasks pass their asynchronous result from one to another, where the result of one task is used as if it were a parameter or an intermediate value of another ongoing task. Tasks allow applications to utilize available hardware resources better and scale much more than using raw threads, since tasks can be suspended, waiting for another task to produce a result, without blocking underlying OS-threads. Tasks bring much more productivity to developers by allowing them to focus more on business-logic and less on low-level concepts like thread management and inter-thread synchronization.

C++ AMP: Accelerated Massive Parallelism with Microsoft Visual C++

  •    C++

Samples for the latest Microsoft Press book on programming with C++AMP using Visual Studio 2012.

Gpars - Groovy parallel systems

  •    Groovy

Gpars is a framework which provides straightforward Java or Groovy-based APIs to declare, which parts of the code should be performed in parallel. Collections can have their elements processed concurrently, closures can be turned into composable asynchronous functions and run in the background on your behalf, mutable data can be protected by agents or software transactional memory.

Co-dfns - High-performance, Reliable, and Parallel APL

  •    APL

The Co-dfns project aims to provide a high-performance, high-reliability compiler for a parallel extension of the Dyalog dfns programming language. The dfns language is a functionally oriented, lexically scoped dialect of APL. The Co-dfns language extends the dfns language to include explicit task parallelism with implicit structures for synchronization and determinism. The language is designed to enable rigorous formal analysis of programs to aid in compiler optimization and programmer productivity, as well as in the general reliability of the code itself. Our mission is to deliver scalable APL programming to information and domain experts across many fields, expanding the scope and capabilities of what you can effectively accomplish with APL.

fbg - FBGraphics (FBG) is a simple C 24 bpp graphics library for the Linux framebuffer with parallelism support

  •    C

FBGraphics (FBG) is a simple C 24 bpp graphics library for the Linux framebuffer with parallelism support. The library is relatively generic, most functions (including parallel ones) only manipulate buffers, it should be easy to extract some parts and adapt it to other means, you may want to check fbg_setup fbg_draw fbg_flipand fbg_close for that.

node-threads-a-gogo - TAGG :: threads_a_gogo :: Simple and fast JavaScript threads for Node.js

  •    Javascript

Threads à gogo (*) is a native module for Node.js that provides an asynchronous, evented and/or continuation passing style API for moving blocking/longish CPU-bound tasks out of Node's event loop to JavaScript threads that run in parallel in the background and that use all the available CPU cores automatically; all from within a single Node process. You need a node with a v8 >= 3.2.4 to run this module. Any node >= 0.5.1 comes with a v8 >= 3.2.4.

worker-threads-pool - Easily manage a pool of Node.js Worker Threads

  •    Javascript

Easily manage a pool of Node.js Worker Threads. Number of active workers in the pool.

skylark - Skylark in Go: the Skylark configuration language, implemented in Go

  •    Go

This is the home of the Skylark in Go project. Skylark in Go is an interpreter for Skylark, implemented in Go.Skylark is a dialect of Python intended for use as a configuration language. Like Python, it is an untyped dynamic language with high-level data types, first-class functions with lexical scope, and garbage collection. Unlike CPython, independent Skylark threads execute in parallel, so Skylark workloads scale well on parallel machines. Skylark is a small and simple language with a familiar and highly readable syntax. You can use it as an expressive notation for structured data, defining functions to eliminate repetition, or you can use it to add scripting capabilities to an existing application.

parallel-webpack - Builds multi-config webpack projects in parallel

  •    Javascript

parallel-webpack allows you to run multiple webpack builds in parallel, spreading the work across your processors and thus helping to significantly speed up your build. For us at trivago it has reduced the build from 16 minutes to just 2 minutes - for 32 variants. That performance improvement naturally comes at the expense of utilizing all available CPU cores.You can choose whether to install parallel-webpack globally or locally. At trivago, we keep our build tools locally to the project so that we have full control over its versions.

parallel - Inspired by GNU Parallel, a command-line CPU load balancer written in Rust.

  •    Rust

This is an attempt at recreating the functionality of GNU Parallel, a work-stealer for the command-line, in Rust under a MIT license. The end goal will be to support much of the functionality of GNU Parallel and then to extend the functionality further for the next generation of command-line utilities written in Rust. While functionality is important, with the application being developed in Rust, the goal is to also be as fast and efficient as possible.See the to-do list for features and improvements that have yet to be done. If you want to contribute, pull requests are welcome. If you have an idea for improvement which isn't listed in the to-do list, feel free to email me and I will consider implementing that idea.

s3-parallel-put - Parallel uploads to Amazon AWS S3

  •    Python

s3-parallel-put speeds the uploading of many small keys to Amazon AWS S3 by executing multiple PUTs in parallel. The program reads your credentials from the environment variables AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY.

ArrayFire - Parallel Computing Library

  •    C++

ArrayFire is a high performance software library for parallel computing with an easy-to-use API. Its array based function set makes parallel programming simple. ArrayFire's multiple backends (CUDA, OpenCL and native CPU) make it platform independent and highly portable. A few lines of code in ArrayFire can replace dozens of lines of parallel computing code, saving you valuable time and lowering development costs.






We have large collection of open source products. Follow the tags from Tag Cloud >>


Open source products are scattered around the web. Please provide information about the open source projects you own / you use. Add Projects.