aiomultiprocess - Take a modern Python codebase to the next level of performance.

  •        198

Take a modern Python codebase to the next level of performance. On their own, AsyncIO and multiprocessing are useful, but limited: AsyncIO still can't exceed the speed of GIL, and multiprocessing only works on one task at a time. But together, they can fully realize their true potential.

https://github.com/jreese/aiomultiprocess

Tags
Implementation
License
Platform

   




Related Projects

aioprocessing - A Python 3.4+ library that integrates the multiprocessing module with asyncio

  •    Python

The aioprocessing objects can be used just like their multiprocessing equivalents - as they are in func above - but they can also be seamlessly used inside of asyncio coroutines, without ever blocking the event loop. In most cases, this library makes blocking calls to multiprocessing methods asynchronous by executing the call in a ThreadPoolExecutor, using asyncio.run_in_executor(). It does not re-implement multiprocessing using asynchronous I/O. This means there is extra overhead added when you use aioprocessing objects instead of multiprocessing objects, because each one is generally introducing a ThreadPoolExecutor containing at least one threading.Thread. It also means that all the normal risks you get when you mix threads with fork apply here, too (See http://bugs.python.org/issue6721 for more info).

tortoise-orm - Familiar asyncio ORM for python, built with relations in mind

  •    Python

Tortoise ORM is an easy-to-use asyncio ORM (Object Relational Mapper) inspired by Django. Tortoise ORM was build with relations in mind and admiration for the excellent and popular Django ORM. It's engraved in it's design that you are working not with just tables, you work with relational data.

billiard - Multiprocessing Pool Extensions

  •    Python

billiard is a fork of the Python 2.7 multiprocessing package. The multiprocessing package itself is a renamed and updated version of R Oudkerk's pyprocessing package. This standalone variant draws its fixes/improvements from python-trunk and provides additional bug fixes and improvements. Please report bugs related to multiprocessing at the Python bug tracker. Issues related to billiard should be reported at http://github.com/celery/billiard/issues.

uvloop - Ultra fast asyncio event loop.

  •    Python

uvloop is a fast, drop-in replacement of the built-in asyncio event loop. uvloop is implemented in Cython and uses libuv under the hood. The project documentation can be found here. Please also check out the wiki.

asyncpg - A fast PostgreSQL Database Client Library for Python/asyncio.

  •    Python

asyncpg is a database interface library designed specifically for PostgreSQL and Python/asyncio. asyncpg is an efficient, clean implementation of PostgreSQL server binary protocol for use with Python's asyncio framework. You can read more about asyncpg in an introductory blog post. asyncpg requires Python 3.5 or later and is supported for PostgreSQL versions 9.2 to 10.


ruia - Async Python 3.6+ web scraping micro-framework based on asyncio.

  •    Python

Ruia is an async web scraping micro-framework, written with asyncio and aiohttp, aims to make crawling url as convenient as possible.

subconscious - redis-backed (in memory) db for python3 that is asyncio compatible

  •    Python

redis-backed (in memory) db for python3 that is asyncio compatible

gino - GINO Is Not ORM - a Python asyncio ORM on SQLAlchemy core.

  •    Python

GINO - GINO Is Not ORM - is a lightweight asynchronous ORM built on top of SQLAlchemy core for Python asyncio. Now (early 2018) GINO supports only one dialect asyncpg. There are a few tasks in GitHub issues marked as help wanted. Please feel free to take any of them and pull requests are greatly welcome.

aiohttp - Async http client/server framework (asyncio)

  •    Python

For this release we completely refactored low-level implementation of http handling. Finally uvloop gives performance improvement. Overall performance improvement should be around 70-90% compared to 1.x version.We took opportunity to refactor long standing api design problems across whole package. Client exceptions handling has been cleaned up and now much more straight forward. Client payload management simplified and allows to extend with any custom type. Client connection pool implementation has been redesigned as well, now there is no need for actively releasing response objects, aiohttp handles connection release automatically.

aiomysql - Library for accessing a MySQL database from the asyncio

  •    Python

aiomysql is a "driver" for accessing a MySQL database from the asyncio (PEP-3156/tulip) framework. It depends on and reuses most parts of PyMySQL . aiomysql tries to be like awesome aiopg library and preserve same api, look and feel.Internally aiomysql is copy of PyMySQL, underlying io calls switched to async, basically yield from and asyncio.coroutine added in proper places)). sqlalchemy support ported from aiopg.

Kyoukai - [OLD] A fully async web framework for Python3.5+ using asyncio

  •    Python

This project will not be getting any more updates. Kyōkai is a fast asynchronous Python server-side web framework. It is built upon asyncio and the Asphalt framework for an extremely fast web server.

trio - Trio – Pythonic async I/O for humans and snake people 🐍

  •    Python

The Trio project's goal is to produce a production-quality, permissively licensed, async/await-native I/O library for Python. Like all async libraries, its main purpose is to help you write programs that do multiple things at the same time with parallelized I/O. A web spider that wants to fetch lots of pages in parallel, a web server that needs to juggle lots of downloads and websocket connections at the same time, a process supervisor monitoring multiple subprocesses... that sort of thing. Compared to other libraries, Trio attempts to distinguish itself with an obsessive focus on usability and correctness. Concurrency is complicated; we try to make it easy to get things right. Trio was built from the ground up to take advantage of the latest Python features, and draws inspiration from many sources, in particular Dave Beazley's Curio. The resulting design is radically simpler than older competitors like asyncio and Twisted, yet just as capable. Trio is the Python I/O library I always wanted; I find it makes building I/O-oriented programs easier, less error-prone, and just plain more fun. Perhaps you'll find the same.

hbmqtt - MQTT client/broker using Python asynchronous I/O

  •    Python

HBMQTT is an open source MQTT client and broker implementation. Built on top of asyncio, Python's standard asynchronous I/O framework, HBMQTT provides a straightforward API based on coroutines, making it easy to write highly concurrent applications.

aredis - redis client for Python asyncio (has support for redis server, sentinel and cluster)

  •    Python

aredis requires a running Redis server.Please run test script in benchmarks dir to confirm the benchmark.

databases - Async database support for Python. 🗄

  •    Python

Databases gives you simple asyncio support for a range of databases. It allows you to make queries using the powerful SQLAlchemy Core expression language, and provides support for PostgreSQL, MySQL, and SQLite.

awesome-asyncio - A curated list of awesome Python asyncio frameworks, libraries, software and resources

  •    

A carefully curated list of awesome Python asyncio frameworks, libraries, software and resources. The Python asyncio module introduced to the standard library with Python 3.4 provides infrastructure for writing single-threaded concurrent code using coroutines, multiplexing I/O access over sockets and other resources, running network clients and servers, and other related primitives.

pytest-asyncio - Pytest support for asyncio.

  •    Python

pytest-asyncio is an Apache2 licensed library, written in Python, for testing asyncio code with pytest. asyncio code is usually written in the form of coroutines, which makes it slightly more difficult to test using normal testing tools. pytest-asyncio provides useful fixtures and markers to make testing easier.

asyncio-redis - Redis client for Python asyncio (PEP 3156)

  •    Python

Redis client for the PEP 3156 Python event loop.This Redis library is a completely asynchronous, non-blocking client for a Redis server. It depends on asyncio (PEP 3156) and therefor it requires Python 3.3 or greater. If you're new to asyncio, it can be helpful to check out the asyncio documentation first.