aioprocessing - A Python 3.4+ library that integrates the multiprocessing module with asyncio

  •        2015

The aioprocessing objects can be used just like their multiprocessing equivalents - as they are in func above - but they can also be seamlessly used inside of asyncio coroutines, without ever blocking the event loop. In most cases, this library makes blocking calls to multiprocessing methods asynchronous by executing the call in a ThreadPoolExecutor, using asyncio.run_in_executor(). It does not re-implement multiprocessing using asynchronous I/O. This means there is extra overhead added when you use aioprocessing objects instead of multiprocessing objects, because each one is generally introducing a ThreadPoolExecutor containing at least one threading.Thread. It also means that all the normal risks you get when you mix threads with fork apply here, too (See http://bugs.python.org/issue6721 for more info).

https://github.com/dano/aioprocessing

Tags
Implementation
License
Platform

   




Related Projects

billiard - Multiprocessing Pool Extensions

  •    Python

billiard is a fork of the Python 2.7 multiprocessing package. The multiprocessing package itself is a renamed and updated version of R Oudkerk's pyprocessing package. This standalone variant draws its fixes/improvements from python-trunk and provides additional bug fixes and improvements. Please report bugs related to multiprocessing at the Python bug tracker. Issues related to billiard should be reported at http://github.com/celery/billiard/issues.

awesome-asyncio - A curated list of awesome Python asyncio frameworks, libraries, software and resources

  •    

A carefully curated list of awesome Python asyncio frameworks, libraries, software and resources. The Python asyncio module introduced to the standard library with Python 3.4 provides infrastructure for writing single-threaded concurrent code using coroutines, multiplexing I/O access over sockets and other resources, running network clients and servers, and other related primitives.

pytest-asyncio - Pytest support for asyncio.

  •    Python

pytest-asyncio is an Apache2 licensed library, written in Python, for testing asyncio code with pytest. asyncio code is usually written in the form of coroutines, which makes it slightly more difficult to test using normal testing tools. pytest-asyncio provides useful fixtures and markers to make testing easier.

asyncio-redis - Redis client for Python asyncio (PEP 3156)

  •    Python

Redis client for the PEP 3156 Python event loop.This Redis library is a completely asynchronous, non-blocking client for a Redis server. It depends on asyncio (PEP 3156) and therefor it requires Python 3.3 or greater. If you're new to asyncio, it can be helpful to check out the asyncio documentation first.

asyncpg - A fast PostgreSQL Database Client Library for Python/asyncio.

  •    Python

asyncpg is a database interface library designed specifically for PostgreSQL and Python/asyncio. asyncpg is an efficient, clean implementation of PostgreSQL server binary protocol for use with Python's asyncio framework. You can read more about asyncpg in an introductory blog post. asyncpg requires Python 3.5 or later and is supported for PostgreSQL versions 9.2 to 10.


ruia - Async Python 3.6+ web scraping micro-framework based on asyncio.

  •    Python

Ruia is an async web scraping micro-framework, written with asyncio and aiohttp, aims to make crawling url as convenient as possible.

aiofiles - File support for asyncio

  •    Python

aiofiles is an Apache2 licensed library, written in Python, for handling local disk files in asyncio applications. Ordinary local file IO is blocking, and cannot easily and portably made asynchronous. This means doing file IO may interfere with asyncio applications, which shouldn't block the executing thread. aiofiles helps with this by introducing asynchronous versions of files that support delegating operations to a separate thread pool.

aioredis-py - asyncio (PEP 3156) Redis support

  •    Python

asyncio (3156) Redis client library. The library is intended to provide simple and clear interface to Redis based on asyncio.

aioredis - asyncio (PEP 3156) Redis support

  •    Python

asyncio (PEP 3156) Redis client library.hiredis is preferred requirement. Pure-python protocol parser is implemented as well and can be used through parser parameter.

aiomysql - Library for accessing a MySQL database from the asyncio

  •    Python

aiomysql is a "driver" for accessing a MySQL database from the asyncio (PEP-3156/tulip) framework. It depends on and reuses most parts of PyMySQL . aiomysql tries to be like awesome aiopg library and preserve same api, look and feel.Internally aiomysql is copy of PyMySQL, underlying io calls switched to async, basically yield from and asyncio.coroutine added in proper places)). sqlalchemy support ported from aiopg.

aiopg - Library for accessing a PostgreSQL database from the asyncio

  •    Python

aiopg is a library for accessing a PostgreSQL database from the asyncio (PEP-3156/tulip) framework. It wraps asynchronous features of the Psycopg database driver.

uvloop - Ultra fast asyncio event loop.

  •    Python

uvloop is a fast, drop-in replacement of the built-in asyncio event loop. uvloop is implemented in Cython and uses libuv under the hood. The project documentation can be found here. Please also check out the wiki.

gino - GINO Is Not ORM - a Python asyncio ORM on SQLAlchemy core.

  •    Python

GINO - GINO Is Not ORM - is a lightweight asynchronous ORM built on top of SQLAlchemy core for Python asyncio. Now (early 2018) GINO supports only one dialect asyncpg. There are a few tasks in GitHub issues marked as help wanted. Please feel free to take any of them and pull requests are greatly welcome.

Kyoukai - [OLD] A fully async web framework for Python3.5+ using asyncio

  •    Python

This project will not be getting any more updates. Kyōkai is a fast asynchronous Python server-side web framework. It is built upon asyncio and the Asphalt framework for an extremely fast web server.

tortoise-orm - Familiar asyncio ORM for python, built with relations in mind

  •    Python

Tortoise ORM is an easy-to-use asyncio ORM (Object Relational Mapper) inspired by Django. Tortoise ORM was build with relations in mind and admiration for the excellent and popular Django ORM. It's engraved in it's design that you are working not with just tables, you work with relational data.

gain - Web crawling framework based on asyncio.

  •    Python

Web crawling framework for everyone. Written with asyncio, uvloop and aiohttp. You can add proxy setting to spider as above.

autobahn-python - WebSocket and WAMP in Python for Twisted and asyncio

  •    Python

WebSocket & WAMP for Python on Twisted and asyncio. for Python 2 and 3, and running on Twisted and asyncio.

piccolo - A fast, user friendly ORM and query builder which supports asyncio.

  •    Python

A fast, user friendly ORM and query builder which supports asyncio. Read the docs. The syntax is clean and expressive.






We have large collection of open source products. Follow the tags from Tag Cloud >>


Open source products are scattered around the web. Please provide information about the open source projects you own / you use. Add Projects.