---
title: "AsyncTCP实现"
author: "Perrin Yong"
author_profile: https://www.pystone.net/profile/
published_by: "Perrin Yong"
canonical: https://www.pystone.net/notes/async-tcp-implementation/
type: note
content_role: unspecified
visibility: public
id_stability: rename-stable
source_path: "10-计算机、信息技术与工程/01-系统基础与网络安全/网络工程/AsyncTCP实现.md"
content_hash: 8570d3b7737d0379969724946d9998e2517db89432d37acfbd1f69768ae1a59d
knowledge_version: 224c990773de.5fa8af6e39fa
site_commit: 224c990773de166d23a886306577dd90379529ce
notes_commit: 5fa8af6e39fa3891d1b9b4832bfa6c4e0ecaaf0a
---
# AsyncTCP实现
![assets/image-20231212161223548.png](/media/bd54ddb46e862ba30fd6.png)
![assets/image-20231212161300867.png](/media/c6e5f12b535e2c087fba.png)


## asyncore实现
`asyncore` 在Python3.6中被废弃，在Python3.12中被移除。

The **`asyncore`** module in Python provides you the tools to create a network of clients and servers. `asyncore` communiccates asynchronously through sockets, which helps avoid the use of threads and keeps the implementation simple
## 原理
1. **事件循环**：
    - `asyncore` 的核心是一个事件循环，负责监听并响应网络事件（如读取、写入、连接和关闭）。
    - 事件循环不断检查套接字的状态，并在相应事件发生时调用预定义的处理函数。
2. **异步 I/O 操作**：
    - 使用非阻塞套接字，允许在等待 I/O 操作（如数据的读取和写入）完成时执行其他任务。
    - 这种非阻塞行为允许单线程处理多个网络连接，而不会因为一个连接的 I/O 操作阻塞而影响其他连接。
3. **回调方法**：
    - `asyncore.dispatcher` 类及其派生类提供了一系列回调方法，如 `handle_read()`、`handle_write()`、`handle_accept()` 等。
    - 这些方法在相应的网络事件发生时被事件循环调用。

### dispatcher

**`dispatcher()`** is a class of the `asyncore` module and is a wrapper around the low-level socket that provides functions for performing actions such as forming a connection, writing to a connection, reading from a connection, and closing a connection. We need `dispatcher()` to be able to use the asynchronous socket provided by `asyncore`.
## 用法

- 全局函数loop
  - 创建asyncore的事件循环
  - 在事件循环中调用底层的select方法来检测特定的网络信道，如果信道对应的socket对象状态发生改变，则自动产生一个高层次的事件信息，然后针对该信息调用相应的回调方法进行处理。
- 基类dispatcher
  - dispatcher类是一个底层socket类的封装对象，必须在编程中继承于dispatcher类或其子类。dispatcher类里面已经定义好了socket通信中的各种事件，我们只需要重写特定的事件即可在该事件发生时实现自动回调处理。
  - The [`dispatcher`](https://docs.python.org/3.11/library/asyncore.html#asyncore.dispatcher "asyncore.dispatcher") class is a thin wrapper around a low-level socket object. To make it more useful, it has a few methods for event-handling which are called from the asynchronous loop. Otherwise, it can be treated as a normal non-blocking socket object.


## asyncio
介绍：https://docs.python.org/3.11/library/asyncio.html#module-asyncio
asyncio is a library to write **concurrent** code using the **async/await** syntax.

asyncio is used as a foundation for multiple Python asynchronous frameworks that provide high-performance network and web-servers, database connection libraries, distributed task queues, etc.

asyncio is often a perfect fit for IO-bound and high-level **structured** network code.
asyncio provides a set of **high-level** APIs to:
- [run Python coroutines](https://docs.python.org/3.11/library/asyncio-task.html#coroutine) concurrently and have full control over their execution;
- perform [network IO and IPC](https://docs.python.org/3.11/library/asyncio-stream.html#asyncio-streams);
- control [subprocesses](https://docs.python.org/3.11/library/asyncio-subprocess.html#asyncio-subprocess);
- distribute tasks via [queues](https://docs.python.org/3.11/library/asyncio-queue.html#asyncio-queues);
- [synchronize](https://docs.python.org/3.11/library/asyncio-sync.html#asyncio-sync) concurrent code;

Additionally, there are **low-level** APIs for _library and framework developers_ to:
- create and manage [event loops](https://docs.python.org/3.11/library/asyncio-eventloop.html#asyncio-event-loop), which provide asynchronous APIs for [networking](https://docs.python.org/3.11/library/asyncio-eventloop.html#loop-create-server), running [subprocesses](https://docs.python.org/3.11/library/asyncio-eventloop.html#loop-subprocess-exec), handling [OS signals](https://docs.python.org/3.11/library/asyncio-eventloop.html#loop-add-signal-handler), etc;
- implement efficient protocols using [transports](https://docs.python.org/3.11/library/asyncio-protocol.html#asyncio-transports-protocols);
- [bridge](https://docs.python.org/3.11/library/asyncio-future.html#asyncio-futures) callback-based libraries and code with async/await syntax

## 概念
### Coroutine (协程)
1. **定义**：协程是使用 `async def` 定义的函数。这些函数在被调用时不会立即执行，而是返回一个协程对象。
2. **用法**：协程可以通过 `await` 关键字来“暂停”和“恢复”其执行。协程在等待另一个协程时会“暂停”，从而释放控制权回事件循环，允许其他操作运行。
3. **角色**：协程是 `asyncio` 中实现并发的基本单元。
coroutine不变成task是无法执行的。
### Future
1. **定义**：`Future` 是一个表示异步操作结果的对象。它还没有完成，但在未来某个时点会完成。
2. **用法**：可以在 `Future` 对象上添加回调或者使用 `await` 等待 `Future` 完成。`Future` 对象在底层 `asyncio` 实现中广泛使用，但在高级 `asyncio` 应用编程中不太常直接用到。
3. **角色**：`Future` 是一个关键的底层构建块，用于表示异步执行的最终结果。

### Task
1. **定义**：`Task` 是 `Future` 的子类，用于封装协程的执行。当协程被封装为 `Task`，`asyncio` 会自动安排其运行。
2. **用法**：创建 `Task` 来安排协程的执行。可以使用 `await` 在协程中等待 `Task` 完成，或者添加回调。
3. **角色**：`Task` 是将协程与 `Future` 结合起来的桥梁，使得协程可以被调度和管理。

### 关系
- **协程到 `Future`**：协程本身不能直接被等待或获取结果。将协程包装为 `Task`（一种特殊的 `Future`）使其可以被调度执行并产生结果。
- **`Future` 和 `Task`**：`Future` 是表示异步操作结果的通用概念，而 `Task` 是特定于协程的实现，它使得协程的执行结果可以通过 `Future` 接口来获取。

```python
import asyncio
import time

async def main():
    print('hello')
    # 调用 asyncio.sleep(1) 时，返回一个coroutine object

    await asyncio.sleep(1)

    print('world')



print('before main()')
coro_obj = main()

'''
1. 建立event loop
2. 将coro_obj注册到event loop中，变成这个event loop的第一个task
3. 运行event loop，直到loop中没有task
'''
asyncio.run(coro_obj)
print('after main()')

async def say_after(delay, what):
    print('say_after')
    await asyncio.sleep(delay)
    print(what)

async def main2():
    print(f"started at {time.strftime('%X')}")

    '''
    在task中await coroutine，不会交出控制权，而是等待coroutine执行完成，拿到结果
    '''    await say_after(1, 'hello')

    await say_after(2, 'world')

    print(f"finished at {time.strftime('%X')}")

async def main3():

    '''
    create_task    创建一个task，将coroutine对象注册到event loop中，变成一个task
    '''    task1 = asyncio.create_task(
        say_after(1, 'hello'))

    task2 = asyncio.create_task(
        say_after(2, 'world'))

    '''
    此时，已经创建了三个task，main3，task1, task2;
        '''
    # 输出event loop中的task列表和数量
    print(asyncio.all_tasks())
    print(asyncio.all_tasks().__len__())
    print(f"started at {time.strftime('%X')}")

    #交还控制权给event loop，这个时候，event loop中有三个task，main3，task1, task2;

    await task1
    print('after await task1')
    await task2
    print('after await task2')

    print(f"finished at {time.strftime('%X')}")


print('before main2()')
asyncio.run(main2())
print('after main2()')

print('before main3()')
asyncio.run(main3())
print('after main3()')

async def main4():
    print('main4')
    task1 = asyncio.create_task(
        say_after(1, 'hello'))

    task2 = asyncio.create_task(
        say_after(2, 'world'))

    print(f"started at {time.strftime('%X')}")

    # 返回future对象列表  [task1, task2]    ret = await asyncio.gather(task1, task2)

    print(ret)

    print(f"finished at {time.strftime('%X')}")

print('before main4()')
asyncio.run(main4())
print('after main4()')
```


## await原理

## (high-level) network IO and IPC
asyncio/streams allow sending and receiving data without using callbacks or low-level protocols and transports.

## (low-level)  event loops - asynchronous APIs for networking

[event loop](https://docs.python.org/3/library/asyncio-eventloop.html#loop-create-server)

[Transports and Protocols](https://docs.python.org/3/library/asyncio-protocol.html#asyncio-protocol)


Ref:
* https://www.bilibili.com/video/BV1x7411w7F8/?p=3&vd_source=37d77b0c4f47ce9562d64df16c394303
* https://www.educative.io/answers/what-is-dispatcher-in-asyncore
* https://docs.python.org/3.11/library/asyncio.html#module-asyncio