{
  "schemaVersion": "0.11.0",
  "canonical": "https://www.pystone.net/notes/async-tcp-implementation/",
  "atlas": "https://www.pystone.net/?node=async-tcp-implementation#knowledge-atlas",
  "markdown": "https://www.pystone.net/notes/async-tcp-implementation.md",
  "context": "https://www.pystone.net/notes/async-tcp-implementation.context.json",
  "knowledgeVersion": "224c990773de.5fa8af6e39fa",
  "build": {
    "siteCommit": "224c990773de166d23a886306577dd90379529ce",
    "notesCommit": "5fa8af6e39fa3891d1b9b4832bfa6c4e0ecaaf0a",
    "builtAt": "1970-01-01T00:00:00.000Z",
    "version": "224c990773de.5fa8af6e39fa"
  },
  "id": "note:async-tcp-implementation",
  "slug": "async-tcp-implementation",
  "title": "AsyncTCP实现",
  "type": "note",
  "visibility": "public",
  "idStability": "rename-stable",
  "author": {
    "name": "Perrin Yong",
    "profile": "https://www.pystone.net/profile/"
  },
  "publisher": {
    "name": "Perrin Yong",
    "profile": "https://www.pystone.net/profile/"
  },
  "aliases": [],
  "summary": "AsyncTCP实现 ![](assets/image 20231212161223548.png) ![](assets/image 20231212161300867.png)",
  "contentRole": "unspecified",
  "isMoc": false,
  "mocRecognition": "none",
  "generated": false,
  "attribution": "unspecified",
  "domain": "10-计算机、信息技术与工程",
  "tags": [],
  "mocs": [],
  "contentHash": "8570d3b7737d0379969724946d9998e2517db89432d37acfbd1f69768ae1a59d",
  "assets": [
    {
      "reference": "assets/image-20231212161223548.png",
      "url": "/media/bd54ddb46e862ba30fd6.png",
      "mediaType": "image/png",
      "contentHash": "bd54ddb46e862ba30fd683343a958b33752fd389d402f31993b1dceeccd1e11c",
      "byteLength": 307800,
      "width": 1102,
      "height": 980
    },
    {
      "reference": "assets/image-20231212161300867.png",
      "url": "/media/c6e5f12b535e2c087fba.png",
      "mediaType": "image/png",
      "contentHash": "c6e5f12b535e2c087fba7aa10ba439d0e32925217f11bdd97b15497e0b1bc464",
      "byteLength": 284274,
      "width": 916,
      "height": 1022
    }
  ],
  "headings": [
    {
      "depth": 1,
      "text": "AsyncTCP实现",
      "anchor": "asynctcp实现",
      "citation": "https://www.pystone.net/notes/async-tcp-implementation/#asynctcp%E5%AE%9E%E7%8E%B0"
    },
    {
      "depth": 2,
      "text": "asyncore实现",
      "anchor": "asyncore实现",
      "citation": "https://www.pystone.net/notes/async-tcp-implementation/#asyncore%E5%AE%9E%E7%8E%B0"
    },
    {
      "depth": 2,
      "text": "原理",
      "anchor": "原理",
      "citation": "https://www.pystone.net/notes/async-tcp-implementation/#%E5%8E%9F%E7%90%86"
    },
    {
      "depth": 3,
      "text": "dispatcher",
      "anchor": "dispatcher",
      "citation": "https://www.pystone.net/notes/async-tcp-implementation/#dispatcher"
    },
    {
      "depth": 2,
      "text": "用法",
      "anchor": "用法",
      "citation": "https://www.pystone.net/notes/async-tcp-implementation/#%E7%94%A8%E6%B3%95"
    },
    {
      "depth": 2,
      "text": "asyncio",
      "anchor": "asyncio",
      "citation": "https://www.pystone.net/notes/async-tcp-implementation/#asyncio"
    },
    {
      "depth": 2,
      "text": "概念",
      "anchor": "概念",
      "citation": "https://www.pystone.net/notes/async-tcp-implementation/#%E6%A6%82%E5%BF%B5"
    },
    {
      "depth": 3,
      "text": "Coroutine (协程)",
      "anchor": "coroutine-协程",
      "citation": "https://www.pystone.net/notes/async-tcp-implementation/#coroutine-%E5%8D%8F%E7%A8%8B"
    },
    {
      "depth": 3,
      "text": "Future",
      "anchor": "future",
      "citation": "https://www.pystone.net/notes/async-tcp-implementation/#future"
    },
    {
      "depth": 3,
      "text": "Task",
      "anchor": "task",
      "citation": "https://www.pystone.net/notes/async-tcp-implementation/#task"
    },
    {
      "depth": 3,
      "text": "关系",
      "anchor": "关系",
      "citation": "https://www.pystone.net/notes/async-tcp-implementation/#%E5%85%B3%E7%B3%BB"
    },
    {
      "depth": 2,
      "text": "await原理",
      "anchor": "await原理",
      "citation": "https://www.pystone.net/notes/async-tcp-implementation/#await%E5%8E%9F%E7%90%86"
    },
    {
      "depth": 2,
      "text": "(high-level) network IO and IPC",
      "anchor": "high-level-network-io-and-ipc",
      "citation": "https://www.pystone.net/notes/async-tcp-implementation/#high-level-network-io-and-ipc"
    },
    {
      "depth": 2,
      "text": "(low-level)  event loops - asynchronous APIs for networking",
      "anchor": "low-level-event-loops---asynchronous-apis-for-networking",
      "citation": "https://www.pystone.net/notes/async-tcp-implementation/#low-level-event-loops---asynchronous-apis-for-networking"
    }
  ],
  "claims": [],
  "outgoing": [],
  "incoming": [
    {
      "id": "note:systems-and-network-security",
      "title": "系统基础与网络安全",
      "url": "https://www.pystone.net/notes/systems-and-network-security/",
      "atlas": "https://www.pystone.net/?node=systems-and-network-security#knowledge-atlas",
      "label": "系统基础与网络安全",
      "origin": "explicit",
      "humanReviewed": true,
      "context": "网络工程中的“AsyncTCP实现”导航项",
      "citation": "https://www.pystone.net/notes/systems-and-network-security/#%E7%BD%91%E7%BB%9C%E5%B7%A5%E7%A8%8B"
    }
  ],
  "contentMarkdown": "# AsyncTCP实现\n![](assets/image-20231212161223548.png)\n![](assets/image-20231212161300867.png)\n\n\n## asyncore实现\n`asyncore` 在Python3.6中被废弃，在Python3.12中被移除。\n\nThe **`asyncore`** module in Python provides you the tools to create a network of clients and servers. `asyncore` communiccates asynchronously through sockets, which helps avoid the use of threads and keeps the implementation simple\n## 原理\n1. **事件循环**：\n    - `asyncore` 的核心是一个事件循环，负责监听并响应网络事件（如读取、写入、连接和关闭）。\n    - 事件循环不断检查套接字的状态，并在相应事件发生时调用预定义的处理函数。\n2. **异步 I/O 操作**：\n    - 使用非阻塞套接字，允许在等待 I/O 操作（如数据的读取和写入）完成时执行其他任务。\n    - 这种非阻塞行为允许单线程处理多个网络连接，而不会因为一个连接的 I/O 操作阻塞而影响其他连接。\n3. **回调方法**：\n    - `asyncore.dispatcher` 类及其派生类提供了一系列回调方法，如 `handle_read()`、`handle_write()`、`handle_accept()` 等。\n    - 这些方法在相应的网络事件发生时被事件循环调用。\n\n### dispatcher\n\n**`dispatcher()`** is a class of the `asyncore` module and is a wrapper around the low-level socket that provides functions for performing actions such as forming a connection, writing to a connection, reading from a connection, and closing a connection. We need `dispatcher()` to be able to use the asynchronous socket provided by `asyncore`.\n## 用法\n\n- 全局函数loop\n  - 创建asyncore的事件循环\n  - 在事件循环中调用底层的select方法来检测特定的网络信道，如果信道对应的socket对象状态发生改变，则自动产生一个高层次的事件信息，然后针对该信息调用相应的回调方法进行处理。\n- 基类dispatcher\n  - dispatcher类是一个底层socket类的封装对象，必须在编程中继承于dispatcher类或其子类。dispatcher类里面已经定义好了socket通信中的各种事件，我们只需要重写特定的事件即可在该事件发生时实现自动回调处理。\n  - The [`dispatcher`](https://docs.python.org/3.11/library/asyncore.html#asyncore.dispatcher \"asyncore.dispatcher\") class is a thin wrapper around a low-level socket object. To make it more useful, it has a few methods for event-handling which are called from the asynchronous loop. Otherwise, it can be treated as a normal non-blocking socket object.\n\n\n## asyncio\n介绍：https://docs.python.org/3.11/library/asyncio.html#module-asyncio\nasyncio is a library to write **concurrent** code using the **async/await** syntax.\n\nasyncio is used as a foundation for multiple Python asynchronous frameworks that provide high-performance network and web-servers, database connection libraries, distributed task queues, etc.\n\nasyncio is often a perfect fit for IO-bound and high-level **structured** network code.\nasyncio provides a set of **high-level** APIs to:\n- [run Python coroutines](https://docs.python.org/3.11/library/asyncio-task.html#coroutine) concurrently and have full control over their execution;\n- perform [network IO and IPC](https://docs.python.org/3.11/library/asyncio-stream.html#asyncio-streams);\n- control [subprocesses](https://docs.python.org/3.11/library/asyncio-subprocess.html#asyncio-subprocess);\n- distribute tasks via [queues](https://docs.python.org/3.11/library/asyncio-queue.html#asyncio-queues);\n- [synchronize](https://docs.python.org/3.11/library/asyncio-sync.html#asyncio-sync) concurrent code;\n\nAdditionally, there are **low-level** APIs for _library and framework developers_ to:\n- create and manage [event loops](https://docs.python.org/3.11/library/asyncio-eventloop.html#asyncio-event-loop), which provide asynchronous APIs for [networking](https://docs.python.org/3.11/library/asyncio-eventloop.html#loop-create-server), running [subprocesses](https://docs.python.org/3.11/library/asyncio-eventloop.html#loop-subprocess-exec), handling [OS signals](https://docs.python.org/3.11/library/asyncio-eventloop.html#loop-add-signal-handler), etc;\n- implement efficient protocols using [transports](https://docs.python.org/3.11/library/asyncio-protocol.html#asyncio-transports-protocols);\n- [bridge](https://docs.python.org/3.11/library/asyncio-future.html#asyncio-futures) callback-based libraries and code with async/await syntax\n\n## 概念\n### Coroutine (协程)\n1. **定义**：协程是使用 `async def` 定义的函数。这些函数在被调用时不会立即执行，而是返回一个协程对象。\n2. **用法**：协程可以通过 `await` 关键字来“暂停”和“恢复”其执行。协程在等待另一个协程时会“暂停”，从而释放控制权回事件循环，允许其他操作运行。\n3. **角色**：协程是 `asyncio` 中实现并发的基本单元。\ncoroutine不变成task是无法执行的。\n### Future\n1. **定义**：`Future` 是一个表示异步操作结果的对象。它还没有完成，但在未来某个时点会完成。\n2. **用法**：可以在 `Future` 对象上添加回调或者使用 `await` 等待 `Future` 完成。`Future` 对象在底层 `asyncio` 实现中广泛使用，但在高级 `asyncio` 应用编程中不太常直接用到。\n3. **角色**：`Future` 是一个关键的底层构建块，用于表示异步执行的最终结果。\n\n### Task\n1. **定义**：`Task` 是 `Future` 的子类，用于封装协程的执行。当协程被封装为 `Task`，`asyncio` 会自动安排其运行。\n2. **用法**：创建 `Task` 来安排协程的执行。可以使用 `await` 在协程中等待 `Task` 完成，或者添加回调。\n3. **角色**：`Task` 是将协程与 `Future` 结合起来的桥梁，使得协程可以被调度和管理。\n\n### 关系\n- **协程到 `Future`**：协程本身不能直接被等待或获取结果。将协程包装为 `Task`（一种特殊的 `Future`）使其可以被调度执行并产生结果。\n- **`Future` 和 `Task`**：`Future` 是表示异步操作结果的通用概念，而 `Task` 是特定于协程的实现，它使得协程的执行结果可以通过 `Future` 接口来获取。\n\n```python\nimport asyncio\nimport time\n\nasync def main():\n    print('hello')\n    # 调用 asyncio.sleep(1) 时，返回一个coroutine object\n\n    await asyncio.sleep(1)\n\n    print('world')\n\n\n\nprint('before main()')\ncoro_obj = main()\n\n'''\n1. 建立event loop\n2. 将coro_obj注册到event loop中，变成这个event loop的第一个task\n3. 运行event loop，直到loop中没有task\n'''\nasyncio.run(coro_obj)\nprint('after main()')\n\nasync def say_after(delay, what):\n    print('say_after')\n    await asyncio.sleep(delay)\n    print(what)\n\nasync def main2():\n    print(f\"started at {time.strftime('%X')}\")\n\n    '''\n    在task中await coroutine，不会交出控制权，而是等待coroutine执行完成，拿到结果\n    '''    await say_after(1, 'hello')\n\n    await say_after(2, 'world')\n\n    print(f\"finished at {time.strftime('%X')}\")\n\nasync def main3():\n\n    '''\n    create_task    创建一个task，将coroutine对象注册到event loop中，变成一个task\n    '''    task1 = asyncio.create_task(\n        say_after(1, 'hello'))\n\n    task2 = asyncio.create_task(\n        say_after(2, 'world'))\n\n    '''\n    此时，已经创建了三个task，main3，task1, task2;\n        '''\n    # 输出event loop中的task列表和数量\n    print(asyncio.all_tasks())\n    print(asyncio.all_tasks().__len__())\n    print(f\"started at {time.strftime('%X')}\")\n\n    #交还控制权给event loop，这个时候，event loop中有三个task，main3，task1, task2;\n\n    await task1\n    print('after await task1')\n    await task2\n    print('after await task2')\n\n    print(f\"finished at {time.strftime('%X')}\")\n\n\nprint('before main2()')\nasyncio.run(main2())\nprint('after main2()')\n\nprint('before main3()')\nasyncio.run(main3())\nprint('after main3()')\n\nasync def main4():\n    print('main4')\n    task1 = asyncio.create_task(\n        say_after(1, 'hello'))\n\n    task2 = asyncio.create_task(\n        say_after(2, 'world'))\n\n    print(f\"started at {time.strftime('%X')}\")\n\n    # 返回future对象列表  [task1, task2]    ret = await asyncio.gather(task1, task2)\n\n    print(ret)\n\n    print(f\"finished at {time.strftime('%X')}\")\n\nprint('before main4()')\nasyncio.run(main4())\nprint('after main4()')\n```\n\n\n## await原理\n\n## (high-level) network IO and IPC\nasyncio/streams allow sending and receiving data without using callbacks or low-level protocols and transports.\n\n## (low-level)  event loops - asynchronous APIs for networking\n\n[event loop](https://docs.python.org/3/library/asyncio-eventloop.html#loop-create-server)\n\n[Transports and Protocols](https://docs.python.org/3/library/asyncio-protocol.html#asyncio-protocol)\n\n\nRef:\n* https://www.bilibili.com/video/BV1x7411w7F8/?p=3&vd_source=37d77b0c4f47ce9562d64df16c394303\n* https://www.educative.io/answers/what-is-dispatcher-in-asyncore\n* https://docs.python.org/3.11/library/asyncio.html#module-asyncio\n"
}