{ "schemaVersion": "0.11.0", "canonical": "https://www.pystone.net/notes/texture-filtering-techniques/", "atlas": "https://www.pystone.net/?node=texture-filtering-techniques#knowledge-atlas", "markdown": "https://www.pystone.net/notes/texture-filtering-techniques.md", "context": "https://www.pystone.net/notes/texture-filtering-techniques.context.json", "knowledgeVersion": "224c990773de.5fa8af6e39fa", "build": { "siteCommit": "224c990773de166d23a886306577dd90379529ce", "notesCommit": "5fa8af6e39fa3891d1b9b4832bfa6c4e0ecaaf0a", "builtAt": "1970-01-01T00:00:00.000Z", "version": "224c990773de.5fa8af6e39fa" }, "id": "note:texture-filtering-techniques", "slug": "texture-filtering-techniques", "title": "纹理滤波相关知识整理", "type": "note", "visibility": "public", "idStability": "rename-stable", "author": { "name": "Perrin Yong", "profile": "https://www.pystone.net/profile/" }, "publisher": { "name": "Perrin Yong", "profile": "https://www.pystone.net/profile/" }, "aliases": [], "summary": "纹理滤波相关知识整理", "contentRole": "unspecified", "isMoc": false, "mocRecognition": "none", "generated": false, "attribution": "unspecified", "domain": "10-计算机、信息技术与工程", "tags": [], "mocs": [], "contentHash": "2e1a23e30495afbda3e50649a5cdd5c5f8a1cd168b4ef612b0886fb1d6575e17", "assets": [ { "reference": "assets/1603809936492.png", "url": "/media/b9aa2e2e2964b909b73d.png", "mediaType": "image/png", "contentHash": "b9aa2e2e2964b909b73d05b30bc84eead00a66279ca8687113ee1b25e622ee5c", "byteLength": 28484, "width": 800, "height": 800 }, { "reference": "assets/1605006132395.png", "url": "/media/96fe2c4e216e32e63b47.png", "mediaType": "image/png", "contentHash": "96fe2c4e216e32e63b477cf72d19b49711acab89875892ad1e15bdb1b8fe9e91", "byteLength": 250963, "width": 557, "height": 261 }, { "reference": "assets/bg2017121302.png", "url": "/media/282a9635d81fd65aa293.png", "mediaType": "image/png", "contentHash": "282a9635d81fd65aa2938d68fd98450384ca777975fef28ec474b249bcc709b0", "byteLength": 37635, "width": 880, "height": 440 }, { "reference": "assets/1603787295513.png", "url": "/media/06f2acce78822f1a06c6.png", "mediaType": "image/png", "contentHash": "06f2acce78822f1a06c6a7f3df787937f41c27c9664a78f6fdb8477c2c28e2d2", "byteLength": 481127, "width": 817, "height": 412 }, { "reference": "assets/1603787306380.png", "url": "/media/bf4d82182413d6d1dcd0.png", "mediaType": "image/png", "contentHash": "bf4d82182413d6d1dcd06be2913c776bddb6651bdf6027ac6008b7a1b6821674", "byteLength": 405574, "width": 815, "height": 410 }, { "reference": "assets/MemoryOfMipmap.png", "url": "/media/9a533e92dc5f8e601b14.png", "mediaType": "image/png", "contentHash": "9a533e92dc5f8e601b1491a08116535133c0869c4f1ffb56ff30964867f6d32b", "byteLength": 4486, "width": 254, "height": 108 }, { "reference": "assets/1603790161075.png", "url": "/media/6cd5377847abd8c29de6.png", "mediaType": "image/png", "contentHash": "6cd5377847abd8c29de6ea4559d35eac40824cf991a9b41ba850699afb637c02", "byteLength": 67101, "width": 1280, "height": 1010 }, { "reference": "assets/1603854909317.png", "url": "/media/805eeddc794c5ca22064.png", "mediaType": "image/png", "contentHash": "805eeddc794c5ca22064af92daf586ab9f33c4d55ee3b0bba39cfb527fafe5a5", "byteLength": 304952, "width": 935, "height": 453 }, { "reference": "assets/1603855058444.png", "url": "/media/87c750861899c36b7efd.png", "mediaType": "image/png", "contentHash": "87c750861899c36b7efd7c0ff18b8e7ff82d9e2e001436707b726a2a26044d40", "byteLength": 230801, "width": 945, "height": 400 }, { "reference": "assets/1603810157411.png", "url": "/media/472e4f2627347d39ca61.png", "mediaType": "image/png", "contentHash": "472e4f2627347d39ca61a143b9727aa7afef6a0eeb7c9cebd91d67b2c47bae91", "byteLength": 40676, "width": 1024, "height": 967 }, { "reference": "assets/1603857020919.png", "url": "/media/32072bbf0503d197e8e0.png", "mediaType": "image/png", "contentHash": "32072bbf0503d197e8e0dfd682a6735ed399199ae74a239678be6f6b88416581", "byteLength": 8796, "width": 364, "height": 87 }, { "reference": "assets/1603857053404.png", "url": "/media/6bc73b4e34a7edbce0cf.png", "mediaType": "image/png", "contentHash": "6bc73b4e34a7edbce0cfdb3c5da8d781a1e36827c3c99122e0510ea3098d9e88", "byteLength": 7331, "width": 383, "height": 54 }, { "reference": "assets/1603855288637.png", "url": "/media/514f676643289fde5de5.png", "mediaType": "image/png", "contentHash": "514f676643289fde5de575208691290adcc078e374bbe7c883a60d24757352c2", "byteLength": 377605, "width": 922, "height": 464 }, { "reference": "assets/1603855360374.png", "url": "/media/eccab153dc7380bc2c27.png", "mediaType": "image/png", "contentHash": "eccab153dc7380bc2c27852c5d0ce7026b5dcc96d7cbd899c429e0ef16a4e870", "byteLength": 376940, "width": 948, "height": 475 }, { "reference": "assets/1609139946633.png", "url": "/media/81cb90eb90b1da6e3f2e.png", "mediaType": "image/png", "contentHash": "81cb90eb90b1da6e3f2e09670c99440df28dcb334a001764f439cd450355b439", "byteLength": 333543, "width": 511, "height": 511 }, { "reference": "assets/1603855799619.png", "url": "/media/34098c8d409a29556cb3.png", "mediaType": "image/png", "contentHash": "34098c8d409a29556cb33f8675784416a2ca3f0842347f67c8f07b7ab28ee0dd", "byteLength": 340762, "width": 946, "height": 426 }, { "reference": "assets/1603855820036.png", "url": "/media/f445b5977cd7f70802e4.png", "mediaType": "image/png", "contentHash": "f445b5977cd7f70802e41b0e01bfaaca3541a95a01d410e31832031895c476b2", "byteLength": 385518, "width": 945, "height": 451 }, { "reference": "assets/1603865152983.png", "url": "/media/cc3211bd0e020b521045.png", "mediaType": "image/png", "contentHash": "cc3211bd0e020b5210450d5d69436dbf096a3d02520965219acd3ef6a552d004", "byteLength": 31136, "width": 378, "height": 225 }, { "reference": "assets/1603864791127.png", "url": "/media/f4148bb77fdd7256c71f.png", "mediaType": "image/png", "contentHash": "f4148bb77fdd7256c71f4d554bdc8809c092bdf4c8ccc217b4f26b32b292ef95", "byteLength": 10615, "width": 366, "height": 68 } ], "headings": [ { "depth": 1, "text": "纹理滤波相关知识整理", "anchor": "纹理滤波相关知识整理", "citation": "https://www.pystone.net/notes/texture-filtering-techniques/#%E7%BA%B9%E7%90%86%E6%BB%A4%E6%B3%A2%E7%9B%B8%E5%85%B3%E7%9F%A5%E8%AF%86%E6%95%B4%E7%90%86" }, { "depth": 2, "text": "相关概念", "anchor": "相关概念", "citation": "https://www.pystone.net/notes/texture-filtering-techniques/#%E7%9B%B8%E5%85%B3%E6%A6%82%E5%BF%B5" }, { "depth": 3, "text": "纹理映射 与 纹理采样", "anchor": "纹理映射-与-纹理采样", "citation": "https://www.pystone.net/notes/texture-filtering-techniques/#%E7%BA%B9%E7%90%86%E6%98%A0%E5%B0%84-%E4%B8%8E-%E7%BA%B9%E7%90%86%E9%87%87%E6%A0%B7" }, { "depth": 3, "text": "插值(Interpolation)的概念", "anchor": "插值interpolation的概念", "citation": "https://www.pystone.net/notes/texture-filtering-techniques/#%E6%8F%92%E5%80%BCinterpolation%E7%9A%84%E6%A6%82%E5%BF%B5" }, { "depth": 4, "text": "线性插值(Linear interpolation)", "anchor": "线性插值linear-interpolation", "citation": "https://www.pystone.net/notes/texture-filtering-techniques/#%E7%BA%BF%E6%80%A7%E6%8F%92%E5%80%BClinear-interpolation" }, { "depth": 2, "text": "纹理滤波（Texture Filtering）", "anchor": "纹理滤波texture-filtering", "citation": "https://www.pystone.net/notes/texture-filtering-techniques/#%E7%BA%B9%E7%90%86%E6%BB%A4%E6%B3%A2texture-filtering" }, { "depth": 3, "text": "为什么需要纹理滤波", "anchor": "为什么需要纹理滤波", "citation": "https://www.pystone.net/notes/texture-filtering-techniques/#%E4%B8%BA%E4%BB%80%E4%B9%88%E9%9C%80%E8%A6%81%E7%BA%B9%E7%90%86%E6%BB%A4%E6%B3%A2" }, { "depth": 3, "text": "为什么称为滤波(Filter)", "anchor": "为什么称为滤波filter", "citation": "https://www.pystone.net/notes/texture-filtering-techniques/#%E4%B8%BA%E4%BB%80%E4%B9%88%E7%A7%B0%E4%B8%BA%E6%BB%A4%E6%B3%A2filter" }, { "depth": 4, "text": "图像是波", "anchor": "图像是波", "citation": "https://www.pystone.net/notes/texture-filtering-techniques/#%E5%9B%BE%E5%83%8F%E6%98%AF%E6%B3%A2" }, { "depth": 4, "text": "频率（frequency）", "anchor": "频率frequency", "citation": "https://www.pystone.net/notes/texture-filtering-techniques/#%E9%A2%91%E7%8E%87frequency" }, { "depth": 4, "text": "滤波器（filter）及在图像中的应用", "anchor": "滤波器filter及在图像中的应用", "citation": "https://www.pystone.net/notes/texture-filtering-techniques/#%E6%BB%A4%E6%B3%A2%E5%99%A8filter%E5%8F%8A%E5%9C%A8%E5%9B%BE%E5%83%8F%E4%B8%AD%E7%9A%84%E5%BA%94%E7%94%A8" }, { "depth": 3, "text": "Mipmapping", "anchor": "mipmapping", "citation": "https://www.pystone.net/notes/texture-filtering-techniques/#mipmapping" }, { "depth": 4, "text": "使用原因", "anchor": "使用原因", "citation": "https://www.pystone.net/notes/texture-filtering-techniques/#%E4%BD%BF%E7%94%A8%E5%8E%9F%E5%9B%A0" }, { "depth": 4, "text": "实现方法", "anchor": "实现方法", "citation": "https://www.pystone.net/notes/texture-filtering-techniques/#%E5%AE%9E%E7%8E%B0%E6%96%B9%E6%B3%95" }, { "depth": 4, "text": "如何选择Mipmap Level", "anchor": "如何选择mipmap-level", "citation": "https://www.pystone.net/notes/texture-filtering-techniques/#%E5%A6%82%E4%BD%95%E9%80%89%E6%8B%A9mipmap-level" }, { "depth": 4, "text": "优势", "anchor": "优势", "citation": "https://www.pystone.net/notes/texture-filtering-techniques/#%E4%BC%98%E5%8A%BF" }, { "depth": 4, "text": "缺点", "anchor": "缺点", "citation": "https://www.pystone.net/notes/texture-filtering-techniques/#%E7%BC%BA%E7%82%B9" }, { "depth": 2, "text": "常用滤波方法（插值算法）", "anchor": "常用滤波方法插值算法", "citation": "https://www.pystone.net/notes/texture-filtering-techniques/#%E5%B8%B8%E7%94%A8%E6%BB%A4%E6%B3%A2%E6%96%B9%E6%B3%95%E6%8F%92%E5%80%BC%E7%AE%97%E6%B3%95" }, { "depth": 3, "text": "最近邻插值采样(Nearest-neighbor interpolation)", "anchor": "最近邻插值采样nearest-neighbor-interpolation", "citation": "https://www.pystone.net/notes/texture-filtering-techniques/#%E6%9C%80%E8%BF%91%E9%82%BB%E6%8F%92%E5%80%BC%E9%87%87%E6%A0%B7nearest-neighbor-interpolation" }, { "depth": 4, "text": "优势", "anchor": "优势-1", "citation": "https://www.pystone.net/notes/texture-filtering-techniques/#%E4%BC%98%E5%8A%BF-1" }, { "depth": 4, "text": "不足", "anchor": "不足", "citation": "https://www.pystone.net/notes/texture-filtering-techniques/#%E4%B8%8D%E8%B6%B3" }, { "depth": 5, "text": "块状（blocky ）", "anchor": "块状blocky", "citation": "https://www.pystone.net/notes/texture-filtering-techniques/#%E5%9D%97%E7%8A%B6blocky" }, { "depth": 5, "text": "高频走样（波纹和闪烁现象）", "anchor": "高频走样波纹和闪烁现象", "citation": "https://www.pystone.net/notes/texture-filtering-techniques/#%E9%AB%98%E9%A2%91%E8%B5%B0%E6%A0%B7%E6%B3%A2%E7%BA%B9%E5%92%8C%E9%97%AA%E7%83%81%E7%8E%B0%E8%B1%A1" }, { "depth": 5, "text": "性能上", "anchor": "性能上", "citation": "https://www.pystone.net/notes/texture-filtering-techniques/#%E6%80%A7%E8%83%BD%E4%B8%8A" }, { "depth": 4, "text": "示例", "anchor": "示例", "citation": "https://www.pystone.net/notes/texture-filtering-techniques/#%E7%A4%BA%E4%BE%8B" }, { "depth": 3, "text": "最近点插值+mipmap的方式", "anchor": "最近点插值mipmap的方式", "citation": "https://www.pystone.net/notes/texture-filtering-techniques/#%E6%9C%80%E8%BF%91%E7%82%B9%E6%8F%92%E5%80%BCmipmap%E7%9A%84%E6%96%B9%E5%BC%8F" }, { "depth": 3, "text": "双线性插值 (Bilinear interpolation)", "anchor": "双线性插值-bilinear-interpolation", "citation": "https://www.pystone.net/notes/texture-filtering-techniques/#%E5%8F%8C%E7%BA%BF%E6%80%A7%E6%8F%92%E5%80%BC-bilinear-interpolation" }, { "depth": 3, "text": "三线性插值 (Trilinear interpolation)", "anchor": "三线性插值-trilinear-interpolation", "citation": "https://www.pystone.net/notes/texture-filtering-techniques/#%E4%B8%89%E7%BA%BF%E6%80%A7%E6%8F%92%E5%80%BC-trilinear-interpolation" }, { "depth": 4, "text": "不足", "anchor": "不足-1", "citation": "https://www.pystone.net/notes/texture-filtering-techniques/#%E4%B8%8D%E8%B6%B3-1" }, { "depth": 3, "text": "各向异性过滤(Anisotropic filtering)", "anchor": "各向异性过滤anisotropic-filtering", "citation": "https://www.pystone.net/notes/texture-filtering-techniques/#%E5%90%84%E5%90%91%E5%BC%82%E6%80%A7%E8%BF%87%E6%BB%A4anisotropic-filtering" }, { "depth": 4, "text": "概念", "anchor": "概念", "citation": "https://www.pystone.net/notes/texture-filtering-techniques/#%E6%A6%82%E5%BF%B5" }, { "depth": 4, "text": "原理解释", "anchor": "原理解释", "citation": "https://www.pystone.net/notes/texture-filtering-techniques/#%E5%8E%9F%E7%90%86%E8%A7%A3%E9%87%8A" }, { "depth": 2, "text": "OpenGL对Minification/Magnification Filter的选择", "anchor": "opengl对minificationmagnification-filter的选择", "citation": "https://www.pystone.net/notes/texture-filtering-techniques/#opengl%E5%AF%B9minificationmagnification-filter%E7%9A%84%E9%80%89%E6%8B%A9" }, { "depth": 2, "text": "拓展知识", "anchor": "拓展知识", "citation": "https://www.pystone.net/notes/texture-filtering-techniques/#%E6%8B%93%E5%B1%95%E7%9F%A5%E8%AF%86" }, { "depth": 3, "text": "纹理环绕（Texture Wrap）", "anchor": "纹理环绕texture-wrap", "citation": "https://www.pystone.net/notes/texture-filtering-techniques/#%E7%BA%B9%E7%90%86%E7%8E%AF%E7%BB%95texture-wrap" }, { "depth": 3, "text": "上/下采样", "anchor": "上下采样", "citation": "https://www.pystone.net/notes/texture-filtering-techniques/#%E4%B8%8A%E4%B8%8B%E9%87%87%E6%A0%B7" }, { "depth": 4, "text": "downsample（下采样）", "anchor": "downsample下采样", "citation": "https://www.pystone.net/notes/texture-filtering-techniques/#downsample%E4%B8%8B%E9%87%87%E6%A0%B7" }, { "depth": 4, "text": "upsample（上采样）", "anchor": "upsample上采样", "citation": "https://www.pystone.net/notes/texture-filtering-techniques/#upsample%E4%B8%8A%E9%87%87%E6%A0%B7" }, { "depth": 2, "text": "Ref", "anchor": "ref", "citation": "https://www.pystone.net/notes/texture-filtering-techniques/#ref" } ], "claims": [], "outgoing": [], "incoming": [ { "id": "note:game-graphics-and-runtime", "title": "游戏图形与运行时", "url": "https://www.pystone.net/notes/game-graphics-and-runtime/", "atlas": "https://www.pystone.net/?node=game-graphics-and-runtime#knowledge-atlas", "label": "游戏图形与运行时", "origin": "explicit", "humanReviewed": true, "context": "渲染基础中的“纹理滤波相关知识整理”导航项", "citation": "https://www.pystone.net/notes/game-graphics-and-runtime/#%E6%B8%B2%E6%9F%93%E5%9F%BA%E7%A1%80" } ], "contentMarkdown": "# 纹理滤波相关知识整理\n\n> 创建时间：2020/10/27 10:39\n\n * 纹理滤波相关知识整理\n * 相关概念\n * 纹理映射 与 纹理采样\n * 插值(Interpolation)的概念\n * 线性插值(Linear interpolation)\n * 纹理滤波（Texture Filtering）\n * 为什么需要纹理滤波\n * 为什么称为滤波(Filter)\n * 图像是波\n * 频率（frequency）\n * 滤波器（filter）及在图像中的应用\n * Mipmapping\n * 使用原因\n * 实现方法\n * 如何选择Mipmap Level\n * 优势\n * 缺点\n * 常用滤波方法（插值算法）\n * 最近邻插值采样(Nearest-neighbor interpolation)\n * 优势\n * 不足\n * 块状（blocky ）\n * 高频走样（波纹和闪烁现象）\n * 性能上\n * 示例\n * 最近点插值+mipmap的方式\n * 双线性插值 (Bilinear interpolation)\n * 三线性插值 (Trilinear interpolation)\n * 不足\n * 各向异性过滤(Anisotropic filtering)\n * 概念\n * 原理解释\n * OpenGL对Minification/Magnification Filter的选择\n * 拓展知识\n * 纹理环绕（Texture Wrap）\n * 上/下采样\n * downsample（下采样）\n * upsample（上采样）\n * Ref\n\n## 相关概念\n\n### 纹理映射 与 纹理采样\n\n纹理映射：使用uv坐标将纹理像素(texel)映射到屏幕像素(pixel)的过程。\n\n纹理采样（Texture Sample）：既然纹理是一张图片，那么自然就有分辨率的存在，纹理采样便是从纹理图片中采集一个像素颜色的操作。\n\n### 插值(Interpolation)的概念\n\n在数学的数值分析领域中，内插或称插值（英语：interpolation）是一种通过已知的、离散的数据点，在范围内推求新数据点的过程或方法。求解科学和工程的问题时，通常有许多数据点借由采样、实验等方法获得，这些数据可能代表了有限个数值函数，其中自变量的值。而根据这些数据，我们往往希望得到一个连续的函数（也就是曲线）；或者更密集的离散方程与已知数据互相吻合，这个过程叫做拟合。\n\n常用的插值方法有：最近邻插值（Nearest-neighbor interpolation）、线性插值（线性插值）、多项式插值（Polynomial interpolation）、样条曲线插值（Spline interpolation）等。\n\n高维度空间的计算中，需要用到多元插值（Multivariate interpolation）。方法有：\n\n * 双线性插值(bilinear interpolation) 和 双三次插值(bicubic interpolation) in two dimensions\n\n * trilinear interpolation in three dimensions.\n\n#### 线性插值(Linear interpolation)\n\nIn mathematics, linear interpolation is a method of curve fitting using linear polynomials to construct new data points within the range of a discrete set of known data points.\n\n![Alt text](assets/1603809936492.png)\n\n## 纹理滤波（Texture Filtering）\n\n**纹理滤波（Texture Filtering）** 又称为 **纹理平滑（texture smoothing ）** \\- the method used to determine the texture color for a texture mapped pixel, using the colors of nearby texels (pixels of the texture).\n\n> 当三维空间里面的多边形经过坐标变换、投影、光栅化等过程，变成二维屏幕上的一组象素的时候，对每个象素需要到相应纹理图像中进行采样， **纹理过滤** 或者说 **纹理平滑** 是在纹理采样中使采样结果更加合理，以减少各种人为产生的穿帮现象的技术。\n\n纹理滤波主要分为两类：\n\n * **放大过滤（magnification filtering）** : reconstruction filter where sparse data is interpolated to fill gaps. （填充缺口）。一个纹理单元对应着多个像素。需要进行插值，平滑，防止马赛克现象。\n\n * **缩小过滤（minification filtering）** : anti-aliasing (AA), where texture samples exist at a higher frequency than required for the sample frequency needed for texture fill. （抗锯齿）。一个像素对应着多个纹理单元。\n\n> There are many methods of texture filtering, which make different trade-offs between computational complexity, memory bandwidth and image quality.\n\n### 为什么需要纹理滤波\n\n对于任意的3D表面在纹理映射过程中，需要进行纹理查找来找到屏幕上的一个像素对应于纹理上的哪个位置，而在纹理采样过程中，可能出现如下问题：\n\n 1. 三维图形上的pixel中心和纹理上的texel中心并不一至。屏幕上像素的纹理映射坐标可能不会完美地落到像素格上。\n\n 2. 取决于观察者，textured surface可能在任意的距离和方向，一个像素（pixel）通常不会直接对应一个纹素（texel），可能一个像素被映射到许多纹理像素，或者许多个像素都映射到相同纹素。\n因此，需要执行一些形式的滤波来决定屏幕像素的最佳颜色。滤波不足或不正确将在图像中表现为伪像（图像中的错误），例如“块状”，锯齿状或闪烁。\n\n* * *\n\n * 纹理放大(Texture Magnification)：the texels are larger than screen pixels, and need to be scaled up appropriately - a process known as texture magnification.\n当最后呈现到屏幕中的尺寸比纹理实际尺寸大的时候，许多个像素都映射到同一纹素。纹理放大如果不进行过滤，会造成马赛克现象。此时需要对纹理进行放大过滤（magnification filtering），使用插值方法，进行平滑。\n\n * 纹理缩小(Texture Minification)：each texel is smaller than a pixel, and so one pixel covers multiple texels. In this case an appropriate color has to be picked based on the covered texels, via texture minification.\n当最后呈现到屏幕中的尺寸比纹理实际尺寸小的时候，一个像素被映射到许多纹理像素上。此时需要对纹理进行缩小过滤（minification filtering）。纹理缩小会由于精度损失造成贴图的扭曲。\n\n如图所示，过度的纹理缩小会造成高频走样问题，贴图出现扭曲（波纹）。\n\n![Alt text](assets/1605006132395.png)\n\n### 为什么称为滤波(Filter)\n\n**图像其实是一种波，可以用波的算法处理图像**\n\n#### 图像是波\n\n每个像素的颜色，可以用红、绿、蓝、透明度四个值描述，大小范围都是0 ～ 255，比如黑色是[0, 0, 0, 255]，白色是[255, 255, 255, 255]。\n\n![Alt text](assets/bg2017121302.png)\n\n每条曲线都在不停的上下波动。有些区域的波动比较小，有些区域突然出现了大幅波动（比如 54 和 324 这两点）。曲线波动较大的地方，也是图像出现突变的地方。这说明波动与图像是紧密关联的。图像本质上就是各种色彩波的叠加。\n\n#### 频率（frequency）\n\n图像就是色彩的波动：波动大，就是色彩急剧变化；波动小，就是色彩平滑过渡。因此，波的各种指标可以用来描述图像。\n频率（frequency）是波动快慢的指标，单位时间内波动次数越多，频率越高，反之越低。\n**色彩剧烈变化的地方，就是图像的高频区域；色彩稳定平滑的地方，就是低频区域。**\n\n#### 滤波器（filter）及在图像中的应用\n\n物理学对波的研究已经非常深入，提出了很多处理波的方法，其中就有滤波器（filter）：过滤掉某些波，保留另一些波。物理中两种常见的滤波器：\n\n * 低通滤波器（lowpass）：减弱或阻隔高频信号，保留低频信号\n\n * 高通滤波器（highpass）：减弱或阻隔低频信号，保留高频信号\n\n应用在图像中：\n**lowpass** 使得图像的高频区域变成低频，即色彩变化剧烈的区域变得平滑，也就是出现 **模糊效果** 。\n\n![Alt text](assets/1603787295513.png)\n\n**highpass** 正好相反，过滤了低频，只保留那些变化最快速最剧烈的区域，也就是图像里面的物体边缘，所以常用于 **边缘识别** 。\n\n![Alt text](assets/1603787306380.png)\n\n### Mipmapping\n\nMipmapping is a standard technique used to **save some of the filtering work** needed during **texture minification**.\n为了加快渲染速度和减少纹理锯齿，贴图被处理成由一系列被预先计算和优化过的图片组成的文件，这样的贴图被称为Mipmap。\n\n#### 使用原因\n\n背景：Textured objects can be viewed at different distances from the viewpoint\n\nProblem：Which level of detail (Resolution) should one use for the texture image?\n\n * Too high resolution: Aliasing effects\n\n * Too small resolution: Too few details visible\n\nSolution: Use different levels of detail according to the distance between object and viewpoint → **mipmaps**\n\n> 名称由来：mip = “multum in parvo”，拉丁语。\n\n在纹理缩小过程中，被贴上纹理的多边形距离很远时，纹理的多个纹素可能只落在一个屏幕像素中。需要读取多个纹素并通过合并他们的值的方式来决定屏幕像素的颜色，这样计算开销很大。Mipmapping会对纹理进行预过滤，并以较小的分辨率进行存储，从而避免运行时进行如此计算的开销。\n\n> 在3D场景中，由于距离问题，物体的纹理需要经常被缩放、拉伸，使用Mipmap会减小运行过程中的计算量，提升GPU效率。而在UI中，一般不需要繁缩放图片（特殊需求除外），因此不建议开启Mipmap，否则会造成内存浪费。\n\n#### 实现方法\n\nSolves LOD problem by generating a pyramid of textures：\n\n * Highest texture resolution at pyramid level 0\n\n * Halfed Resolution at each subsequent level\n\n预先生成一系列以2为倍数缩小（长宽）的纹理序列，在采样纹理时根据图形的大小自动选择相近等级的Mipmap进行采样。\n\n> As the textured surface moves farther away, the texture being applied switches to the prefiltered smaller size. Different sizes of the mipmap are referred to as ‘levels’, with Level 0 being the largest size (used closest to the viewer), and increasing levels used at increasing distances.\n\n所需空间：(1 + 1/3) times the space\n\n![Alt text](assets/MemoryOfMipmap.png)\n\nOpenGL automatically determines the mipmap level to use based on the projected size of the object.\n\n> Mipmap通过下采样（downsampling）原始纹理生成。具体算法有：Nearest Neighbour, Boxfilter, Gaussian filter等\n\n#### 如何选择Mipmap Level\n\n像素着色器是以像素为单位运行的，采样时该如何得知图形的大小呢？\n现在的GPU在光栅化后不是逐像素单个渲染，而是分成2x2=4个的像素分块，每个线程处理4个。之所以这样处理是为了保存像素的偏导数信息，在计算mipmap等级和面法线等时候很有用。OpenGL提供了（唯一）一组获取相邻像素信息的函数——偏导函数dFdx和dFdy。偏导数代表了函数在某一方向的变化率，那么如果相邻两个像素间纹理坐标变化很大，则说明绘制的图形很小。\n如下，计算出纹理坐标在纵向和横向的偏导数（并取最大值）来计算Mipmap级别：\n\n```cpp\nstring s = String.Empty;\n\n```\n\nMipmapLOD计算详解可见：\n\n#### 优势\n\nMipmap除了能消除采样率过低带来的失真问题，还有一个重要的优点是 **节约显存带宽** ，注意是带宽而不是容量。Mipmap实际消耗的显存大约增加了1/3，但每次仅从需要的mipmap级别进行读取，而不必每次都访问原始大小的纹理，因此可以节约带宽。\n\n#### 缺点\n\n 1. Mipmap贴图文件占用更多硬盘空间 – 解决方法：使用dds进行压缩存储\n\n 2. Mipmap纹理占用更多内存、显存空间\n\n## 常用滤波方法（插值算法）\n\n不同的过滤模式，计算复杂度不一样，会得到不同的效果。\n\n### 最近邻插值采样(Nearest-neighbor interpolation)\n\n> OpenGL中对应的滤波方式为GL_NEAREST，Unity中对应的为Point。\n> Unity官方文档描述: Point 单点插值，纹理将变得块状化（blocky up close）\n\n**Nearest-neighbor interpolation** (also known as **proximal interpolation** or, **point sampling** )\n临近返回采样点所落在的像素内，即中心距离采样点曼哈顿距离最近的texel。\n\nThe nearest neighbor algorithm selects the value of the nearest point and does not consider the values of neighboring points at all, yielding a piecewise-constant interpolant.\n\n![Alt text](assets/1603790161075.png)\n\n#### 优势\n\n临近采样的结果始终是纹理中的颜色，在做一些像素风格游戏的时候采用临近。临近通常带来锋利的结果（不是锯齿），颜色之间没有过渡。\n\n当纹理的大小与贴图的三维图形的大小差不多时，这种方法非常有效和快捷。\n\n#### 不足\n\n##### 块状（blocky ）\n\n不同的UV可能会得到相同的颜色，在放大观察时(Magify状态)会有明显色块(马赛克）。当一个texel对应多个pixel时，所有落在这个texel的pixel都会得到同样的颜色值，在屏幕上就会产生明显的像素锯齿，如果是3D游戏的话，就会显得很假。\n\n> 双线性插值解决了texel magnify时像素锯齿的问题，能根据采样点得到不同的颜色值，增加像素细节，只要不是一个uv，就会根据插值得到不同的结果。\n\n##### 高频走样（波纹和闪烁现象）\n\n当minify时，一个pixel”覆盖”多个texel，相邻的两个pixel取的texel太远，在纹理图上可能是两个完全不相邻、完全没有关系的像素。对于高频图像，大量信息被压缩在相对比较小的空间中，则像素变化微小时，色彩变化会剧烈，这个问题在物理上叫做高频走样。在绘制地板和远处物体时会由于这个原因，导致 **波纹（扭曲）和闪烁（sparkling）** 。\n\n在看向地板远端的时候，会发现采样的结果并不连续，而是扭曲的波纹，移动摄像机后，会出现闪烁的问题，看起来非常糟糕。这是由于纹理在远处采样时pixel和它相邻的pixel跨越的texel太长，使在纹理中不相邻的两个texel（或者它们双线性插值的结果）被绘制在了一起。\n\n * 波纹是由于屏幕像素有限，原本不相邻的像素被绘制在一起导致的\n\n * 闪烁是由于图像的高频，采样的低频，轻微偏移就会产生巨大变化导致的。\n\n##### 性能上\n\n放大的情况下非常快，但是在缩小时开销却极高，因为屏幕上相邻的像素点，可能对应于纹理上距离很远的两个点，而这会破坏纹理采样时的内存连续性，导致L1或者L2缓存的命中率极低，使得纹理采样性能大大降低。\n\n#### 示例\n\n![Alt text](assets/1603854909317.png)\n\n如图所示，近处纹理放大过滤，造成了明显的马赛克现象。远处出现闪烁。\n\n### 最近点插值+mipmap的方式\n\n这种方式在最近点插值的基础上，引入了mipmap，当距离相机很近时，使用miplevel0，此时和最近点插值完全一样。当距离下相机很远时，使用更高的miplevel等级，此时使用最近点插值采样更小尺寸的纹理。因此可以缓解缩小时的闪烁和锯齿现象，并且能够充分利用纹理采样时的内存连续性，使得纹理采样性能提高。但是在纹理放大时，不能解决产生的色块现象。\n\n![Alt text](assets/1603855058444.png)\n\n加了mipmap之后可以看到噪点消失，但是远处纹理明显模糊，而且不同mipmap过渡处有明显分界，而近处的纹理仍然有色块现象。\n\n### 双线性插值 (Bilinear interpolation)\n\n> OpenGL对应的设置为GL_LINEAR，unity对应的设置为bilinear。\n> Unity 官方文档描述：Bilinear 双线性插值，纹理将变得模糊（blurry up close）；\n\n**Bilinear interpolation** is also called **bilinear filtering** or **bilinear texture mapping**.\n在数学上，双线性插值是对线性插值在二维直角网格上的扩展\nBilinear interpolation is performed using linear interpolation first in one direction, and then again in the other direction. Although each step is linear in the sampled values and in the position, the interpolation as a whole is not linear but rather quadratic in the sample location.\n首先在 X 轴方向插值两次，对得到的两个插值结果再在 Y 轴方向上进行一次插值。\n\n![Alt text](assets/1603810157411.png)\n\n![Alt text](assets/1603857020919.png)\n\n![Alt text](assets/1603857053404.png)\n\n图像的双线性插值放大算法中，目标图像中新创造的象素值，是由源图像位置在它附近的2*2区域4个邻近象素的值通过加权平均计算得出的。\n\n双线性过滤对于锯齿问题会有一个很明显的提升。该方法中，采样目标点附近的4个纹素，并且根据权重(距离中心点的距离)进行加权平均。这种方法使得放大纹理时的色块现象得以消失，因为此时两个相邻像素之间是平滑过度的。当缩小纹理时可以结合mipmap进行使用，尽管当缩小很多时，依然会有和最近点过滤方法一样的锯齿和闪烁现象，但是对于大部分合理的缩小比例，可以作为一种开销较少的有硬件加速的纹理超采样方案。\n\n![Alt text](assets/1603855288637.png)\n\n近处纹理放大导致的马赛克现象明显消失，但是不同mipmap等级间的分界依然明显\n\n### 三线性插值 (Trilinear interpolation)\n\nTrilinear interpolation is a method of multivariate interpolation on a 3-dimensional regular grid.\n对两个相邻的mipmap等级的纹理进行双线性过滤采样，并对两个采样结果线性插值得到最终的颜色。\n三线性过滤是对双线性过滤中当纹理距离相机的距离刚好处于两个mipmap等级的交界处时的明显的一个过渡现象的解决方案。\n通过对两个相邻的mipmap等级的纹理进行双线性过滤采样，并对两个采样结果线性插值得到最终的颜色。这样当纹理到相机的距离逐渐增加时，可以得到平滑的一个过渡，而不是突兀的变化。当然，对于足够近的纹理，因为使用miplevel0这个等级，因此和双线性过滤完全一致。\n\n经过mipmap+三线性插值，解决了\n\n * 点采样法引起的像素值不连续\n\n * 远景的高频走样\n\n * 不同lod level区域的明显分界\n\n * 切换lod level的突变\n\n![Alt text](assets/1603855360374.png)\n\n#### 不足\n\n然而mipmap自身定义上就导致严重的问题。为了解决高频走样，引入了mipmap，强行将高频区域变为低频区域，减少信息量。这样的后果就是随着lod越来越远，采样结果就越模糊。在游戏中，往往表现为感觉越往远看，地面上就越感觉蒙着一层雾，像眼睛近视了一样。\n\n为了增强远景的清晰度，显卡厂商增加了一个新的过程，即各向异性过滤。这个效果对GPU的要求较高，所以在游戏中属于“最高画质”的选项。\n\n### 各向异性过滤(Anisotropic filtering)\n\n#### 概念\n\nIn 3D computer graphics, anisotropic filtering (abbreviated AF) is a method of enhancing the image quality of textures on surfaces of computer graphics that are at oblique viewing angles with respect to the camera where the projection of the texture (not the polygon or other primitive on which it is rendered) appears to be non-orthogonal (thus the origin of the word: “an” for not, “iso” for same, and “tropic” from tropism, relating to direction; anisotropic filtering does not filter the same in every direction). ( _via. Wikipedia, Anisotropic filtering_ )\n\nLike bilinear and trilinear filtering, anisotropic filtering eliminates aliasing effects, but improves on these other techniques by reducing blur and preserving detail at extreme viewing angles. _(via. Wikipedia, Anisotropic filtering)_\n\n![Alt text](assets/1609139946633.png)\n\nAn example of anisotropic mipmap image storage*(via. Wikipedia)*\n\n#### 原理解释\n\n当一个物体的表面和相机有很大的夹角时，纹理在屏幕上的对应填充区域就不是方形的。例如一个地板，距离相机远的地方，填充区域的宽高是不对等的，此时方形的纹理贴图就不是很合适了，此时就会导致模糊或者闪烁或者两者皆有。各向异性过滤通过采样一个非方形纹理解决了这个问题。\n\nIf we need to texture a horizontal plane which is at an oblique angle to the camera, traditional MIP map minification would give us insufficient horizontal resolution due to the reduction of image frequency in the vertical axis. This is because in MIP mapping each MIP level is isotropic, so a 256 × 256 texture is downsized to a 128 × 128 image, then a 64 × 64 image and so on, so resolution halves on each axis simultaneously, so a MIP map texture probe to an image will always sample an image that is of equal frequency in each axis. Thus, when sampling to avoid aliasing on a high-frequency axis, the other texture axes will be similarly downsampled and therefore potentially blurred. ( _via. Wikipedia, Anisotropic filtering_ )\n\nWith MIP map anisotropic filtering, in addition to downsampling to 128 × 128, images are also sampled to 256 × 128 and 32 × 128 etc. These anisotropically downsampled images can be probed when the texture-mapped image frequency is different for each texture axis. Therefore, one axis need not blur due to the screen frequency of another axis, and aliasing is still avoided. ( _via. Wikipedia, Anisotropic filtering_ )\n\n在Unity中通过Aniso Level设置该过滤效果。\n\n> Texture.anisoLevel - Anisotropic filtering level of the texture. Anisotropic filtering makes textures look better when viewed at a shallow angle, but comes at a performance cost in the graphics hardware. Usually you use it on floor, ground or road textures to make them look better.\n> The value range of this variable goes from 1 to 9, where 1 equals no filtering applied and 9 equals full filtering applied. As the value gets bigger, the texture is clearer at shallow angles. Lower values mean the texture will be more blurry at shallow angles.\n\nAniso Level == 0\n\n![Alt text](assets/1603855799619.png)\n\nAniso Level == 9\n\n![Alt text](assets/1603855820036.png)\n\n## OpenGL对Minification/Magnification Filter的选择\n\n此处摘录自CHAI’S BLOG：\nOpenGL允许设置纹理的过滤，分为GL_TEXTURE_MIN_FILTER和GL_TEXTURE_MAG_FILTER两种。\n如果采样点的像素被认为是Minify了的，则用GL_TEXTURE_MIN_FILTER，如果是为认为是Magnify了的，用GL_TEXTURE_MAG_FILTER，决定用哪一个和mipmap无关，mipmap的三线性过滤通过_MIPMAP_NEAREAST和_MIPMAP_LINEAR决定。\n默认情况下，GL_TEXTURE_MAG_FILTER 的过滤方式是GL_LINEAR，GL_TEXTURE_MIN_FILTER 的过滤方式是GL_NEAREST_MIPMAP_LINEAR。\n\nopengl提供了dfdx,dfdy两个函数对片段的属性做屏幕空间的偏微分，如果对uv做偏微分的话就是:\n![Alt text](assets/1603865152983.png)\n\n∂u/∂x和∂v/∂x是左下角的像素和它水平方向的邻居的差，∂u/∂y和∂v/∂y是左下角的像素和它上面的邻居的差，由于∂x=1，∂y=1，则\ndfdx(uv) = P1.uv - P0.uv\ndfdy(uv) = P3.uv - P0.uv\n\n使用以下公式判断此纹素是min还是mag\n\n![Alt text](assets/1603864791127.png)\n\n如果ρ<=1，此纹素被放大了，即mag\n如果ρ>1，此纹素被缩小了，即min\n\n## 拓展知识\n\n### 纹理环绕（Texture Wrap）\n\n纹理环绕的作用是为了处理超出0.0~1.0范围的纹理坐标，例如采用重复（REPEAT）的环绕方式，采样（4.5,-4.5）的纹理坐标，实际采样的纹理坐标应是（0.5，0.5）。对于负数纹理坐标，采样的实际位置应是1-uv，例如（-0.2,-0.6）应该采样（0.8,0.4）\n\n### 上/下采样\n\n#### downsample（下采样）\n\n目的：\n\n * 使得图像符合显示区域的大小；\n\n * 生成对应图像的缩略图；\n\n方法：对于一幅图像尺寸为M*N，对其进行s倍的下采样，即得到（M/s）*（N/s）尺寸的分辨率图像，当然，s应该是M和N的公约数才可以，如果考虑是矩阵形式的图像，就是把原始图像s*s窗口内的图像变成一个像素，这个像素点就是窗口内所有像素的均值 Pk = (∑ Xi)/ S^2\n\n#### upsample（上采样）\n\n放大图像（或称为上采样（upsampling）或图像插值（interpolating））的主要目的是放大原图像，从而可以显示在更高分辨率的显示设备上。\n\n上采样的原理：图像放大几乎都是采用内插值方法，即在原有图像像素的基础上在像素点之间采用合适的插值算法插入新的元素。插值算法还包括了传统插值，基于边缘图像的插值，还有基于区域的图像插值。\n\n## Ref\n\n\n\n\n\n\n\n\n\n纹理过滤及采样理论：\n" }