Learning on Somnus的博客https://somnus.top/tags/learning/Recent content in Learning on Somnus的博客Hugo -- gohugo.iozh-cnsomnus0917chen@hotmail.com (Somnus)somnus0917chen@hotmail.com (Somnus)© 2026 SomnusFri, 15 May 2026 22:42:47 +0800Llm04 Position Encodinghttps://somnus.top/notes/llm04-position-encoding/Fri, 15 May 2026 22:42:47 +0800somnus0917chen@hotmail.com (Somnus)https://somnus.top/notes/llm04-position-encoding/<h2 class="relative group">位置编码 <div id="位置编码" class="anchor"></div> <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none"> <a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#%e4%bd%8d%e7%bd%ae%e7%bc%96%e7%a0%81" aria-label="锚点">#</a> </span> </h2> <p>卷积具有局部性，天然地会注意元素之间的相对位置，但是基于自注意力的transformer模型则对位置不敏感，因此必须要把元素的位置信息在embedding阶段传给元素。 比如：</p>LLM03 Tokenizerhttps://somnus.top/notes/llm03-tokenizer/Sat, 09 May 2026 17:02:18 +0800somnus0917chen@hotmail.com (Somnus)https://somnus.top/notes/llm03-tokenizer/<h2 class="relative group">为什么要有tokenizer <div id="为什么要有tokenizer" class="anchor"></div> <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none"> <a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#%e4%b8%ba%e4%bb%80%e4%b9%88%e8%a6%81%e6%9c%89tokenizer" aria-label="锚点">#</a> </span> </h2> <p>tokenizer的作用是把文本序列转换成数字序列，即token编号，作为transformer的输入。</p>Eulerproj01https://somnus.top/notes/eulerproj01/Sun, 29 Mar 2026 20:25:05 +0800somnus0917chen@hotmail.com (Somnus)https://somnus.top/notes/eulerproj01/<h3 class="relative group">p002 <div id="p002" class="anchor"></div> <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none"> <a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#p002" aria-label="锚点">#</a> </span> </h3> <h4 class="relative group">题目 <div id="题目" class="anchor"></div> <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none"> <a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#%e9%a2%98%e7%9b%ae" aria-label="锚点">#</a> </span> </h4> <p>Problem 2: Even Fibonacci numbers</p>