https://somnus.top/categories/learning/Recent content in Learning on Somnus的博客Hugo -- gohugo.iozh-cnsomnus0917chen@hotmail.com (Somnus)somnus0917chen@hotmail.com (Somnus)© 2026 SomnusFri, 15 May 2026 22:42:47 +0800https://somnus.top/notes/llm04-position-encoding/Fri, 15 May 2026 22:42:47 +0800somnus0917chen@hotmail.com (Somnus)https://somnus.top/notes/llm04-position-encoding/<h2 class="relative group">位置编码 <div id="位置编码" class="anchor"></div> <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none"> <a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#%e4%bd%8d%e7%bd%ae%e7%bc%96%e7%a0%81" aria-label="锚点">#</a> </span> </h2> <p>卷积具有局部性，天然地会注意元素之间的相对位置，但是基于自注意力的transformer模型则对位置不敏感，因此必须要把元素的位置信息在embedding阶段传给元素。 比如：</p>https://somnus.top/notes/llm03-tokenizer/Sat, 09 May 2026 17:02:18 +0800somnus0917chen@hotmail.com (Somnus)https://somnus.top/notes/llm03-tokenizer/<h2 class="relative group">为什么要有tokenizer <div id="为什么要有tokenizer" class="anchor"></div> <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none"> <a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#%e4%b8%ba%e4%bb%80%e4%b9%88%e8%a6%81%e6%9c%89tokenizer" aria-label="锚点">#</a> </span> </h2> <p>tokenizer的作用是把文本序列转换成数字序列，即token编号，作为transformer的输入。</p>https://somnus.top/notes/llm02-transformer/Fri, 08 May 2026 18:26:47 +0800somnus0917chen@hotmail.com (Somnus)https://somnus.top/notes/llm02-transformer/<h1 class="relative group">transformer <div id="transformer" class="anchor"></div> <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none"> <a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#transformer" aria-label="锚点">#</a> </span> </h1> <p>transformer是在这篇<a href="https://arxiv.org/abs/1706.03762" target="_blank" rel="noreferrer">attention is all you need</a>中提出来的。</p>https://somnus.top/notes/llm01-self-attention/Fri, 08 May 2026 18:23:20 +0800somnus0917chen@hotmail.com (Somnus)https://somnus.top/notes/llm01-self-attention/<h2 class="relative group">attention <div id="attention" class="anchor"></div> <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none"> <a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#attention" aria-label="锚点">#</a> </span> </h2> <p>注意力机制，最早是在机器翻译论文**<a href="https://arxiv.org/abs/1409.0473" target="_blank" rel="noreferrer">Neural Machine Translation by Jointly Learning to Align and Translate</a>**中提出来的，他的核心是，用一个东西产生的query，去key/value中查询需要的东西。在上述论文中encoder-decoder attention的结构中。</p>https://somnus.top/notes/rust%E5%AD%A6%E4%B9%A001-%E6%89%80%E6%9C%89%E6%9D%83/Sat, 04 Apr 2026 22:39:47 +0800somnus0917chen@hotmail.com (Somnus)https://somnus.top/notes/rust%E5%AD%A6%E4%B9%A001-%E6%89%80%E6%9C%89%E6%9D%83/<h2 class="relative group">Trait <div id="trait" class="anchor"></div> <span class="absolute top-0 w-6 transition-opacity opacity-0 -start-6 not-prose group-hover:opacity-100 select-none"> <a class="text-primary-300 dark:text-neutral-700 !no-underline" href="#trait" aria-label="锚点">#</a> </span> </h2> <p>trait（特征）是一种<strong>定义共享行为</strong>的机制，可以理解成其他语言中的接口，相当于是不管你底层是什么数据类型，只要定义了这个trait，就一定要实现这个trait的功能。</p>