{"id":19889,"date":"2026-09-15T08:55:33","date_gmt":"2026-09-15T08:55:33","guid":{"rendered":"http:\/\/www.max-sperling.bplaced.net\/?p=19889"},"modified":"2026-09-28T10:05:45","modified_gmt":"2026-09-28T10:05:45","slug":"llm-basics","status":"publish","type":"post","link":"http:\/\/www.max-sperling.bplaced.net\/?p=19889","title":{"rendered":"LLM (Basics)"},"content":{"rendered":"

This post shall give a very basic view on LLMs and therefore can be a little bit off the actual behavior.<\/p>\n

\n

Creation<\/strong><\/p>\n

    \n
  1. Data crawling: The crawler searches the internet and extracts the actual text from the web pages.<\/li>\n
  2. Tokenization: The tokenizer translates the text into tokens, represented by unique token IDs. 1<\/sup><\/li>\n
  3. Network pre-training: The training happens in a loop over the tokenized training data. 2, 3<\/sup>\n
      \n
    1. The first\/next token (sequence) from the training data gets fed into the network.<\/li>\n
    2. The network returns a probability distribution over the vocabulary, indicating how likely each possible next token is.<\/li>\n
    3. The network’s weights are adjusted based on the difference between its prediction and the actual next token.<\/li>\n<\/ol>\n<\/li>\n
    4. Network post-training: Trains the model with specific user-defined datasets to improve its behavior.<\/li>\n<\/ol>\n

      1<\/sup> You want to try it out? Here is a tool showing you tokenizers: https:\/\/tiktokenizer.com\/<\/a>.
      \n2<\/sup> The ANN in an LLM is a Transformer, which contains of more than just FFNNs\/MLPs.
      \n3<\/sup> The LLM will actually be trained, processing multiple token sequences in parallel.<\/p>\n

      \n

      Usage<\/strong><\/p>\n

        \n
      1. Prompting: You are entering text for the LLM in the provided input field via the GUI. 4<\/sup><\/li>\n
      2. Tokenization: The tokenizer processes the prompt in the same way as the training data.<\/li>\n
      3. Inference: It’s like the network training process, but without the actual training.\n
          \n
        1. The model receives the prompt and the tokens already generated for the response.<\/li>\n
        2. The model calculates the probability distribution and returns the next token based on it.<\/li>\n<\/ol>\n<\/li>\n<\/ol>\n

          4<\/sup> The actual input for the LLM is called context, which contains more than the prompt from the user.<\/p>\n

          \n

          Summary<\/strong><\/p>\n

          An LLM is a huge ANN
          \n… which has been trained on a lot of tokenized text data
          \n… that responses by repeatedly predicting the next token.<\/p>\n","protected":false},"excerpt":{"rendered":"

          This post shall give a very basic view on LLMs and therefore can be a little bit off the actual<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"inline_featured_image":false},"categories":[30],"tags":[],"_links":{"self":[{"href":"http:\/\/www.max-sperling.bplaced.net\/index.php?rest_route=\/wp\/v2\/posts\/19889"}],"collection":[{"href":"http:\/\/www.max-sperling.bplaced.net\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/www.max-sperling.bplaced.net\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/www.max-sperling.bplaced.net\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"http:\/\/www.max-sperling.bplaced.net\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=19889"}],"version-history":[{"count":66,"href":"http:\/\/www.max-sperling.bplaced.net\/index.php?rest_route=\/wp\/v2\/posts\/19889\/revisions"}],"predecessor-version":[{"id":19986,"href":"http:\/\/www.max-sperling.bplaced.net\/index.php?rest_route=\/wp\/v2\/posts\/19889\/revisions\/19986"}],"wp:attachment":[{"href":"http:\/\/www.max-sperling.bplaced.net\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=19889"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/www.max-sperling.bplaced.net\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=19889"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/www.max-sperling.bplaced.net\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=19889"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}