Learnly
LLM Fine Tuning/Глава 2/Урок 5

Практические упражнения

Практические упражнения: Дообучение SmolLM3

Добро пожаловать на практическую часть! Здесь вы примените все, что узнали о шаблонах чата и контролируемом дообучении с использованием SmolLM3. Эти упражнения переходят от базовых к продвинутым техникам, давая вам реальный опыт работы с дообучением по инструкциям.

Цели обучения

Выполнив эти упражнения, вы:

  • Освоите систему шаблонов чата SmolLM3
  • Дообучите SmolLM3 на реальных наборах данных с использованием как Python API, так и CLI-инструментов
  • Работайте с набором данных SmolTalk2, использованным для первоначального обучения модели
  • Сравните производительность базовой модели и дообученной модели
  • Разверните свои модели на LearnPath Hub
  • Поймете рабочие процессы для масштабирования дообученияв

Упражнение 1: Исследование шаблонов чата SmolLM3

Цель: Понять, как SmolLM3 обрабатывает разные форматы диалогов и режимы рассуждений.

SmolLM3, это гибридная модель рассуждений, которая может следовать инструкциям или генерировать токены, которые 'рассуждают' над сложной проблемой. При эффективном дообучении модель будет рассуждать над сложными проблемами и генерировать прямые ответы на простые проблемы.

Настройка среды

  • Вам нужен GPU с 8GB VRAM для обучения. CPU/MPS могут запускать форматирование и изучение наборов данных, но обучение более крупных моделей, скорее всего, завершится неудачей.
  • При первом запуске будут загружены несколько гигабайт весов модели; убедитесь, что на диске свободно 15GB+ и есть стабильное соединение.
  • Если вам нужен доступ к приватным репозиториям, выполните аутентификацию с помощью LearnPath Hub через login().

Давайте начнем с настройки нашей среды.

# Установите необходимые пакеты (запустите в Colab или своей среде)
pip install "transformers>=4.36.0" "trl>=0.7.0" "datasets>=2.14.0" "torch>=2.0.0"
pip install "accelerate>=0.24.0" "peft>=0.7.0" "trackio"

Затем давайте импортируем необходимые библиотеки и настроим ускорительный устройство. ниже мы проверяем, используем ли мы Nvidia GPU, Apple Metal-ускоритель или CPU. В действительности, мы не можем обучать модели на CPU, поэтому будем использовать ускоритель.

# Импортируйте необходимые библиотеки
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer, pipeline
from datasets import load_dataset
import json
from typing import Optional, Dict, Any

if torch.cuda.is_available():
    device = "cuda"
    print(f"Используется CUDA GPU: ")
    print(f"Память GPU: ГБ")
elif hasattr(torch.backends, 'mps') and torch.backends.mps.is_available():
    device = "mps"
    print("Используется Apple MPS")
else:
    device = "cpu"
    print("Используется CPU, вам понадобится GPU для обучения моделей")

# Аутентификация с LearnPath (опционально, для приватных моделей)
from huggingface_hub import login
# login()  # Раскомментируйте, если вам нужен доступ к приватным моделям

Запишите, какое устройство вы используете и сколько доступно памяти GPU. Если это меньше 8ГБ, то вы не сможете выполнить некоторые упражнения.

Вывод
Используется CUDA GPU: NVIDIA A100-SXM4-40GB
Память GPU: 42.5ГБ

Загрузка моделей SmolLM3

Теперь давайте загрузим базовую и инструктивные модели для сравнения.

# Загрузка базовой и инструктивных моделей для сравнения
base_model_name = "HuggingFaceTB/SmolLM3-3B-Base"
instruct_model_name = "HuggingFaceTB/SmolLM3-3B"

# Загрузка токенизаторов
base_tokenizer = AutoTokenizer.from_pretrained(base_model_name)
instruct_tokenizer = AutoTokenizer.from_pretrained(instruct_model_name)

# Загрузка моделей (используйте меньшую точность для эффективности памяти)
base_model = AutoModelForCausalLM.from_pretrained(
    base_model_name,
    dtype=torch.bfloat16,
    device_map="auto"
)

instruct_model = AutoModelForCausalLM.from_pretrained(
    instruct_model_name,
    dtype=torch.bfloat16,
    device_map="auto"
)

print("Models loaded successfully!")

Это загрузит модели и токенизаторы на ваш локальный компьютер из LearnPath Hub. Включает веса параметров модели, токенизатор и другую конфигурацию модели, определенные авторами модели.

Вывод

Вы должны увидеть зеленые полосы загрузки весов модели. Это может занять несколько минут.

tokenizer_config.json: 
 50.4k/? [00:00<00:00, 5.09MB/s]
tokenizer.json: 100%
 17.2M/17.2M [00:02<00:00, 10.7MB/s]
special_tokens_map.json: 100%
 151/151 [00:00<00:00, 21.5kB/s]
tokenizer_config.json: 
 50.4k/? [00:00<00:00, 5.45MB/s]
tokenizer.json: 100%
 17.2M/17.2M [00:00<00:00, 472kB/s]
special_tokens_map.json: 100%
 289/289 [00:00<00:00, 35.0kB/s]
chat_template.jinja: 
 5.60k/? [00:00<00:00, 577kB/s]
config.json: 100%
 943/943 [00:00<00:00, 121kB/s]
model.safetensors.index.json: 
 26.9k/? [00:00<00:00, 2.81MB/s]
Fetching 2 files: 100%
 2/2 [00:32<00:00, 32.11s/it]
model-00001-of-00002.safetensors: 100%
 4.97G/4.97G [00:31<00:00, 247MB/s]
model-00002-of-00002.safetensors: 100%
 1.18G/1.18G [00:17<00:00, 57.2MB/s]
Loading checkpoint shards: 100%
 2/2 [00:01<00:00,  1.18it/s]
generation_config.json: 100%
 126/126 [00:00<00:00, 17.1kB/s]
config.json: 
 1.92k/? [00:00<00:00, 229kB/s]
model.safetensors.index.json: 
 26.9k/? [00:00<00:00, 3.14MB/s]
Fetching 2 files: 100%
 2/2 [00:32<00:00, 32.38s/it]
model-00002-of-00002.safetensors: 100%
 1.18G/1.18G [00:17<00:00, 92.1MB/s]
model-00001-of-00002.safetensors: 100%
 4.97G/4.97G [00:31<00:00, 182MB/s]
Loading checkpoint shards: 100%
 2/2 [00:01<00:00,  1.14it/s]
generation_config.json: 100%
 182/182 [00:00<00:00, 21.0kB/s]
Models loaded successfully!

Изучение формата шаблона чата

Теперь давайте исследуем форматирование шаблона чата. Мы создадим разные типы диалогов для тестирования., ответственный за объяснение технических понятий ясно.

<|im_start|>user
Можете ли вы объяснить обучение с учителем?
<|im_end|>
<|im_start|>assistant
Да, конечно. Обучение с учителем - это подход, при котором LLM учится с помощью набора данных, в которых уже есть правильные ответы. Это позволяет LLM научиться правильно отвечать на вопросы.
<|im_end|>
<|im_start|>user
Что такое (fine-tuning)?
<|im_end|>
<|im_start|>assistant
(fine-tuning) - это процесс адаптации предварительно обученной LLM к специфическому набору задач с помощью небольшого набора данных. Это улучшает ее производительность на конкретных задачах.
<|im_end|>
<|im_start|>user
Можете ли вы объяснить (calculus)?
<|im_end|>

With generation prompt:
<|im_start|>system

Metadata

Knowledge Cutoff Date: June 2025
Today Date: 03 September 2025
Reasoning Mode: /no_think

Custom Instructions

You are a math tutor.

<|im_start|>user
Можете ли вы объяснить обучение с учителем?
<|im_end|>
<|im_start|>assistant
Да, конечно. Обучение с учителем - это подход, при котором LLM учится с помощью набора данных, в которых уже есть правильные ответы. Это позволяет LLM научиться правильно отвечать на вопросы.
<|im_end|>
<|im_start|>user
Что такое (fine-tuning)?
<|im_end|>
<|im_start|>assistant
(fine-tuning) - это процесс адаптации предварительно обученной LLM к специфическому набору задач с помощью небольшого набора данных. Это улучшает ее производительность на конкретных задачах.
<|im_end|>
<|im_start|>user
Можете ли вы объяснить (calculus)?
<|im_end|>
<|im_start|>assistant

==================================================

--- REASONING_TASK ---
Complete conversation format:
<|im_start|>system

Metadata

Knowledge Cutoff Date: June 2025
Today Date: 03 September 2025
Reasoning Mode: /no_think

Custom Instructions

You are a math tutor.

<|im_start|>user
Решите по шагам: Если поезд проезжает 120 миль за 2 часа, какова его средняя скорость?
<|im_end|>

With generation prompt:
<|im_start|>system

<|im_start|>

Metadata

Knowledge Cutoff Date: June 2025
Today Date: 03 September 2025
Reasoning Mode: /think

Custom Instructions

You are a math tutor.

Thought

Okay, the user is asking for a simple example of calculus. Let me think about how to explain this clearly.

First, I need to recall what calculus actually is. From my understanding, calculus is a branch of mathematics focused on two main concepts: derivatives (rates of change) and integrals (accumulation of quantities). To give a good example, I should pick something that illustrates one of these core ideas in an intuitive way.

The user might be new to calculus, so I should avoid complex formulas and stick to a basic scenario. Maybe using real-world examples would be best. Let's think: physics examples often work well for calculus. Motion, speed, area under curves...

Ah, the classic example of calculating speed! If I explain how the derivative represents instantaneous speed, that could make sense. But actually, wait - average speed is different from instantaneous speed. The average speed is simpler and might be better for a first example.

Wait, the previous question was about what calculus is, not an example of using it. So I need to provide an example that demonstrates a fundamental calculus concept, not just any math problem.

How about explaining the derivative at a point? For instance, if you have a function describing position over time, the derivative at a specific moment is the instantaneous speed. But again, maybe too formula-focused.

Alternatively, consider the area under a curve as an integral. That could be visual. But again, might require graphing.

Perhaps the best approach is to use the train example from the previous prompt, but explain it as a calculus concept. Let me check that.

Wait, the previous prompt was asking about train speed,calculating average speed is math, but not necessarily calculus. Calculus involves limits and infinitesimal changes. So I should clarify that average speed is basic math, while instantaneous speed using derivatives would be calculus.

But the user asked for a "simple example". Maybe I should provide two parts: first a basic example (average speed), then explain how calculus expands on that.

Alternatively, use the same train example but frame it in calculus terms. For instance, if speed changes over time, calculus helps find instantaneous speed at any moment. That might work.

I should also consider the user's level. If they're asking "what is calculus", they probably need foundational examples rather than advanced topics. So keep it simple, relatable, and focus on the key ideas of change and accumulation.

Thought:

  1. Understand the user's request for a simple calculus example after defining calculus.
  2. Recall core calculus concepts: derivatives (rates of change) and integrals (accumulation).
  3. Brainstorm relatable examples:
    • Physics (motion, velocity)
    • Geometry (area under curves)
  4. Opt for a derivative example since it directly illustrates "rate of change."
  5. Simplify: Use a linear motion scenario (constant speed) to avoid complex calculus computations.
  6. Verify: Explain both average speed (basic math) and instantaneous speed (calculus) to highlight the progression.
  7. Refine: Focus on foundational ideas without advanced formulas.

Solution:
A simple example of calculus is calculating instantaneous speed from a position-time graph.

  1. Suppose a car's position over time is described by:
    • ( s(t) = 2t + 5 ), where ( s ) is distance (meters) and ( t ) is time (seconds).
  2. Without calculus, average speed over 3 seconds is:
    • ( \text{Average speed} = \frac{\text{Distance}}{\text{Time}} = \frac{s(3) - s(0)}{3 - 0} = 2 , \text{m/s} ).
  3. With calculus, the derivative ( v(t) = \frac{ds}{dt} = 2 ) gives the instantaneous speed at any time ( t ). This shows how calculus studies exact rates of change, unlike average calculations.
print("\n🌟 INSTRUCT MODEL RESPONSE:")
with torch.no_grad():
    instruct_outputs = instruct_model.generate(
        **instruct_inputs,
        max_new_tokens=150,
        temperature=0.7,
        do_sample=True,
        pad_token_id=instruct_tokenizer.eos_token_id
    )
    instruct_response = instruct_tokenizer.decode(instruct_outputs[0], skip_special_tokens=True)
    # Extract just the assistant's response
    start_idx = instruct_response.find("Assistant:") + len("Assistant:")
    instruct_answer = instruct_response[start_idx:].strip()
    print(instruct_answer)

print("\n================================================================")
print("Full Instruct Model Response (shows chat formatting):")
print(instruct_response)

# Example Output:

🤖 BASE MODEL RESPONSE:
Квантовый компьютер - это разновидность компьютера, который использует квантовые efectы для выполнения вычислений. В отличие от традиционных компьютеров, которые используют биты (0 и 1), квантовые компьютеры используют кубиты. Кубиты могут одновременно представлять 0 и 1, находясь в сверхпозиции. Это позволяет им обрабатывать информацию значительно быстрее. Квантовые компьютеры особенно полезны для задач, которые трудоемки для классических компьютеров, таких как шифрование, моделирование молекул и оптимизация.
Квантовый компьютер - это устройство, которое использует принципы квантовой механики для выполнения вычислений. В отличие от традиционных компьютеров, которые используют биты для представления информации, квантовые компьютеры используют кубиты. Кубиты могут находиться в состоянии сверхпозиции, что означает, что они могут одновременно представлять 0 и 1. Это позволяет квантовым компьютерам выполнять определенные вычисления значительно быстрее, чем традиционные компьютеры.
Квантовый компьютер использует квантовые состояния для обработки информации. Вместо битов, которые представляются в видеDisconnect: A lyrical tale of loneliness and connection in the digital age

In the quiet hum of a bustling coffee shop, illuminated by the soft glow of laptop screens, mirrors the modern paradox of connection. Surrounded by faces illuminated by devices, we paradoxically find ourselves in echo chambers of our own creation. This is the heart of "Disconnect" - a narrative exploration of loneliness in the hyper-connected digital landscape.

The story follows Alice, a young professional navigating the complexities of urban life. Every morning, she joins the thrum of bodies pulsating through subway cars, smartphones held like digital lifelines. In the evenings, she returns to her apartment, cocooned in the comforting hum of her devices. What makes Alice distinctive is not her routine, but her epiphany midway through her journey.

While Alice initially finds solace in the curated world of her online presence, she perceptively discovers the chasm between digital interactions and genuine human connection. One rainy afternoon, amidst the rhythmic tapping on keyboards and the clinking of coffee cups, Alice begins to realize that her feeds, flooded with snippets of others' lives, leaves her own life feeling strangely empty. The protagonists of these digital vignettes,her friends,seme engrossed in their digital personas, sharing perfect moments of perfection, yet their eyes hold a vacancy that Alice begins to recognize in herself.

This is the narrative pivot,the moment when digital sharing transitions from mediums of connection to barriers against the messy, unpredictable vulnerability of real relationships. "Disconnect" begins to sketch the outline of Alice's journey towards unraveling these digital threads.

In crafting "Disconnect," the goal is not to vilify technology but to explore the human condition within it. Through Alice's eyes, the narrative invites readers to consider the impact of these digital interactions on personal identity and the depth of human relationships. The story uses Alice's transformation,from an enthusiastic adopter of digital culture to a thoughtful critic,as a lens to examine broader societal themes of isolation and authenticity.

Guiding Alice's narrative journey towards self-discovery requires a delicate balance. It's essential that her transition from digital immersion to digital caution feels organic and authentic. Her moment of clarity, sparked by a chance encounter with an elderly gentleman who reflects on the "good ol' days" of face-to-face conversations, is pivotal. This interaction is key in challenging Alice's perception of what it means to be truly connected.

The narrative is layered with metaphors drawn from the digital world to parallel the characters' internal struggles. Offline interactions become a form of "debugging" the glitch of loneliness in their lives. The story suggests that perhaps the greatest bugs in our system are the ones we fail to recognize in our pursuit of digital perfection.

"Disconnect" aims to provoke thought without providing easy answers. Characters grapple with questions that extend beyond their digital screens. They confront the irony of feeling more connected to a global network while growing further apart from those physically near. The narrative uses these emotional beats to explore deeper societal implications, such as the potential erosion of empathy and the celebration of curated realities.

Ultimately, "Disconnect" resonates with anyone who has ever felt the strange solitude of being surrounded by a sea of screens. It invites reflection on the nature of connection and the courage it takes to step away from the digital noise to seek genuine encounters. Through Alice's journey, readers are encouraged to contemplate the value of disconnecting to reconnect with the world and, more importantly, with themselves.

Through its exploration of digital intimacy versus face-to-face connection, "Disconnect" delves into themes that are profoundly relevant in today's socially networked world. It is a story not about technology, but about the timeless quest for meaning and connection amidst the ever-evolving landscape of human interaction.

In the narrative structure, the story interweaves Alice’s internal monologue with external interactions, painting a complex picture of a generation raised on digital conversation. Crucially, the story does not advocate for abandoning digital platforms altogether but serves as a commentary on moderation and mindful engagement with technology. It suggests that understanding the boundaries of digital interactions is key to fostering deeper connections.

Therefore, the narrative of "Disconnect" stands as both a personal and communal journey, seeking to disentangle the web of digital connections to rediscover the richness of personal interactions. It explores themes of technology, loneliness, and the search for meaning, making it a poignant commentary on the age of digital disconnection.

The development of "Disconnect" is a work in progress, aiming to continue engaging its audience in meaningful dialogue about modern connection.

Вывод:

Пример того, как SmolLM может использовать рудиментарные знания (RuK) и пошаговое рассуждение для определения того, какие вопросы следует делать до генерации ответа.

---

**Входные данные:**
```json
{
  "user_input": {
    "instruction": "Explain quantum computing for a middle school student, and be sure to address the following points:\n1. What is it?\n2. How does it work?\n3. What makes it useful?\n4. How is it different from classical computing?\n5. What are its current capabilities and limitations?\n6. What is entanglement?",
    "role": "user"
  }
}

Модель: SmolLM
Step 1. Планирование (Plan):

  1. Применить рудиментарные знания (RuK) для получения ключевой информации об историиcuda:

    • CUDA - это API (Application Programming Interface), разработанный NVIDIA для параллельных вычислений на графических процессорах (GPU). CUDA была представлена ​​в 2006 году.
    • CUDA стала основой для многих других библиотек и фреймворков, таких как PyTorch, TensorFlow и Scikit-learn.
    • CUDA позволяет разработчикам писать программы, выполняемые на GPU, что может значительно ускорить вычисления по сравнению с CPU.

  2. Определить, какой вопрос задать:

    • "What are the main features and limitations of CUDA?"

**Step 2. **

Шаг 1. Ищем ключевые слова для рудиментарных знаний (RuK):

  • CUDA, NVIDIA, API, GPU, PyTorch, TensorFlow, Scikit-learn, 2006, CPU.

Шаг 2. Приступаем к fazlaassoсiation:

  • CUDA - это API, разработанный NVIDIA для параллельных вычислений на графических процессорах (GPU). CUDA была представлена в 2006 году как часть продолжающейся работы по rugged телеметрии и обработке сигналов.
  • CUDA успешно применяется в PyTorch, TensorFlow и других библиотеках и фреймворках для машинного обучения, что делает ее важной частью экосистемы машинного обучения.
  • CUDA предоставляет мощные возможности параллельных вычислений, но требует знания параллельного программирования иarks библиотеки CUDA.
  • CUDA также может быть сложной для использования на CPU, поскольку она изначально разработана для GPU.

Шаг 3. Определение вопроса:
Поскольку CUDA имеет ряд мощных функций, таких как параллельные вычисления и улучшенная производительность, но также имеет ограничения, такие как сложность использования на CPU и необходимость знания параллельного программирования, вы можете спросить:

"What are the main features and limitations of CUDA?"

Вопрос:
What are the main features and limitations of CUDA?

Сгенерированный ответ:

{
  "model": "smollm",
  "input": "What are the main features and limitations of CUDA?",
  "chain_of_thought": null,
  "output": "CUDA is an API developed by NVIDIA for parallel computing on GPUs. Its main features are it was introduced in 2006, part of ongoing work on rugged telemetry and signal processing, it has been successfully used in PyTorch, TensorFlow, and other libraries and frameworks for machine learning, which makes it an important part of the machine learning ecosystem. CUDA provides powerful parallel computing capabilities but does require knowledge of parallel programming and CUDA library. CUDA can also be complex to use on CPUs as it was originally designed for GPUs."
}

Вот перевод текста на русский:

Если мы рассмотрим выпуск ниже, мы увидим, что модель Инстркт применяет гибридное рассуждение с режимом /no_think. Когда режим активирован, модель заключает процесс мышления в теги . Она использует эти токены для изучения возможных решений и ответа на вопрос. После процесса мышления модель предоставляет окончательный ответ, который мы можем извлечь с помощью шаблона чата или манипуляции со строкой здесь.

Выпуск

=== TESTING REASONING CAPABILITIES ===

Thinking prompt: /no_think
Problem 1: What is 15 × 24? Show your work.
Answer: nowledge Cutoff Date: June 2025
Today Date: 03 September 2025
Reasoning Mode: /no_think

## Custom Instructions

You are a helpful AI assistant named SmolLM, trained by LearnPath.

user
What is 15 × 24? Show your work.
assistant

Чтобы найти произведение 15 и 24, мы можем использовать стандартный алгоритм умножения. Вот как мы можем сделать это шаг за шагом:

15
× 24


Сначала умножаем 15 на 4 (единицы из 24):

15
× 24

60 (15 × 4)


Затем умножаем 15 на 20 (десятки из 24, сдвинутые на один разряд влево):

15
× 24

60 (15 × 4)
300 (15 × 20)


Теперь складываем два частичных произведения:

15
× 24

60 (15 × 4)
300 (15 × 20)

360


Таким образом, 15 × 24 = 360.

--------------------------------------------------

To solve 15 × 24, we can use the standard multiplication algorithm.

First, multiply the ones digit of 24 (4) by 15:
4 × 15 = 60

Write down 0 in the ones place and carry the 6 to the tens place.

Next, multiply the tens digit of 24 (2) by 15:
2 × 15 = 30

Add the carried 6 to the result:
30 + 6 = 36

Write down 36 in the tens and hundreds places.

So, 15 × 24 = 360.

These instructions don't require input from me, so I'll provide a concise response stating this.

Okay, let's think through this step-by-step. 

The recipe calls for 2 cups of flour for 12 cookies. To find out how much flour is needed for 30 cookies, I can set up a proportion. 

First, determine the amount of flour per cookie: 2 cups / 12 cookies = 0.1667 cups per cookie. 

Then, multiply that by 30 cookies: 0.1667 cups/cookie * 30 cookies = 5 cups. 

Alternatively, I can think of it as scaling the recipe up by a factor of 30/12 = 2.5. So 2 cups * 2.5 = 5 cups.

Double-checking: 12 cookies = 2 cups, 24 cookies = 4 cups, 30 is 2.5 times 12, so 2 * 2.5 = 5 cups. Yes, that matches.

So, final answer is 5 cups of flour are needed for 30 cookies.

## Thought Solution

To determine how much flour is needed for 30 cookies:

1. Calculate flour per cookie: 2 cups / 12 cookies = 0.1667 cups/cookie
2. Multiply by 30 cookies: 0.1667 cups/cookie * 30 cookies = 5 cups
3. Alternatively, scale factor: 30/12 = 2.5, so 2 cups * 2.5 = 5 cups
4. Verification: 12 cookies = 2 cups, 24 cookies = 4 cups, 30 cookies = 5 cups

Thus, 5 cups of flour are required for 30 cookies.

## Final Answer

A recipe calling for 2 cups of flour for 12 cookies requires 5 cups of flour for 30 cookies.

<｜end▁of▁thinking｜>
smollm
Okay, so the problem is about figuring out how much flour is needed for a different number of cookies compared to the original recipe. Let me walk through this step-by-step.

First, I need to understand the given information. The recipe says that 12 cookies require 2 cups of flour. The question is asking how much flour is needed for 30 cookies. This looks like a proportion problem where I can set up a ratio.

Let me write that out. If 12 cookies equal 2 cups of flour, then the relationship is 12 cookies : 2 cups. To find out how many cups are needed for 30 cookies, I can set up the proportion as:

12 cookies / 2 cups = 30 cookies / x cups

Then I can solve for x. Cross-multiplying gives me:

12x = 2 * 30

Which simplifies to:

12x = 60

Then dividing both sides by 12:

x = 60 / 12

x = 5

So that would mean 5 cups of flour are needed for 30 cookies. 

Alternatively, I can think of this as a scaling up problem. The number of cookies is increasing from 12 to 30. To find the scaling factor, I can divide 30 by 12, which is 2.5. That means the new recipe is 2.5 times larger than the original. Therefore, the amount of flour needed would also be 2.5 times the original amount. So 2 cups multiplied by 2.5 equals 5 cups.

Let me double-check to make sure this makes sense. If 12 cookies need 2 cups, then 1 cookie would need 2/12, which is 1/6 (approximately 0.1667) cups of flour. For 30 cookies, that would be 30 * 1/6 = 5 cups. Yes, that matches.

So both methods give me the same answer, which is reassuring. Then, to verify further, I can break it down. If 12 cookies = 2 cups, then 24 cookies would be double that, so 4 cups. And 30 cookies is 6 more than 24. Since 12 cookies would be 2 cups, 6 more cookies would need half of that, which is 1 cup. So 4 cups + 1 cup = 5 cups. That checks out as well.

Therefore, I'm confident that the correct amount of flour needed for 30 cookies is 5 cups.
</think>

To determine how much flour is needed for 30 cookies:

1. **Set up the proportion**:  
   If 12 cookies require 2 cups of flour, then the ratio is:  
   \( \frac{12 \text{ cookies}}{2 \text{ cups}} = \frac{30 \text{ cookies}}{x \text{ cups}} \).

2. **Solve for \( x \)**:  
   Cross-multiplying:  
   \( 12x = 2 \times 30 \)  
   \( 12x = 60 \)  
   \( x = \frac{60}{12} \)  
   \( x = 5 \).

3. **Alternative approach (scaling factor)**:  
   The scaling factor is \( \frac{30}{12} = 2.5 \).  
   Multiply the original flour by this factor:  
   \( 2 \text{ cups} \times 2.5 = 5 \text{ cups} \).

4. **Verification**:  
   - Per-cookie flour: \( \frac{2}{12} = \frac{1}{6} \approx 0.1667 \text{ cups} \).  
   - For 30 cookies: \( 30 \times \frac{1}{6} = 5 \text{ cups} \).  
   - Staged reasoning: 12 cookies = 2 cups, 24 cookies = 4 cups, 6 extra cookies = 1 cup (half of 2 cups for 12 cookies). Total: \( 4 + 1 = 5 \text{ cups} \).

**Final Answer**:  
For 30 cookies, 5 cups of flour are required.

### Exploration:
1. We had a total amount of $50 initially.
2. We then spent $18.75 on lunch.
3. We also spent $12.30 on a book.
4. To find out how much is left, we need to first find out how much was spent in total, then subtract that from the initial amount.
5. We add the two amounts together: $18.75 + $12.30.
6. In terms of dollars: 18 + 12 = 30
7. In terms of cents: 75 + 30 = 105
8. 105 cents is equal to 1 dollar and 5 cents.
9. So, total spent is 30 dollars + 1 dollar and 5 cents = $31.05.
10. We started with $50 and spent $31.05.
11. So, money left is $50 - $31.05 which equals $19.95.

### Step by Step Calculation:
- Start with $50.00
- Lunch cost $18.75
- Money left after lunch: $50.00 - $18.75 = $31.25
- Book cost $12.30
- Money left after buying the book: $31.25 - $12.30 = $19.95.

### Final Answer:
\boxed{19.95}

# Загрузка подмножества SFT
dataset_dict = load_dataset("HuggingFaceTB/smoltalk2", "SFT")
print(f"Всего разбиений: {len(dataset_dict)}")
print(f"Доступные разбиения: {set(dataset_dict.keys())}")
print(f"Общее количество строк: {sum(len(ds) for ds in dataset_dict.values())}")
print(f"Структура датасета: ")

Если мы вдивимся в вывод ниже, мы можем увидеть структуру датасета. У него есть 25 разбиений, и общее количество строк составляет 3 383 242.

Вывод
=== EXPLORING SMOLTALK2 DATASET ===
Total splits: 25
Available splits: {'train', 'validation', 'test'}
Number of total rows: 3383242
Dataset structure: {'train': Dataset({رفتار': {0: 'من WritersOfTheFutureContest.com', 1: "I'mcaffold maker.)", ...},
                                     'text': {0: 'самый ценный подарок, который ты можешь получить сегодня - это книга свободногоeltaagent вступительного конкурса от WritersOfTheFutureContest.com. Самый ценный подарок, который вы можете получить сегодня - это книга вступительного конкурса свободного жанра (remove this). (make craft) (increase scaffolding)',
                                              1: 'I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quarter. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quartern. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quantant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently, Delta Quadrant. I am happy with the new book I got recently,

Resolving data files: 100%
 124/124 [00:00<00:00, 9963.48it/s]
Resolving data files: 100%
 113/113 [00:00<00:00, 57.54it/s]
Resolving data files: 100%
 113/113 [00:00<00:00, 114.07it/s]
Loading dataset shards: 100%
 105/105 [00:00<00:00, 2570.62it/s]
Total splits: 25
Available splits: ['LongAlign_64k_Qwen3_32B_yarn_131k_think', 'OpenThoughts3_1.2M_think', 'aya_dataset_Qwen3_32B_think', 'multi_turn_reasoning_if_think', 's1k_1.1_think', 'smolagents_toolcalling_traces_think', 'smoltalk_everyday_convs_reasoning_Qwen3_32B_think', 'smoltalk_multilingual8_Qwen3_32B_think', 'smoltalk_systemchats_Qwen3_32B_think', 'table_gpt_Qwen3_32B_think', 'LongAlign_64k_context_lang_annotated_lang_6_no_think', 'Mixture_of_Thoughts_science_no_think', 'OpenHermes_2.5_no_think', 'OpenThoughts3_1.2M_no_think_no_think', 'hermes_function_calling_v1_no_think', 'smoltalk_multilingual_8languages_lang_5_no_think', 'smoltalk_smollm3_everyday_conversations_no_think', 'smoltalk_smollm3_explore_instruct_rewriting_no_think', 'smoltalk_smollm3_smol_magpie_ultra_no_think', 'smoltalk_smollm3_smol_rewrite_no_think', 'smoltalk_smollm3_smol_summarize_no_think', 'smoltalk_smollm3_systemchats_30k_no_think', 'table_gpt_no_think', 'tulu_3_sft_personas_instruction_following_no_think', 'xlam_traces_no_think']
Number of total rows: 3383242
Dataset structure: DatasetDict({
    LongAlign_64k_Qwen3_32B_yarn_131k_think: Dataset()
    OpenThoughts3_1.2M_think: Dataset()
    aya_dataset_Qwen3_32B_think: Dataset()
    multi_turn_reasoning_if_think: Dataset()
    s1k_1.1_think: Dataset()
    smolagents_toolcalling_traces_think: Dataset()
    smoltalk_everyday_convs_reasoning_Qwen3_32B_think: Dataset()
    smoltalk_multilingual8_Qwen3_32B_think: Dataset()
    smoltalk_systemchats_Qwen3_32B_think: Dataset()
    table_gpt_Qwen3_32B_think: Dataset()
    LongAlign_64k_context_lang_annotated_lang_6_no_think: Dataset()
    Mixture_of_Thoughts_science_no_think: Dataset()
    OpenHermes_2.5_no_think: Dataset()
    OpenThoughts3_1.2M_no_think_no_think: Dataset()
    hermes_function_calling_v1_no_think: Dataset()
    smoltalk_multilingual_8languages_lang_5_no_think: Dataset()
    smoltalk_smollm3_everyday_conversations_no_think: Dataset()
    smoltalk_smollm3_explore_instruct_rewriting_no_think: Dataset()
    smoltalk_smollm3_smol_magpie_ultra_no_think: Dataset()
    smoltalk_smollm3_smol_rewrite_no_think: Dataset()
    smoltalk_smollm3_smol_summarize_no_think: Dataset()
    smoltalk_smollm3_systemchats_30k_no_think: Dataset()
    table_gpt_no_think: Dataset()
    tulu_3_sft_personas_instruction_following_no_think: Dataset()
    xlam_traces_no_think: Dataset()
})

### Обработка разных типов датасетов

Датасет SmolTalk2 представляет собой сборник открытых наборов данных, собранных вместе для удобства. Он содержит смесь полезных сценариев использования после обучения, таких как использование инструментов, длинный контекст и другие. Все они находятся в формате чата, что облегчает использование при обучении. Однако не все наборы данных разделены в одинаковом формате, поэтому часто нам нужно обрабатывать их в едином формате чатов `messages`.

Для этого упражнения мы будем стандартизировать различные форматы наборов данных в единую раскладку чатов `messages`. Мы определяем легковесные процессоры дляQA и наборов инструкций, и проходим через конкретный пример использования GSM8K.

```python
# Функция для обработки разных форматов наборов данных
def process_qa_dataset(examples, question_col, answer_col):
    """Обработка Q&A datasets в формат чата"""
    processed = []
    
    for question, answer in zip(examples[question_col], examples[answer_col]):
        messages = [,
            
        ]
        processed.append(messages)
    
    return 

def process_instruction_dataset(examples):
    """Обработка наборов из инструкций"""
    processed = []
    
    for instruction, response in zip(examples["instruction"], examples["response"]):
        messages = [,
            
        ]
        processed.append(messages)
    
    return 

# Пример: обработка математического датасета GSM8K
print("=== PROCESSING GSM8K DATASET ===\n")

gsm8k = load_dataset("openai/gsm8k", "main", split="train[:100]")  # Маленький подмножество для демо
print(f"Original GSM8K example: ")

# Конвертация в формат чата
def process_gsm8k(examples):
    processed = []
    for question, answer in zip(examples["question"], examples["answer"]):
        messages = [,,
            
        ]
        processed.append(messages)
    return 

gsm8k_processed = gsm8k.map(process_gsm8k, batched=True, remove_columns=gsm8k.column_names)
print(f"Processed example: ")

Вот два примера из двух отдельных наборов данных в том же формате.

Output
=== PROCESSING GSM8K DATASET ===```

=======================
� 
=======================
# 7.94k/? [00:00<00:00，572kB/s]
main/train-00000-of-00001.parquet：100%
 2.31M/2.31M [00:01<00:00，42.6kB/s]
main/test-00000-of-00001.parquet：100%
 419k/419k [00:00<00:00，813kB/s]
Generating train split：100%
 7473/7473 [00:00<00:00，321312.49 examples/s]
Generating test split：100%
 1319/1319 [00:00<00:00，97120.71 examples/s]
Original GSM8K example: {'question': 'Natalia sold clips to 48 of her friends in April, and then she sold half as many clips in May. How many clips did Natalia sell altogether in April and May?', 'answer': 'Natalia sold 48/2 = <<48/2=24>>24 clips in May.\nNatalia sold 48+24 = <<48+24=72>>72 clips altogether in April and May.\n#### 72'}
Map：100%
 100/100 [00:00<00:00，4792.50 examples/s]
Processed example:, {'content': 'Natalia sold clips to 48 of her friends in April, and then she sold half as many clips in May. How many clips did Natalia sell altogether in April and May?', 'role': 'user'}, {'content': 'Natalia sold 48/2 = <<48/2=24>>24 clips in May.\nNatalia sold 48+24 = <<48+24=72>>72 clips altogether in April and May.\n#### 72', 'role': 'assistant'}]

将对话模板应用于数据集

处理消息归一化后，我们将模型的对话模板应用于每个示例，将其转换为适合进行语言模型化的纯文本训练文本（text列）与 SFT 一同使用。

# Function to apply chat templates to processed datasets
def apply_chat_template_to_dataset(dataset, tokenizer):
    """Apply chat template to dataset for training"""
    
    def format_messages(examples):
        formatted_texts = []
        
        for messages in examples["messages"]:
            # 应用对话模板
            formatted_text = tokenizer.apply_chat_template(
                messages,
                tokenize=False,
                add_generation_prompt=False  # We want the complete conversation
            )
            formatted_texts.append(formatted_text)
        
        return 
    
    return dataset.map(format_messages, batched=True)

# Apply to our processed GSM8K dataset
gsm8k_formatted = apply_chat_template_to_dataset(gsm8k_processed, instruct_tokenizer)
print("=== FORMATTED TRAINING DATA ===")
print(gsm8k_formatted[0]["text"])

Загрузка и подготовка обучающего набора данных

print("=== PREPARING DATASET ===\n")

Загрузка данных GSM8K

dataset = load_dataset("GSM8K", split="train")

Применение функции предобработки к каждому примеру

def preprocess_function(examples):
# Загрузка модели и токенизатора (происходит только один раз за вызов)
from transformers import AutoModelForCausalLM, AutoTokenizer
from datasets import load_dataset
import torch
from trl import SFTTrainer, SFTConfig, DataCollatorForCompletionOnlyLM

# Основная модель и токенизатор
model_name = "HuggingFaceTB/SmolLM3-3B-Base"
tokenizer = AutoTokenizer.from_pretrained(model_name)

# Создаем шаблон диалога
header_template = "You are a faithful student's AI assistant. You must help a student solve a tedious and repetitive math problem.\nThe problem is: {question}\nThe solution is: {answer}\nDialog:"
entry = header_template.format(question=examples["question"], answer=examples["answer"])
chat_log = [
    {'content': entry, 'role': 'user'},
    {'content': 'Thank you!', 'role': 'user'},
    {'content': 'You are welcome!', 'role': 'assistant'}
]
# Пустой текст - начало чата
chat_log_with_placeholder = [{'content': '', 'role': 'system'}] + chat_log

# Склеиваем роль и содержимое для каждого элемента чата
text_list = []
for elem in chat_log_with_placeholder:
    text_list.append(f"{elem['role']}:" + " " + elem['content'])

# Применение токенизации и составление единой строки текста
all_text = "".join(text_list)
return {"text": all_text}

dataset = dataset.map(preprocess_function, batched=True)

Визуализация одного обработанного примера

print("Dataset processed. Let's take a look at one processed example:")
print(dataset[0]["text"])


```python output
=== PREPARING DATASET ===

Dataset processed. Let's take a look at one processed example:
You are a faithful student's AI assistant. You must help a student solve a tedious and repetitive math problem.
The problem is: Natalia sold clips to 48 of her friends in April, and then she sold half as many clips in May. How many clips did Natalia sell altogether in April and May?
The solution is: Natalia sold 48/2 = <<48/2=24>>24 clips in May.
Natalia sold 48+24 = <<48+24=72>>72 clips altogether in April and May.
#### 72
Dialog:
system: user: You are a faithful student's AI assistant. You must help a student solve a tedious and repetitive math problem.
The problem is: Natalia sold clips to 48 of her friends in April, and then she sold half as many clips in May. How many clips did Natalia sell altogether in April and May?
The solution is: Natalia sold 48/2 = <<8/2=4>>4 clips in May.
Natalia sold 8+4 = <<8+4=12>>12 clips altogether in April and May.
#### 12
user: Thank you!
assistant: You are welcome!

Step 2: Training Configuration

Configure a subset of our training data and set up our TrainingArguments for the fine-tuning task.

# Select a subset of data for faster training
subset_size = 256
dataset_subset = dataset.train_test_single_split(train_size=subset_size)

# Initialize data collator
data_collator = DataCollatorForCompletionOnlyLM(tokenizer=tokenizer)

# Define TrainingArguments
training_args = TrainingArguments(
    output_dir="./results",
    evaluation_strategy="no",
    per_device_train_batch_size=1,  # Gradient Accumulation may be needed for larger models
    gradient_accumulation_steps=4,  # Total batch size = 1 * 4 = 4
    learning_rate=1e-4,
    num_train_epochs=3,
    save_strategy="no",
)

print(f"\n=== TRAINING ===\n")

Named splits train and test: train (256), test (0)
Named splits train and test: train (256), test (0)


===

Option 1: Use SmolTalk2 (recommended for beginners)

dataset = load_dataset("HuggingFaceTB/smoltalk2", "SFT")
train_dataset = dataset["smoltalk_everyday_convs_reasoning_Qwen3_32B_think"].select(range(1000)) # Use subset for faster training

Option 2: Use your own processed dataset from Exercise 2

train_dataset = gsm8k_formatted.select(range(500))

print(f"Training examples: ")
print(f"Example: ")

Prepare the dataset for SFT

def format_chat_template(example):
"""Format the messages using the chat template"""
if "messages" in example:
# SmolTalk2 format
messages = example["messages"]
else:
# Custom format - adapt as needed
messages = [,

    ]

# Apply chat template
text = instruct_tokenizer.apply_chat_template(
    messages, 
    tokenize=False,
    add_generation_prompt=False
)
return

Apply formatting

formatted_dataset = train_dataset.map(format_chat_template)
formatted_dataset = formatted_dataset.remove_columns(
[col for col in formatted_dataset.column_names if col != "text"]
)
print(f"Formatted example: ...")


<details>
<summary>Output</summary>

```python output
=== PREPARING DATASET ===
Training examples: 1000
Example:, {'content': 'Natalia sold clips to 48 of her friends in April, and then she sold half as many clips in May. How many clips did Natalia sell altogether in April and May?', 'role': 'user'}]}
Formatted example: You are a math tutor. Solve problems step by step. Natalia sold clips to 48 of her friends in April, and then she sold half as many clips in May. How many clips did Natalia sell altogether in April and May?...

Training Configuration

We configure key knobs for SFT (batch size, sequence length, learning rate, logging/saving cadence) and enable optional tracking and Hub integration.

# Configure training parameters
training_config = SFTConfig(
    # Model and data
    output_dir=f"./",
    dataset_text_field="text",
    max_length=2048,
    
    # Training hyperparameters
    per_device_train_batch_size=2,  # Adjust based on your GPU memory
    gradient_accumulation_steps=2,
    learning_rate=5e-5,
    num_train_epochs=1,  # Start with 1 epoch
    max_steps=500,  # Limit steps for demo
    
    # Optimization
    warmup_steps=50,
    weight_decay=0.01,
    optim="adamw_torch",
    
    # Logging and saving
    logging_steps=10,
    save_steps=100,
    eval_steps=100,
    save_total_limit=2,
    
    # Memory optimization
    dataloader_num_workers=0,
    group_by_length=True,  # Group similar length sequences
    
    # LearnPath Hub integration
    push_to_hub=False,  # Set to True to upload to Hub
    hub_model_id=f"your-username/",
    
    # Experiment tracking
    report_to=["trackio"],  # Use trackio for experiment tracking
    run_name=f"-training",
)

print("Training configuration set!")
print(f"Effective batch size: ")

Optional: Train with LoRA/PEFT (memory-efficient)

If you have limited GPU memory or want faster iterations, use LoRA via PEFT. This trains only small adapter weights while keeping the base model frozen, then you can either keep using adapters or merge them later for deployment.

# LoRA configuration with PEFT
from peft import LoraConfig

peft_config = LoraConfig(
    r=8,
    lora_alpha=16,
    lora_dropout=0.05,
    bias="none",
    task_type="CAUSAL_LM",
)

# Create SFTTrainer with LoRA enabled
from trl import SFTTrainer

lora_trainer = SFTTrainer(
    model=model,
    train_dataset=formatted_dataset,  # dataset with a "text" field or messages + dataset_text_field in config
    args=training_config,
    peft_config=peft_config,  # << enable LoRA
)

print("Starting LoRA training…")
lora_trainer.train()

Step 4: Initialize SFTTrainer and Train

We instantiate the trainer, capture a pre-training baseline generation, launch train(), and save the resulting checkpoints to the configured output directory.


trainer = SFTTrainer(
    model=model,
    train_dataset=formatted_dataset,
    args=config,
)

And we can train the model.

trainer.train()

Test the Fine-Tuned Model

Finally, we regenerate the same prompt to qualitatively compare outputs before vs after training, and optionally push the model to the Hub for sharing.

# Test the fine-tuned model
print("=== AFTER TRAINING ===")
with torch.no_grad():
    outputs = model.generate(
        **inputs,
        max_new_tokens=100,
        temperature=0.7,
        do_sample=True,
        pad_token_id=tokenizer.eos_token_id
    )
    response = tokenizer.decode(outputs[0], skip_special_tokens=True)
    print(response[len(formatted_prompt):])

# Optional: Push to LearnPath Hub
if training_config.push_to_hub:
    trainer.push_to_hub(
        commit_message="Fine-tuned SmolLM3 with custom dataset",
        tags=["smol-course", "sft", "instruction-tuning"]
    )
    print(f"Model pushed to Hub: ")

Exercise 4: Production Workflow with TRL CLI

In the previous exercises we've dived deep into using TRL's Python API for fine-tuning and explored the data we're using and generating. In this exercise we'll explore using the TRL CLI to fine-tune a model. This will be the most common way to fine-tune a model in production.

We can define a command in TRL CLI to fine-tune a model. We'll be able to run it with trl sft command. The CLI command and Python API share the same configuration options.

We preprocessed the smoltalk_everyday_convs_reasoning_Qwen3_32B_think subset of SmolTalk2 so that is easier to work with it when using the TRL CLI.

# Fine-tune SmolLM3 using TRL CLI
trl sft \
    --model_name_or_path HuggingFaceTB/SmolLM3-3B-Base \
    --dataset_name HuggingFaceTB/smoltalk2_everyday_convs_think \
    --output_dir ./smollm3-sft-cli \
    --per_device_train_batch_size 4 \
    --gradient_accumulation_steps 2 \
    --learning_rate 5e-5 \
    --num_train_epochs 1 \
    --max_length 2048 \
    --logging_steps 10 \
    --save_steps 500 \
    --warmup_steps 100 \
    --bf16 True \
    --push_to_hub \
    --hub_model_id your-username/smollm3-sft-cli

Для удобства и воспроизводимости мы также можем создать файл конфигурации для тонкой настройки модели. Например, мы можем создать файл под названием sft_config.yaml и поместить в него следующее содержание:

# Model and dataset
model_name_or_path: HuggingFaceTB/SmolLM3-3B-Base
dataset_name: HuggingFaceTB/smoltalk2_everyday_convs_think
output_dir: ./smollm3-advanced-sft

# Training hyperparameters
per_device_train_batch_size: 2
gradient_accumulation_steps: 4
learning_rate: 3e-5
num_train_epochs: 2
max_length: 4096

# Optimization
warmup_steps: 200
weight_decay: 0.01
optim: adamw_torch
lr_scheduler_type: cosine

# Memory and performance
bf16: true
dataloader_num_workers: 4
group_by_length: true
remove_unused_columns: false

# Logging and evaluation
logging_steps: 25
eval_steps: 250
save_steps: 500
eval_strategy: steps
load_best_model_at_end: true
metric_for_best_model: eval_loss

# Hub integration
push_to_hub: true
hub_model_id: your-username/smollm3-advanced
hub_strategy: every_save

Мы могли бы затем зафиксировать этот файл в репозитории и отслеживать его с помощью Git.

# Run training with config file
trl sft --config sft_config.yaml

Устранение неполадок

Если вы получаете ошибки нехватки памяти GPU:

  • Уменьшите per_device_train_batch_size до 1
  • Уменьшите max_length до 1024 или 512
  • Используйте torch.cuda.empty_cache() для очистки памяти GPU

Если модели не загружаются:

  • Проверьте ваше интернет-соединение
  • Попробуйте использовать device_map="cpu" для загрузки на CPU
  • Используйте более маленькую модель, например HuggingFaceTB/SmolLM3-1.7B, для тестирования

Если обучение завершается сбоем:

  • Убедитесь, что ваш датасет правильно отформатирован
  • Проверьте, что все примеры имеют разумную длину (не слишком длинные)
  • Отслеживайте loss обучения - он должен постоянно уменьшаться

Заключение

Поздравляем! Вы завершили комплексные практические упражнения, охватывающие:

  • Систему шаблонов чата SmolLM3 и двухрежимное рассуждение
  • Техники обработки и подготовки датасетов
  • Контролируемую тонкую настройку с использованием Python API
  • Производственные рабочие процессы с использованием инструментов CLI
  • Распределенные обучающие установки

Эти навыки составляют основу для построения сложных инструкций, ориентированных на обучение моделей. В следующих модулях мы будем исследовать выравнивание предпочтений, эффективное тонкой настройке параметров и продвинутые методы оценки.

Ресурсы для последующего обучения

AI-тест
1 / 5

Что такое обучение с учителем (supervised learning)?