📖 英文と日本語訳（一文ずつ）

🎧 通し読み（全文）

リスニング・シャドーイング用の全文です。

The Qwen team has introduced Qwen-Scope, an interpretability toolkit designed to elucidate the internal workings of large language models (LLMs). This toolkit was trained based on the Qwen3 and Qwen3.5 series models and employs a methodology of inserting and training Sparse Autoencoders (SAEs) within the models' hidden layers. By imposing sparsity constraints, SAE decomposes the model's dense hidden representations into sparse, decoupled, and interpretable features, enabling the visualization of the internal mechanisms underlying Qwen's behavior. Fourteen sets of SAEs have been released, corresponding to a total of seven LLMs from the Qwen3 and Qwen3.5 series, including both dense and Mixture of Experts (MoE) models. To ensure broad feature coverage, strong semantic validity, and stable training, 500 million tokens were sampled from each model's pre-training data for the training process. SAE widths range from 32,000 to 128,000, and two types of L0 values, 50 and 100, are provided. The scope of application for Qwen-Scope spans four areas: inference, data, training, and evaluation. During inference, controlling feature activations enables the targeted control of outputs such as language, entities, and style without the need for explicit natural language instructions. On the data front, high-precision classification is achieved without additional training by selecting highly relevant features from a small amount of seed data. Furthermore, by utilizing inactive feature information to synthesize data that complements long-tail capabilities, training data efficiency has reportedly improved by approximately 15 times compared to conventional methods. Regarding its application in the training phase, to address language-mixing issues such as the unexpected inclusion of Chinese in English responses, abnormal activation patterns are identified, and a loss function is designed to suppress them during supervised fine-tuning (SFT), thereby reducing the frequency of undesirable outputs. Furthermore, to address the issue of infinite repetitive generation—which is difficult to sample during the reinforcement learning (RL) stage—amplifying the corresponding abnormal activation features increases the sampling probability of these failure cases, enabling efficient optimization. In terms of evaluation, analyzing the feature coverage of test sets allows for a quantitative understanding of evaluation redundancy across benchmarks. According to an analysis by the Qwen team, some widely used evaluation datasets show overlaps in activated features, confirming a decrease in the practical significance of repeated evaluations. This enables the selection of benchmarks that achieve both higher coverage and lower evaluation costs. Qwen-Scope is available for testing on Hugging Face and ModelScope, and a technical report has also been released. The Qwen team stated that the tool is not merely a means for post-analysis but can serve as a core engine driving model evolution, and expressed their openness to feedback from the community.

📝 学習のヒント

1. 1まず英文を読む — 知らない単語にあたりをつけてから音声へ。
2. 2一文ずつ確認 — 日本語訳と照合し、構文を理解する。
3. 3通し読み Normal で耳を作る — 内容を追いながらリピート。
4. 4Fast でシャドーイング — 口を慣らし、リスニング速度を上げる。
5. 5翌日に復習 — 1日空けて再聴すると長期記憶に定着しやすい。