📦 Release Notes (English)

ModelScope Download Module Refactor

• Architecture Upgrade: Rebuilt download module with producer-consumer pipeline for significantly improved throughput and responsiveness.
• Memory Optimization: Added server-side prefix filtering + paginated listing to reduce memory footprint for large-scale snapshots.
• Reliability Enhancements:
  • Retry mechanism for hash validation failures
  • Larger buffer size for hash computation
  • Fault-tolerant reporting with strict exception handling for pagination/partial-download errors
• Error Handling: Introduced FileDownloadError for granular single-file download failure tracking.
• Config Updates:
  • Default dataset page size set to 200
  • Updated build dependencies (pip, setuptools)

📦 发布说明（中文）

ModelScope 下载模块重构

• 架构升级：采用生产者 - 消费者流水线重构下载模块，显著提升下载吞吐与响应速度。
• 内存优化：支持服务端 prefix 过滤 + 分页列表拉取，大幅降低大规模快照下载时的内存占用。
• 可靠性增强：
  • 哈希校验失败自动重试
  • 增大哈希计算缓冲区
  • 容错上报机制，确保分页错误/部分下载失败时严格抛出异常，避免快照状态不一致
• 错误处理：新增 FileDownloadError 异常类，支持单文件下载失败的精细化追踪。
• 配置更新：
  • 默认数据集分页大小调整为 200
  • 更新构建依赖（pip, setuptools）

💡 Migration Tip: This is a backward-compatible refactor. No user code changes required, but users will notice faster and more stable downloads, especially for large-scale datasets.

What's Changed

• [Refactor] Refactor the modelscope download module by @wangxingjun778 in #1683

Full Changelog: v1.36.0...v1.36.1

中文版

✨ 新特性

• 文件夹上传优化：新增生产者-消费者管道机制，并发上传与进度追踪，大幅提升超大型数据集上传效率（#1671）

问题修复

• CSV 预览加载：修复流式加载 CSV 时的兼容性问题，支持 engine=python 并修复字段长度溢出错误（#1665）
• Swift 4.0 兼容：适配 ms-swift 4.0 包结构调整，更新模块导入路径，保持向后兼容（#1680）

部署

• GPG 密钥修复：更新 Dockerfile 中 GPG KEY 配置，支持多密钥验证（#1678）
• Ascend 镜像优化：切换至预构建 vllm-ascend 基础镜像，精简构建步骤，加快镜像生成速度（#1679）

🇬🇧 English Version

✨ Features

• Folder Upload Enhancement: Introduced producer-consumer pipeline for folder uploads with concurrent management, and progress tracking, significantly improving large-scale upload efficiency (#1671)

Bug Fixes

• CSV Preview Loading: Fixed streaming CSV loading issues, added engine=python support and resolved field size OverflowError (#1665)
• Swift 4.0 Compatibility: Updated import paths to align with ms-swift 4.0 package restructuring while maintaining backward compatibility (#1680)

Deployment

• GPG KEY Fix: Updated Dockerfile GPG KEY configuration to support multi-key verification (#1678)
• Ascend Dockerfile Optimization: Migrated to prebuilt vllm-ascend base image and streamlined build process for faster image generation (#1679)

What's Changed

• [Feature] Add producer-consumer pipeline for uploading folder by @wangxingjun778 in #1671
• [Fix] Fix preview csv stream loading by @wangxingjun778 in #1665
• Fix GPG KEY by @wangxingjun778 in #1678
• [bugfix] compat swift 4.0 by @Jintao-Huang in #1680
• update ascend dockerfile by @addsubmuldiv in #1679

New Contributors

• @addsubmuldiv made their first contribution in #1679

Full Changelog: v1.35.4...v1.36.0

📝 Release Notes / 发布说明

🇨🇳 Chinese (中文)

新功能 (Features)

• 支持 Collections 功能：引入了对模型/数据集集合（Collections）的支持，方便用户更好地组织和管理资源。 (#1656)

修复与优化 (Fixes & Improvements)

• 提升下载 CLI 健壮性：增强了 ModelScope 命令行下载工具的稳定性，改善用户体验。 (#1662)
• 修复下载 Token 认证问题：解决了在下载过程中可能出现的 Token 认证失败问题，确保权限验证正常。 (#1674)
• 修复 Skill 目录拼接错误：修正了 Skill 模块中目录路径拼接的逻辑错误。 (#1657)
• 优化预览加载与 HF 数据集工具：修复了预览加载问题，并对 Hugging Face 数据集相关工具类进行了重构，提升代码质量和可维护性。 (#1654)

🇺🇸 English

Features

• Collections Support: Added support for Collections, allowing users to better organize and manage models and datasets. (#1656)

Fixes & Improvements

• Enhanced Download CLI Robustness: Improved the stability and robustness of the ModelScope command-line download tool. (#1662)
• Fixed Token Authentication for Downloads: Resolved issues related to token authentication during the download process to ensure secure and successful access. (#1674)
• Fixed Skill Directory Concatenation: Corrected a bug in the directory path concatenation logic within the Skill module. (#1657)
• Preview Loading Fix & HF Utils Refactor: Fixed issues with preview loading and refactored Hugging Face dataset utilities for better code structure and maintainability. (#1654)

What's Changed

• merge 1.35.1 to master by @tastelikefeet in #1653
• Feat/collections by @wangxingjun778 in #1656
• [Fix] Skill DIR concat by @wangxingjun778 in #1657
• [Fix | Refactor] Fix preview loading and refactor hf datasets utils by @wangxingjun778 in #1654
• [Fix] Enhance the robustness of ModelScope download cli by @wangxingjun778 in #1662
• [Fix] fix token auth for downloading by @wangxingjun778 in #1674

Full Changelog: v1.35.3...v1.35.4

中文版本

新特性

1. 提升与 datasets 库的兼容性，适配了 4.0 及以上版本的变更，修复了因文件大小获取错误导致的数据预览加载问题，同时重构了代码结构并优化了 API 缓存机制以提升性能。

English Version

New Feature

1. Improves compatibility with datasets library 4.0+, fixes dataset preview loading issues caused by incorrect file size reporting, and refactors the codebase with optimized API caching for better performance.

Full Changelog: v1.35.2...v1.35.3

中文版本

新特性

1. 支持skills的下载

# Skills will be default placed to `~/.agents/skills`
# Download a collection of skills
modelscope download --collection MiniMax/MiniMax-Office-skills [--token xxx] [--local_dir ./temp]
# Download a batch skills of a single skill
modelscope skills add @MiniMax-AI/minimax-pdf @xxx/xxx [--token xxx]

English Version

New Feature

1. Support the downloading of skills

# Skills will be default placed to `~/.agents/skills`
# Download a collection of skills
modelscope download --collection MiniMax/MiniMax-Office-skills [--token xxx] [--local_dir ./temp]
# Download a batch skills of a single skill
modelscope skills add @MiniMax-AI/minimax-pdf @xxx/xxx [--token xxx]

Full Changelog: v1.35.1...v1.35.2

中文版本

Bug修复

1. 修复了oss2被全局依赖的问题
2. 修复了packaging没有被加入依赖的问题

English Version

BugFix

1. Fix an issue that oss2 was required by global import.
2. Fix an issue that packaging was not included in the dependencies.

What's Changed

• [Fix] lazy import oss2 by @wangxingjun778 in #1649
• fix packaging dependency by @tastelikefeet in #1651

Full Changelog: v1.35.0...v1.35.1

中文版本

新特性

1. ms_dataset模块重构，适配最新的datasets依赖（4.x大版本）
2. HubApi中的upload_file和upload_folder功能，增加对数据分块的重试功能
3. 重写镜像打包逻辑，将原有cpu、gpu镜像替换为old版本进行维护；原有的LLM镜像扩展出一个CPU镜像作为主镜像；同时原有swift镜像替换为latest镜像，表示依赖为最新版的镜像
4. 更换了CI镜像为Stable GPU镜像（原LLM镜像）

Bug修复

1. 修复了跨模型加载Config时，下载了全模型的问题
2. 修复了下载config和tokenizer文件时，正则表达式无效把模型文件也下载下来的问题
3. 修复pkg_resources问题
4. 修复了transformers5.0带来的CI报错，跳过了不兼容的小模型CI用例

English Version

New Features

1. Refactored the ms_dataset module to support the latest datasets dependency (major version 4.x)
2. Added retry functionality for chunked uploads in the upload_file and upload_folder features of HubApi
3. Rewrote the image packaging logic: replaced the original CPU and GPU images with "old" versions for maintenance; created a new CPU image from the original LLM image to serve as the main image; renamed the original swift image to "latest" to indicate it contains the most up-to-date dependencies
4. Switched the CI image to the Stable GPU image (formerly the LLM image)

Bug Fixes

1. Fixed an issue where the entire model was downloaded when loading Config across different models
2. Fixed an issue where an invalid regular expression caused model files to be downloaded along with config and tokenizer files
3. Fixed the pkg_resources issue
4. Fixed CI errors caused by transformers 5.0 by skipping incompatible small model CI test cases

What's Changed

• Fix #1316 by @wangxingjun778 in #1612
• [Fix] Fix aigc cover image by @wangxingjun778 in #1613
• fix downloading txt files by @tastelikefeet in #1610
• lint code by @tastelikefeet in #1619
• Fix downloading repos in automap by @tastelikefeet in #1630
• fix unitlock by @tastelikefeet in #1635
• compat with tf5.0 by @tastelikefeet in #1618
• [Bugfix] Make local_files_only configurable by @Potabk in #1632
• Feat/update docker file by @tastelikefeet in #1640
• [Feat] dataset module refactor by @wangxingjun778 in #1623
• add metax dockerfile and its requirements for ms-swift by @vx120 in #1643
• [Fix] fix dataset util by @wangxingjun778 in #1645

New Contributors

• @Potabk made their first contribution in #1632
• @vx120 made their first contribution in #1643

Full Changelog: v1.34.0...v1.35.0

中文版

• HubApi 认证优化 (#1573): 更新了 HubAPI 的 cookie 获取逻辑，支持在初始化时传入 token 参数，并优化了 token 的获取优先级（函数参数 > 实例属性 > 环境变量）。
• 修复并完善了模型与数据集操作中 token 参数的透传逻辑 (#1592, #1593): 统一认证令牌传递机制，确保 API 调用携带有效认证凭据。同时优化了 Cookie 处理逻辑，提升与 ModelScope Hub 交互的可靠性与安全性。
• 修复 OSS STS 认证过期问题 (#1589): 重构了 OSS 工具类，新增 CredentialProviderWrapper 来自动处理临时凭证的刷新，解决了长时间数据集下载时 STS token 过期导致的失败问题。
• 弃用仓库删除相关 API (#1588): 出于安全考虑，暂时弃用 delete_repo、delete_model 和 delete_dataset 方法。这些方法现在会发出 DeprecationWarning 警告，将在未来版本中基于完善的 token 认证机制重新启用。
• 修复 FSMN 模型 ONNX 导出问题 (#1576, #1577): 为 UniDeepFsmnDilated 模块的 squeeze 操作显式指定 dim 参数，修复了 ONNX 导出后在 TensorRT 中运行失败的问题。
• 修复 Docker 镜像构建的 APT 源问题 (#1575): 移除 Dockerfile 中的 nvidia.cn APT 源，解决了在某些网络环境下 apt-get update 失败的问题。
• 修复数据集下载的依赖问题 (#1581): 将 is_relative_path 函数从 datasets 库的导入改为使用内置实现，解决了与特定版本 datasets 库的兼容性问题。
• 修复 ignore_patterns 重置问题 (#1578): 修复了当 allow_patterns 包含 ** 时错误地清空 ignore_patterns 的问题。

English Version

• HubAPI authentication enhancement (#1573): Updated HubAPI cookie retrieval logic to support passing a token parameter during initialization, with optimized token priority (function parameter > instance attribute > environment variable).
• Fixed and improved token propagation in model and dataset operations (#1592, #1593): Unified authentication token handling to ensure API calls carry valid credentials. Also enhanced cookie handling logic to improve reliability and security when interacting with ModelScope Hub.
• Fixed OSS STS authentication expiry issue (#1589): Refactored OSS utility classes by introducing CredentialProviderWrapper to automatically handle temporary credentials refresh, resolving failures during long-running dataset downloads due to STS token expiration.
• Deprecated repository deletion APIs (#1588): For security reasons, temporarily deprecated delete_repo, delete_model, and delete_dataset methods. These methods now emit DeprecationWarning and will be re-enabled in future versions with proper token authentication mechanisms.
• Fixed FSMN model ONNX export issue (#1576, #1577): Explicitly specified the dim parameter for the squeeze operation in UniDeepFsmnDilated module, fixing failures when running ONNX-exported models in TensorRT.
• Fixed Docker image build APT source issue (#1575): Removed nvidia.cn APT sources from Dockerfile to fix apt-get update failures in certain network environments.
• Fixed dataset download dependency issue (#1581): Replaced is_relative_path import from the datasets library with a built-in implementation, resolving compatibility issues with specific versions of the datasets library.
• Fixed ignore_patterns reset issue (#1578): Fixed an issue where ignore_patterns was incorrectly cleared when allow_patterns contained **.

What's Changed

• Feat: update hubapi get cookie by @Yunnglin in #1573
• Fix: remove reset ignore_patterns by @wangxingjun778 in #1578
• Fix: datasets dependency for snapshot download by @wangxingjun778 in #1581
• fix: Remove nvidia.cn from APT sources to fix apt failures in certain… by @suluyana in #1575
• Fix TensorRT parse failure for MossFormer2 ONNX Squeeze axes by @Jzz1943 in #1577
• Fix: deprecate delete_repo, delete_model and delete_dataset due to token a… by @wangxingjun778 in #1588
• [Fix]oss utils sts auth expire issue by @wangxingjun778 in #1589
• [Fix] hub push model with token by @Yunnglin in #1592
• [Fix] HubAPI token usage by @Yunnglin in #1593

New Contributors

• @Jzz1943 made their first contribution in #1577

Full Changelog: v1.33.0...v1.34.0

中文版

主要更新

• 更新默认分支版本（default revision）策略，以提升仓库版本管理的一致性与易用性 #1546 #1553
• Hub API 新增 set_repo_visibility 接口，支持设置模型仓库可见性，并补充可见性参数校验和错误日志提示 #1564 #1568 #1569 #1570
• 新增 Ascend NPU 的 Dockerfile，便于在昇腾 NPU 环境中部署与使用 ModelScope #1550
• snapshot_download 新增 DEFAULT_MAX_WORKERS 参数配置，优化模型快照下载的并发行为 #1566

AIGC 相关能力增强

• 为 AIGC 的 create_model_tag 接口新增参数，支持更灵活的模型标签创建与管理 #1555
• AIGC 模型新增 base_model_sub_type 字段，便于更精细地描述与区分底座模型类型 #1563

Hub 与上传下载体验优化

• 修正 upload_folder 的默认忽略规则，避免重要文件被错误忽略 #1549
• 修复上传模式匹配（upload patterns）相关问题，提升文件上传的准确性 #1567

稳定性与工程改进

• 修复影响 hasattr 行为的实验类型定义问题，避免潜在属性检测异常 #1552
• 动态模块加载逻辑补丁修复，提升动态模块相关功能的稳定性 #1554
• 更新构建文件与 CI 工作流，改进整体构建发布流程 #1551

English Version

Key Updates

• Updated default revision strategy to improve consistency and usability of repository version management #1546 #1553
• Added set_repo_visibility in Hub API to control repository visibility, with input validation and detailed error logging #1564 #1568 #1569 #1570
• Added Ascend NPU Dockerfile to simplify deployment on Ascend NPU environments #1550
• Introduced DEFAULT_MAX_WORKERS in snapshot_download to better control concurrent download behavior #1566

AIGC Enhancements

• Added new arguments for AIGC create_model_tag for more flexible model tag creation and management #1555
• Added base_model_sub_type for AIGC models to describe and distinguish base model types more precisely #1563

Hub & Upload/Download Improvements

• Fixed default ignore patterns in upload_folder to avoid unintentionally skipping important files #1549
• Fixed upload patterns issues to improve accuracy of file uploads #1567

Stability & Engineering

• Fixed experiment type definition that affected hasattr behavior and could cause attribute detection issues #1552
• Patched dynamic module handling to improve stability of dynamic module related features #1554
• Updated build files and CI workflows to refine the overall build and release pipeline #1551

What's Changed

• Feat/update default revision by @alcholiclg in #1546
• Fix default ignore patterns for upload_folder by @wangxingjun778 in #1549
• Fix exp type which affects hasattr by @tastelikefeet in #1552
• Patch dynamic module by @tastelikefeet in #1554
• Feat: add new args for aigc create_model_tag by @wangxingjun778 in #1555
• [docker] Add Ascend NPU dockerfile by @baymax591 in #1550
• Update build file and workflow by @Yunnglin in #1551
• [WIP]Feat/update default revision by @alcholiclg in #1553
• Feat: add set_repo_visibility in hub api by @wangxingjun778 in #1564
• Add base_model_sub_type for aigc model by @wangxingjun778 in #1563
• Feat: add DEFAULT_MAX_WORKERS in snapshot_download func by @wangxingjun778 in #1566
• Fix/upload patterns by @wangxingjun778 in #1567
• fix set_repo_visibility by @suluyana in #1568
• Fix: add error log for invalid input visibility args by @wangxingjun778 in #1569
• fix: check tasks in HubApi.set_repo_visibility by @suluyana in #1570

New Contributors

• @baymax591 made their first contribution in #1550

Full Changelog: v1.32.0...v1.33.0

中文版

1. 支持使用 pyproject.toml 来安装 modelscope 库，保证与之前安装方式兼容 #1542
2. hub api 的 list_datasets 接口切换为 OpenAPI 端点 #1532
3. AIGC 模型支持 model_source 参数 #1530
4. ModelScope Hub 添加定期自动上传功能：自动定期上传本地文件夹到 ModelScope Hub，增量提交日志、训练产物等文件 #1480
5. AIGC 的 cover_images 支持 mp4 文件 #1533
6. 为 .safetensors 等单个模型文件添加内部云加速 #1529

English Version

1. Support using pyproject.toml to install the modelscope library, ensuring compatibility with previous installation methods #1542
2. Switch the list_datasets interface of hub API to OpenAPI endpoint #1532
3. AIGC models support the model_source parameter #1530
4. Add scheduled automatic upload functionality to ModelScope Hub: automatically upload local folders to ModelScope Hub at regular intervals, with incremental commits of logs, training artifacts, and other files #1480
5. AIGC's cover_images now support mp4 files #1533
6. Add internal cloud acceleration for single model files such as .safetensors #1529

What's Changed

• Feat/support commit scheduler by @alcholiclg in #1480
• Add INTRA_CLOUD_ACCELERATION for single model file downloading by @wangxingjun778 in #1529
• Feat: add model source for aigc model by @wangxingjun778 in #1530
• Add trust_remote_code for _download_additional_modules by @wangxingjun778 in #1521
• Update list datasets to OpenAPI by @Yunnglin in #1532
• [Feature] Add pyproject.toml by @Yunnglin in #1542
• Remove raise FileIntegrityError by @wangxingjun778 in #1517
• [WIP]change DEFAULT_MODEL_REVISION to master by @alcholiclg in #1544
• update ollama: granite4,deepseek-v3.1,gpt-oss by @xiaojianpinga in #1531
• Add mp4 support for AIGC cover_images by @wangxingjun778 in #1533
• fix daily regression by @Yunnglin in #1545

New Contributors

• @xiaojianpinga made their first contribution in #1531

Full Changelog: v1.31.0...v1.32.0