feat(jimeng): 添加即梦AI视频生成功能
- 新增 scripts/jimeng_video_gen.py 脚本支持文生视频和图生视频 - 实现 t2v 和 i2v 两种生成模式，支持多种分辨率和宽高比 - 添加轮询任务状态和视频下载功能 - 更新即梦AI文档，补充视频生成API接口说明 - 新增 .claude/commands/generate-video.md 命令使用指南 - 扩展即梦API文档，包含视频生成参数配置和调用示例
This commit is contained in:
parent
b4a6f9b79c
commit
9408daeeff
62
.claude/commands/generate-video.md
Normal file
62
.claude/commands/generate-video.md
Normal file
@ -0,0 +1,62 @@
|
|||||||
|
# 即梦 AI 视频生成
|
||||||
|
|
||||||
|
使用即梦 AI（火山引擎）生成视频。
|
||||||
|
|
||||||
|
## 使用方式
|
||||||
|
|
||||||
|
用户会提供：
|
||||||
|
1. **prompt**：视频描述（中文）
|
||||||
|
2. **output_path**：输出路径（可选，默认保存到当前工作目录）
|
||||||
|
3. **mode**：生成模式（可选，默认 t2v 文生视频）
|
||||||
|
|
||||||
|
## 执行步骤
|
||||||
|
|
||||||
|
1. 激活项目虚拟环境：`source .venv/bin/activate`
|
||||||
|
2. 调用 `scripts/jimeng_video_gen.py` 生成视频
|
||||||
|
3. 根据用户需求选择合适的参数：
|
||||||
|
- **文生视频**：`--mode t2v`（默认）
|
||||||
|
- **图生视频-首帧**：`--mode i2v --image <图片路径或URL>`
|
||||||
|
- **分辨率**：`--resolution 720p`（默认）或 `--resolution 1080p`
|
||||||
|
- **宽高比**：`--aspect-ratio 16:9`（默认），支持 `16:9, 4:3, 1:1, 3:4, 9:16, 21:9`
|
||||||
|
- **帧数**：`--frames 121`（默认，约5秒）
|
||||||
|
4. 展示生成结果给用户
|
||||||
|
|
||||||
|
## 调用示例
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# 文生视频 (720P, 16:9)
|
||||||
|
source .venv/bin/activate && python scripts/jimeng_video_gen.py "视频描述prompt" "输出路径.mp4"
|
||||||
|
|
||||||
|
# 文生视频 (1080P)
|
||||||
|
source .venv/bin/activate && python scripts/jimeng_video_gen.py "视频描述prompt" "输出路径.mp4" --resolution 1080p
|
||||||
|
|
||||||
|
# 图生视频-首帧
|
||||||
|
source .venv/bin/activate && python scripts/jimeng_video_gen.py "视频描述prompt" "输出路径.mp4" --mode i2v --image "首帧图片.jpg"
|
||||||
|
|
||||||
|
# 竖屏视频
|
||||||
|
source .venv/bin/activate && python scripts/jimeng_video_gen.py "视频描述prompt" "输出路径.mp4" --aspect-ratio 9:16
|
||||||
|
```
|
||||||
|
|
||||||
|
## Prompt 技巧
|
||||||
|
|
||||||
|
- **基础结构**：主体 + 背景 + 镜头 + 动作
|
||||||
|
- **运镜词汇**：镜头切换、平移、推轨、环形跟踪、特写、俯拍、航拍
|
||||||
|
- **多镜头叙事**：支持描述连贯场景切换
|
||||||
|
|
||||||
|
## 注意事项
|
||||||
|
|
||||||
|
- 视频生成耗时较长，通常 20-30 秒完成，最长等待 3 分钟
|
||||||
|
- 支持分辨率：720P（默认）和 1080P
|
||||||
|
- 支持宽高比：16:9、4:3、1:1、3:4、9:16、21:9
|
||||||
|
- 脚本位于 `scripts/jimeng_video_gen.py`，依赖 `scripts/jimeng_gen.py` 的签名逻辑
|
||||||
|
- 虚拟环境位于 `.venv/`，已安装 `requests`
|
||||||
|
|
||||||
|
## 可用模型 (req_key)
|
||||||
|
|
||||||
|
| 功能 | req_key | 分辨率 |
|
||||||
|
|------|---------|--------|
|
||||||
|
| 文生视频 | `jimeng_t2v_v30` | 720P |
|
||||||
|
| 文生视频 | `jimeng_t2v_v30_1080p` | 1080P |
|
||||||
|
| 图生视频-首帧 | `jimeng_i2v_first_v30` | 720P |
|
||||||
|
|
||||||
|
$ARGUMENTS
|
||||||
@ -1,4 +1,4 @@
|
|||||||
# 即梦 AI 图片生成 API
|
# 即梦 AI API（图片生成 & 视频生成）
|
||||||
|
|
||||||
## 密钥
|
## 密钥
|
||||||
|
|
||||||
@ -216,10 +216,139 @@ if __name__ == '__main__':
|
|||||||
print(json.dumps(result, indent=2, ensure_ascii=False))
|
print(json.dumps(result, indent=2, ensure_ascii=False))
|
||||||
```
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 视频生成接口
|
||||||
|
|
||||||
|
### 模型与 req_key 对照表
|
||||||
|
|
||||||
|
| 功能 | req_key | 分辨率 | 说明 |
|
||||||
|
|------|---------|--------|------|
|
||||||
|
| 文生视频 | `jimeng_t2v_v30` | 720P | 文本提示词生成视频 |
|
||||||
|
| 文生视频 | `jimeng_t2v_v30_1080p` | 1080P | 文本提示词生成高清视频 |
|
||||||
|
| 图生视频-首帧 | `jimeng_i2v_first_v30` | 720P | 首帧图片+提示词生成视频 |
|
||||||
|
|
||||||
|
### 通用参数
|
||||||
|
|
||||||
|
| 参数 | 类型 | 必填 | 说明 |
|
||||||
|
|------|------|------|------|
|
||||||
|
| `req_key` | string | 是 | 模型标识，见上表 |
|
||||||
|
| `prompt` | string | 是 | 视频描述提示词 |
|
||||||
|
| `frames` | int | 否 | 视频帧数，默认 121（约5秒） |
|
||||||
|
| `aspect_ratio` | string | 否 | 宽高比：`16:9`、`4:3`、`1:1`、`3:4`、`9:16`、`21:9` |
|
||||||
|
| `seed` | int | 否 | 随机种子，-1 为随机 |
|
||||||
|
|
||||||
|
### 图生视频额外参数
|
||||||
|
|
||||||
|
| 参数 | 类型 | 说明 |
|
||||||
|
|------|------|------|
|
||||||
|
| `image_urls` | string[] | 首帧图片 URL 列表 |
|
||||||
|
| `binary_data_base64` | string[] | 首帧图片 Base64 编码（与 image_urls 二选一） |
|
||||||
|
|
||||||
|
### 文生视频 - 提交任务
|
||||||
|
|
||||||
|
```
|
||||||
|
POST https://visual.volcengineapi.com?Action=CVSync2AsyncSubmitTask&Version=2022-08-31
|
||||||
|
```
|
||||||
|
|
||||||
|
请求体：
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"req_key": "jimeng_t2v_v30",
|
||||||
|
"prompt": "一只橘猫趴在键盘上打字，桌面上有一杯咖啡，温暖的灯光",
|
||||||
|
"frames": 121,
|
||||||
|
"aspect_ratio": "16:9",
|
||||||
|
"seed": -1
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
返回：
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"code": 10000,
|
||||||
|
"data": {
|
||||||
|
"task_id": "9675314167630911764"
|
||||||
|
},
|
||||||
|
"message": "Success"
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
### 查询视频结果
|
||||||
|
|
||||||
|
```
|
||||||
|
POST https://visual.volcengineapi.com?Action=CVSync2AsyncGetResult&Version=2022-08-31
|
||||||
|
```
|
||||||
|
|
||||||
|
请求体：
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"req_key": "jimeng_t2v_v30",
|
||||||
|
"task_id": "9675314167630911764"
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
返回（完成时）：
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"code": 10000,
|
||||||
|
"data": {
|
||||||
|
"status": "done",
|
||||||
|
"video_url": "https://v11-aiop.aigc-cloud.com/...",
|
||||||
|
"aigc_meta_tagged": false
|
||||||
|
},
|
||||||
|
"message": "Success"
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
### 图生视频-首帧 - 提交任务
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"req_key": "jimeng_i2v_first_v30",
|
||||||
|
"prompt": "猫咪缓缓转头看向镜头",
|
||||||
|
"image_urls": ["https://example.com/cat.jpg"],
|
||||||
|
"frames": 121,
|
||||||
|
"seed": -1
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
### Prompt 技巧
|
||||||
|
|
||||||
|
- **基础结构**：主体 + 背景 + 镜头 + 动作
|
||||||
|
- **运镜词汇**：镜头切换、平移、推轨、环形跟踪、特写、俯拍、航拍
|
||||||
|
- **多镜头叙事**：支持描述连贯场景切换（3.0 Pro 特性）
|
||||||
|
|
||||||
|
### Python 调用
|
||||||
|
|
||||||
|
```bash
|
||||||
|
source .venv/bin/activate
|
||||||
|
|
||||||
|
# 文生视频 (720P)
|
||||||
|
python scripts/jimeng_video_gen.py "一只橘猫趴在键盘上打字" output.mp4
|
||||||
|
|
||||||
|
# 文生视频 (1080P)
|
||||||
|
python scripts/jimeng_video_gen.py "一只橘猫趴在键盘上打字" output.mp4 --resolution 1080p
|
||||||
|
|
||||||
|
# 图生视频-首帧
|
||||||
|
python scripts/jimeng_video_gen.py "猫咪转头看镜头" output.mp4 --mode i2v --image first_frame.jpg
|
||||||
|
|
||||||
|
# 指定宽高比和帧数
|
||||||
|
python scripts/jimeng_video_gen.py "描述" output.mp4 --aspect-ratio 9:16 --frames 121
|
||||||
|
```
|
||||||
|
|
||||||
## 参考文档
|
## 参考文档
|
||||||
|
|
||||||
|
### 图片生成
|
||||||
- [即梦AI图片生成4.0 产品介绍](https://www.volcengine.com/docs/85621/1820192)
|
- [即梦AI图片生成4.0 产品介绍](https://www.volcengine.com/docs/85621/1820192)
|
||||||
- [即梦AI图片生成4.0 接口文档](https://www.volcengine.com/docs/85621/1817045)
|
- [即梦AI图片生成4.0 接口文档](https://www.volcengine.com/docs/85621/1817045)
|
||||||
|
|
||||||
|
### 视频生成
|
||||||
|
- [即梦AI视频生成3.0 Pro 产品介绍](https://www.volcengine.com/docs/85621/1783678)
|
||||||
|
- [即梦AI视频生成3.0 Pro 接口文档](https://www.volcengine.com/docs/85621/1777001)
|
||||||
|
- [即梦AI视频生成3.0 720P 接口文档](https://www.volcengine.com/docs/85621/1792710)
|
||||||
|
- [即梦AI视频生成3.0 1080P 接口文档](https://www.volcengine.com/docs/85621/1792711)
|
||||||
|
|
||||||
|
### 通用
|
||||||
- [Python SDK (GitHub)](https://github.com/volcengine/volc-sdk-python)
|
- [Python SDK (GitHub)](https://github.com/volcengine/volc-sdk-python)
|
||||||
- [SDK 使用说明](https://www.volcengine.com/docs/6444/1340578)
|
- [SDK 使用说明](https://www.volcengine.com/docs/6444/1340578)
|
||||||
- [HTTP 请求示例](https://www.volcengine.com/docs/6444/1390583)
|
- [HTTP 请求示例](https://www.volcengine.com/docs/6444/1390583)
|
||||||
|
|||||||
146
scripts/jimeng_video_gen.py
Normal file
146
scripts/jimeng_video_gen.py
Normal file
@ -0,0 +1,146 @@
|
|||||||
|
"""即梦 AI 视频生成工具
|
||||||
|
|
||||||
|
支持：
|
||||||
|
- 文生视频 (t2v): jimeng_t2v_v30 (720P) / jimeng_t2v_v30_1080p (1080P)
|
||||||
|
- 图生视频-首帧 (i2v): jimeng_i2v_first_v30 (720P)
|
||||||
|
"""
|
||||||
|
|
||||||
|
import json
|
||||||
|
import os
|
||||||
|
import sys
|
||||||
|
import time
|
||||||
|
import base64
|
||||||
|
import requests
|
||||||
|
|
||||||
|
from jimeng_gen import jimeng_request, _ensure_dir
|
||||||
|
|
||||||
|
|
||||||
|
# req_key 映射
|
||||||
|
REQ_KEYS = {
|
||||||
|
't2v_720p': 'jimeng_t2v_v30',
|
||||||
|
't2v_1080p': 'jimeng_t2v_v30_1080p',
|
||||||
|
'i2v_720p': 'jimeng_i2v_first_v30',
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def generate_video_t2v(prompt, output_path, resolution='720p', aspect_ratio='16:9', frames=121, seed=-1):
|
||||||
|
"""文生视频：提交任务并轮询获取结果"""
|
||||||
|
req_key = REQ_KEYS.get(f't2v_{resolution}', REQ_KEYS['t2v_720p'])
|
||||||
|
print(f"[即梦视频] 文生视频 ({resolution}): {prompt[:50]}...")
|
||||||
|
|
||||||
|
body = {
|
||||||
|
'req_key': req_key,
|
||||||
|
'prompt': prompt,
|
||||||
|
'frames': frames,
|
||||||
|
'aspect_ratio': aspect_ratio,
|
||||||
|
'seed': seed,
|
||||||
|
}
|
||||||
|
|
||||||
|
return _submit_and_poll(req_key, body, output_path)
|
||||||
|
|
||||||
|
|
||||||
|
def generate_video_i2v(prompt, image_path_or_url, output_path, aspect_ratio='16:9', frames=121, seed=-1):
|
||||||
|
"""图生视频-首帧：输入首帧图片和提示词生成视频"""
|
||||||
|
req_key = REQ_KEYS['i2v_720p']
|
||||||
|
print(f"[即梦视频] 图生视频-首帧: {prompt[:50]}...")
|
||||||
|
|
||||||
|
body = {
|
||||||
|
'req_key': req_key,
|
||||||
|
'prompt': prompt,
|
||||||
|
'frames': frames,
|
||||||
|
'seed': seed,
|
||||||
|
}
|
||||||
|
|
||||||
|
# 判断是 URL 还是本地文件
|
||||||
|
if image_path_or_url.startswith('http'):
|
||||||
|
body['image_urls'] = [image_path_or_url]
|
||||||
|
else:
|
||||||
|
with open(image_path_or_url, 'rb') as f:
|
||||||
|
img_b64 = base64.b64encode(f.read()).decode('utf-8')
|
||||||
|
body['binary_data_base64'] = [img_b64]
|
||||||
|
|
||||||
|
return _submit_and_poll(req_key, body, output_path)
|
||||||
|
|
||||||
|
|
||||||
|
def _submit_and_poll(req_key, body, output_path, max_wait=180, poll_interval=10):
|
||||||
|
"""提交任务并轮询结果，下载视频到本地"""
|
||||||
|
# 1. 提交任务
|
||||||
|
submit_resp = jimeng_request('CVSync2AsyncSubmitTask', body)
|
||||||
|
|
||||||
|
if submit_resp.get('code') != 10000:
|
||||||
|
print(f"[即梦视频] 提交失败: {submit_resp}")
|
||||||
|
return None
|
||||||
|
|
||||||
|
task_id = submit_resp['data']['task_id']
|
||||||
|
print(f"[即梦视频] 任务已提交, task_id: {task_id}")
|
||||||
|
|
||||||
|
# 2. 轮询查询结果
|
||||||
|
max_polls = max_wait // poll_interval
|
||||||
|
for i in range(max_polls):
|
||||||
|
time.sleep(poll_interval)
|
||||||
|
result = jimeng_request('CVSync2AsyncGetResult', {
|
||||||
|
'req_key': req_key,
|
||||||
|
'task_id': task_id,
|
||||||
|
})
|
||||||
|
|
||||||
|
resp_code = result.get('code')
|
||||||
|
resp_data = result.get('data', {})
|
||||||
|
|
||||||
|
if resp_code == 10000 and resp_data.get('status') == 'done':
|
||||||
|
video_url = resp_data.get('video_url')
|
||||||
|
if video_url:
|
||||||
|
# 下载视频
|
||||||
|
print(f"[即梦视频] 生成完成，正在下载...")
|
||||||
|
video_data = requests.get(video_url).content
|
||||||
|
_ensure_dir(output_path)
|
||||||
|
with open(output_path, 'wb') as f:
|
||||||
|
f.write(video_data)
|
||||||
|
print(f"[即梦视频] 视频已保存: {output_path}")
|
||||||
|
return output_path
|
||||||
|
else:
|
||||||
|
print(f"[即梦视频] 任务完成但未找到视频URL: {json.dumps(result, ensure_ascii=False)[:300]}")
|
||||||
|
return None
|
||||||
|
|
||||||
|
if resp_code != 10000 and resp_data.get('status') not in (None, 'running', 'pending'):
|
||||||
|
print(f"[即梦视频] 任务失败: {json.dumps(result, ensure_ascii=False)[:300]}")
|
||||||
|
return None
|
||||||
|
|
||||||
|
print(f"[即梦视频] 等待中... ({i+1}/{max_polls})")
|
||||||
|
|
||||||
|
print("[即梦视频] 超时")
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def main():
|
||||||
|
import argparse
|
||||||
|
parser = argparse.ArgumentParser(description='即梦 AI 视频生成')
|
||||||
|
parser.add_argument('prompt', help='视频描述提示词')
|
||||||
|
parser.add_argument('output', help='输出视频路径 (.mp4)')
|
||||||
|
parser.add_argument('--mode', choices=['t2v', 'i2v'], default='t2v', help='生成模式: t2v=文生视频, i2v=图生视频')
|
||||||
|
parser.add_argument('--resolution', choices=['720p', '1080p'], default='720p', help='分辨率 (仅 t2v)')
|
||||||
|
parser.add_argument('--aspect-ratio', default='16:9', help='宽高比: 16:9, 4:3, 1:1, 3:4, 9:16, 21:9')
|
||||||
|
parser.add_argument('--frames', type=int, default=121, help='视频帧数 (默认121)')
|
||||||
|
parser.add_argument('--image', help='首帧图片路径或URL (仅 i2v 模式)')
|
||||||
|
parser.add_argument('--seed', type=int, default=-1, help='随机种子 (-1=随机)')
|
||||||
|
|
||||||
|
args = parser.parse_args()
|
||||||
|
|
||||||
|
if args.mode == 'i2v':
|
||||||
|
if not args.image:
|
||||||
|
parser.error('i2v 模式需要 --image 参数')
|
||||||
|
result = generate_video_i2v(args.prompt, args.image, args.output,
|
||||||
|
aspect_ratio=args.aspect_ratio, frames=args.frames, seed=args.seed)
|
||||||
|
else:
|
||||||
|
result = generate_video_t2v(args.prompt, args.output,
|
||||||
|
resolution=args.resolution, aspect_ratio=args.aspect_ratio,
|
||||||
|
frames=args.frames, seed=args.seed)
|
||||||
|
|
||||||
|
if result:
|
||||||
|
print(f"\n视频生成成功: {result}")
|
||||||
|
else:
|
||||||
|
print("\n视频生成失败")
|
||||||
|
sys.exit(1)
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == '__main__':
|
||||||
|
main()
|
||||||
BIN
videos/res/卖火柴的小女孩.mp4
Normal file
BIN
videos/res/卖火柴的小女孩.mp4
Normal file
Binary file not shown.
Loading…
Reference in New Issue
Block a user