MiniMax H3 Ref2VA

最近放著正事沒做都在研究 MiniMax H3的Ref2VA

有些測試做了很多次 來寫一下結果

1.加速節點比較:

0.7MP / 1152x640 / 158 幀 實測：

| 後端 | 耗時 | 加速 |銳利度 |

|----------------|-----------|-------|-------|

| Sage | 4.90 min | 2.23x | 99.6% | (Blackwell SM120 SageAttention)| Comfy Kitchen | 6.09 min | 1.79x | 97.1% |

| PyTorch (SDPA) | 10.92 min | 1.00x | 100% | (沒加速的官方基準線)

結論: Sage Attention最快且最忠實

(也有測過Turbo Lora 4Steps之類的, 輸出品質太差用不下去)

2.ref_video自己先用ffmpeg調整好最保險,先準備兩個解析度:

576x320 for 0.2mp用

1152x640 for 0.7mp用

一定要24fps, 畫布問題前一篇解釋過, 這邊不解釋只寫結論,type選None不縮放

3.ref_picture原則上一張全身就夠,選match,背景用中性灰色因為H3只有RGB通道;產出偶爾跑出灰色背景代表prompts沒寫好,偷吃步是背景用ref_video的背景,需額外準備

4.Steps: 選Simple 10steps, 使用MiniMax H3時其實選Simple才能讓它的內部機制正常運作,對Ref2VA來說10Steps就夠

5.ref_video輸入後Qwen只用原生2fps看ref影片,雖然可以改成4或8fps,

但由於H3訓練時就是用2fps,提高fps輸出效果並不好;

2fps的視覺對於一般用途來說是夠的,也就是H3每秒抽2幀來看,中間自由發揮;

對於快速的動作例如舞蹈或是快剪的鏡頭切換來說,2fps完全不夠,

但有其它手段(prompts&shot)可補足

只是需要知道H3並非是frame lock原理的Video AI,只能做到大致接近

6.prompts&shot: 官方文件中其實定義了嚴格的格式和名詞,雖然亂寫也都有用,但如果要精準控制H3動作,就需要符合格式

建議是讓Agent先讀過文件再來寫,格式一定會對,尤其是shot格式寫對就能控制到毫秒級,這大抵不是人類能勝任的作業,我的作法是Opus5搞定切片和shot架構,然後分派作業給gemini用vision抽幀寫每個shot的prompts, 6個frame抽一幀會比較準確

寫prompts是細緻活且量很大, Opus5原本擬了四案:

A.他自己寫整首mv,約需40萬Token

B.他開sub agent給Sonnet5去寫, 便宜一半

C.交給本地AI(Qwen3.8-27b)來寫,實測寫了幾段但都沒通過Opus5制定的基準線,

從這裡可以知道如果不寫詳細prompts,那就是H3自帶的qwen寫,結果不可能太好

D.交給gemini寫,實測結果Opus認為gemini寫得比qwen好很多,甚至比他自己寫得還好,而gemini接到工作委派後自己開了6個sub-agent全速寫

7.請agent寫一個小工具來讓你分析,實際畫面:

右上角: seg shot 鏡頭 frame

以這首mv為例, 一共38個 segments (198 shots), 幾乎是一秒鐘就切一次鏡頭,

一開始使用上次的一個prompt共用到底來測試,結果輸出畫面模糊到無法看,還產

生大量的動態斜紋,推測原因是在快速鏡頭切換又沒有錨定的情況下H3的推測產生

太多可能且無法收斂; 在沒有改任何參數的情況下只是切分shots和寫prompts就解決了這個問題;

8.那麼prompts要寫到多細呢? 我做了非常多測試,同一個shot下

不寫: H3自由發揮結果亂演

寫個大概: 那就演個大概,細節都沒有

寫和畫面不同: 演你寫的,不理畫面

寫和畫面相同而且細節都有寫到: 忠實呈現原場景

實際貼一小段shot prompt當參考:

[Shot 1] A waist-up close shot frames <Subject 1> at center frame, her head in the upper third of the frame, her shoulders spanning most of the frame width and the bottom frame line at her waist so her legs are never visible, with the illuminated clock window and its iron gears filling the background behind her, one member of <Subject 2> visible waist-up toward the left side of the frame, and another member of <Subject 2> partially visible in the background toward the right. <Subject 1> smiles toward the camera as she sings, raising her hand toward the left side of the frame to her lips with the index finger held upright in a hushing gesture, while her other hand rests on her hip with the elbow pointing toward the right side of the frame; the member of <Subject 2> toward the left performs the same hushing gesture on the same beat. <Subject 1> then lowers her raised hand down toward her side while keeping a soft smile facing the camera. The camera holds a static shot.[Shot 2] At 00:00.958, the shot cuts to a wide shot of the whole hall filmed from across the room, with <Subject 1> small at center frame, her head near the middle of the frame and her feet well above the bottom edge leaving a wide expanse of plank floor below, and the plank floor, both side walls, the wooden staircase, the arched windows, the timber scaffolding and the roof beams all inside the frame, with two members of <Subject 2> on each side of her in one horizontal line and additional members near the scaffolding along both side walls. All five figures, including <Subject 1> and the four members of <Subject 2>, raise both arms in unison with elbows bent outward at shoulder height while kicking one leg straight out toward the right side of the frame with a pointed toe tapping the floor, holding that stepping kick in place. The camera holds a static shot.

2個shots寫這麼長交待了了多久的內容呢? 實際上只有1.5秒

所以我認為這只適合交給AI去寫,甚至AI也不可能一次就全寫對,畢竟也是抽幀用vision分析,不像人類是連續視覺,需要先在0.2mp下輸出看過後再跟AI說哪段沒寫好讓他改,等prompts確認過都沒問題後再上到0.7mp一口氣生成,所以非常需要第7點的工具,否則幾乎無法debug

至於抽seed實測在這個方法下不太重要,prompts約束死無論換什麼seed輸出結果都幾乎相同,當然很細微的地方還是會有差異,但影響不大

--