Kobold Sentinel Attack V5
實(shí)驗(yàn)?zāi)繕?biāo):
1.設(shè)法用武器用力的打擊目標(biāo)
實(shí)驗(yàn)設(shè)計(jì):
1.任何弱點(diǎn)觸地皆失敗 (尾巴、武器和Calf並非弱點(diǎn))
2.當(dāng)武器OnCollisionEnter Player
會(huì)傳送collision.impulse
//impulseRewardCoef = 0.05f
public void HitWithWeapon(Vector3 impulse){if(!hitTarget){avgVelocity = velocityBuffer.GetSmoothVal();hitOnVelocity = avgVelocity.normalized;float reward = Vector3.ProjectOnPlane(impulse, hitOnVelocity).magnitude * impulseRewardCoef;lastReward += reward;totalReward += reward;AddReward( reward );arrivedMoment = Time.fixedTime;hitTarget = true;}}
3.
//Set: judge.endEpisode = true
//Set: judge.episodeLength = 3.3f
//Set: weapon, tail not weakness
//Set: useClampReward = true
if(weaknessOnGround)
{
if(inferenceMode)
{
brainMode = BrainMode.GetUp;
SetModel("KoboldGetUp", getUpBrain);
behaviorParameters.BehaviorType = BehaviorType.InferenceOnly;
}
else
{
// ===Train Attack===
if(!hitTarget)
{
float survivedTime = Time.fixedTime - arrivedMoment;
if(survivedTime < judge.episodeLength )
{
AddReward( (survivedTime - judge.episodeLength) * 0.1f );
}
}
judge.outLife++;
judge.Reset();
return;
//===Train Other===
// brainMode = BrainMode.GetUp;
// SetModel("KoboldGetUp", getUpBrain);
// behaviorParameters.BehaviorType = BehaviorType.InferenceOnly;
}
}
else if(koboldRoot.localPosition.y < -1f)
{
if(inferenceMode)
{
brainMode = BrainMode.GetUp;
SetModel("KoboldGetUp", getUpBrain);
behaviorParameters.BehaviorType = BehaviorType.InferenceOnly;
}
else
{
// ===Train Attack===
if(!hitTarget)
{
float survivedTime = Time.fixedTime - arrivedMoment;
if(survivedTime < judge.episodeLength )
{
AddReward( (survivedTime - judge.episodeLength) * 0.3f );
}
}
judge.outY++;
judge.Reset();
return;
// ===Train Other===
// brainMode = BrainMode.GetUp;
// SetModel("KoboldGetUp", getUpBrain);
// behaviorParameters.BehaviorType = BehaviorType.InferenceOnly;
}
}
/*else if( IsCollideWithBody() )
{
if(inferenceMode)
{
brainMode = BrainMode.GetUp;
SetModel("KoboldGetUp", getUpBrain);
behaviorParameters.BehaviorType = BehaviorType.InferenceOnly;
}
else
{
// ===Train Attack===
if(!hitTarget)
{
float survivedTime = Time.fixedTime - arrivedMoment;
if(survivedTime < judge.episodeLength )
{
AddReward( (survivedTime - judge.episodeLength) * 0.1f );
}
}
judge.outLife++;
judge.Reset();
return;
//===Train Other===
// brainMode = BrainMode.GetUp;
// SetModel("KoboldGetUp", getUpBrain);
// behaviorParameters.BehaviorType = BehaviorType.InferenceOnly;
}
}*/
else
{
if(hitTarget)
{
targetSmoothVelocity = targetVelocityBuffer.GetSmoothVal();
lastReward = Vector3.ProjectOnPlane(targetSmoothVelocity, hitOnVelocity).magnitude * 0.05f;
totalReward += lastReward;
AddReward(lastReward);
}
}
//大致來(lái)說(shuō)
在命中目標(biāo)時(shí)紀(jì)錄當(dāng)下自身的移動(dòng)方向
之後，獎(jiǎng)勵(lì)和移動(dòng)方向不一樣的敵人速度值
然後暫時(shí)取消不能碰撞敵人的限制
實(shí)驗(yàn)時(shí)間:
Step: 5e7
Time Elapsed: 100544s (27.93hr)
實(shí)驗(yàn)結(jié)果:
實(shí)驗(yàn)結(jié)果為...應(yīng)該算失敗，但有奇妙的收穫
尤其實(shí)驗(yàn)過(guò)程中看到了，狗頭人會(huì)用前踢把紅蓮踢到天空中，超級(jí)暴力耶
所以才注意到了問(wèn)題
"不同尺寸的狗頭人獎(jiǎng)勵(lì)上應(yīng)該要根據(jù)尺寸比例"
否則大體型的狗頭人本來(lái)就更容易使出更具威力的攻擊
然後狗頭人的前踢更明顯的凸顯了另一個(gè)問(wèn)題
狗頭人不知道水月的肢體位置，所以踢的效果很不穩(wěn)定，常常踢中又被自己的武器勾住
所以如果要以這種接近實(shí)戰(zhàn)類(lèi)型的訓(xùn)練，視線(xiàn)是需要的
但前幾天突然想到的靈感似乎效果不錯(cuò)
"獎(jiǎng)勵(lì) 和打擊瞬間移動(dòng)速度方向 不同的各種向量"
總和上述，目前對(duì)下個(gè)實(shí)驗(yàn)很樂(lè)觀
下個(gè)實(shí)驗(yàn)
1.目標(biāo)會(huì)被改為一顆懸浮球
--每次狗頭人重置也會(huì)重置
--只和武器碰撞
2.打擊後只追蹤0.5秒的目標(biāo)飛行速度
3.將狗頭人比例固定在1.2
前面實(shí)驗(yàn)已經(jīng)證實(shí)不同尺寸是可以一起訓(xùn)練的和練出泛用模型的，但在互動(dòng)上複雜度太高
因此先固定在1.2，等互動(dòng)技術(shù)突破再重新加入泛用目標(biāo)
