Evaluation metrics that do not require flow-ground-truth

[GG-Bonds]

I noticed that the model uses the flow truth value during evaluation.
However, in practice, we want to minimize our reliance on the truth value.
Does the author have a metric that doesn't require the truth value as input?

[Kin-Zhang]

Good point, actually, I think there are two cases:

• w. other gt, like object-level gt, then you can use chamfer distance inside object gt to evaluate the accumulation etc. It's more like a reconstruction. (similar to HiMo project)
• w/o any gt, I think it's might be tricky, maybe use some projection 3d to 2d and use 2d feature as one of metric etc.

I think it's interesting question, I will check later if I can merge above metrics in upcoming papers.

[GG-Bonds]

Yes. I believe that evaluations that do not rely on flow truth value ​​are more in line with the original intention of flow self-supervision.

[Kin-Zhang]

Hahah, Another issue also discussed about the evaluation in self-supervision.

But I think engineer and research (or review) is different, as recent review I received they might not believe the self-supervised metric you proposed and question about the ssl performance etc. But good catch, will update this thread in future if things goes well.

[turswiming]

Another method is to evaluate the rank of the scene flow. For a good scene flow in autonomous driving, the rank should be close to 2 × the number of dynamic objects. You can try it out.

[ajinkyakhoche]

@turswiming could you elaborate, what do you mean by the 'rank' of the scene flow? how to compute it?