Demo_MultiPerson: setting: multi_person = false

[jtoerber]

Hi,

tried your Demo_MultiPerson. Unchanged everything works fine.
When I change multi_person to false I get the following error during marker augmentation:
Augmented marker coordinates are stored at C:\Users<user>.conda\envs\Pose2Sim\Lib\site-packages\Pose2Sim\Demo_MultiPerson\pose-3d\Demo_MultiPerson_0-60_filt_butterworth_LSTM.trc.
Augmented trc files have been converted to c3d.
Traceback (most recent call last):
File "c:\Users<user>.conda\envs\Pose2Sim\Lib\site-packages\Pose2Sim\Demo_MultiPerson\test.py", line 27, in
Pose2Sim.markerAugmentation()
File "C:\Users<user>.conda\envs\Pose2Sim\lib\site-packages\Pose2Sim\Pose2Sim.py", line 461, in markerAugmentation
augmentTRC(config_dict)
File "C:\Users<user>.conda\envs\Pose2Sim\lib\site-packages\Pose2Sim\markerAugmentation.py", line 129, in augmentTRC
pathInputTRCFile = trc_files[p]
IndexError: list index out of range

Event the synchronization seems to have different "offsets".

Comment in the Config.toml sample file:
multi_person = true # .... If false, only the main person in scene is analyzed (and it run much faster).

What is meant by "main person"? Person0 from first camera/video?
I thought I would get a dialog to select the main person.
(E.g. MoveNet says it takes the person nearest to the camera)

[davidpagnon]

According to the doc,

If multi_person is set to false, the algorithm chooses the person for whom the reprojection error is smallest.
If multi_person is set to true, it associates across views the people for whom the distances between epipolar lines are the smallest. People are then associated across frames according to their displacement speed.

This is likely the person who is in the center of the scene, correctly seen by all cameras. But you are right, it should be made clearer in the Config files as well. I changed it to "If false, only the person with lowest reprojection error is analyzed."

---

Indeed, your problem comes from synchronization. In single_person mode, there is supposed to be one single person is in the field of view (potentially a few others in the background). The problem here is that there are two very outstanding people, and in some views the synthetic one has a #0 index while in some others its index is #1. This messes up with synchronization, and then triangulation does not work, and so on with the later stages.

In the future, I think I'll change the multiperson parameter to:

multiperson = true # true, or 'single_auto', or 'single_click'. 'single_auto' selects the person with lowest reprojection error, and 'single_click' lets the user manually select the person of interest.

[jtoerber]

In training we have influence on our Athletes e.g. color of their clothes. So a difficulty hierarchy could be Like:

1. One Person
2. Table Tennis or Tennis Single
3. At least one of several cameras separates two persons
4. Judo: one Fighter is blue, one white
5. Soccer: at least the Team is recognizable
6. Arbitrary people:
   a fast moving If temporarily occluded
   b slow moving If temporarily occluded
   c Many occlusions and slow to no movement
   So why not do Person recognition based on colors?
   Btw. support for Tracking a ball or other devices would be nice.