How to properly disassemble selenium/video to generate evidence in gitlab-ci?

[alexandremariano4]

I have the following test scenario:

I have a gitlab that is using runner that I have configured on my machine, so the configuration is something simple, it is being done as follows:

concurrent = 1
check_interval = 0
shutdown_timeout = 0

[session_server]
  session_timeout = 1800

[[runners]]
  name = "myname"
  url = "https://gitlab.com"
  id = myid
  token = "mytoken"
  token_obtained_at = 2023-05-31T11:15:39Z
  token_expires_at = 0001-01-01T00:00:00Z
  executor = "docker"
  [runners.cache]
    MaxUploadedArchiveSize = 0
  [runners.docker]
    tls_verify = false
    image = "alpine"
    privileged = true
    disable_entrypoint_overwrite = false
    oom_kill_disable = false
    disable_cache = false
    volumes = ["/cache"]
    shm_size = 2000000000

And my gitlab-ci.yml file, which is basically my pipelnie, looks like this

stages:
  - "e2e"

e2e:
  image: python
  stage: e2e
  services:
    - name: selenium/standalone-chrome
      alias: browser
    - name: selenium/video
      alias: video

  variables:
    FF_NETWORK_PER_BUILD: "true"
    SE_VIDEO_FOLDER: "/cache"
    SE_VIDEO_FILE_NAME: 'video.mp4'
    DISPLAY_CONTAINER_NAME: browser

  before_script: 
    - python3 -m venv .venv
    - source .venv/bin/activate
    - pip3 install poetry
    - poetry install --only main
    - cd project/

  script:
    - python test.py

after_script:
    - mkdir -p ${CI_PROJECT_DIR}/evidency
    - cp -r /cache/video.mp4 ${CI_PROJECT_DIR}/evidency

  artifacts:
    when: on_failure
    paths:
      - ./evidency
    expire_in: 1 week

Using this way, I'm getting the audio file to be correctly sent to my project folder so that I can use it as evidence and later visualization in case of errors.

However, the file is coming in at 256KB, which basically means that there is no content in this video file. From what I've investigated, I've come to the conclusion that the video container that runs inside Runner wasn't having a "soft drop", I don't know what expression to use for that.

But basically, while on my machine I could use a docker compose file to do "docker compose up -d" and then "docker compose down" which gives the container time to shut down and compile the video correctly, when this scenario goes to gitlab's Runner this no longer happens, as it is shut down abruptly.

---

In the image below you can see the exact moment when the local container is closed and selenium/video has time to correctly create the .mp4 file

---

Now, on the other hand, when the selenium/video container that is running in the gitlab runner is shut down, it shuts down dramatically and consequently doesn't mount the video correctly.

---

Has anyone had this problem and managed to solve it in any way other than "docker:dind"?

[VietND96]

May I know video image tag is used in this setup?

[alexandremariano4]

Im not using any tag.

The example that i'm put here is exactly what i'm using.

I think this problem is because the container of selenium video is not shutdown propertly.
But i dont have any idea what i need to change in my gitlabci. :(

[VietND96]

So, do you know the last version that worked properly in your env?
Our CI also has tests for video recording and checking video integrity, it passed https://github.com/SeleniumHQ/docker-selenium/actions/runs/9121853320

[VietND96]

@alexandremariano4, I would suggest a thing, can you try and confirm
Remove this SE_VIDEO_FOLDER: "/cache" from environment. That means unset it
Correct this step in after_script cp -r /videos/video.mp4 ${CI_PROJECT_DIR}/evidency

[alexandremariano4]

I used the image provided (6.1.1), and also made a change.
I didn't know this before, but now I know that "/builds" is shared between all containers, so I used it to correctly capture the artifact and no need change nothing in runner config.toml , and the result I'm getting is unfortunately still the same.

stages:
  - "e2e"

e2e:
  image: python
  stage: e2e
  cache: []
  services:
    - name: selenium/standalone-chrome
      alias: browser
    - name: selenium/video:ffmpeg-6.1.1-20240517
      alias: video


  variables:
    FF_NETWORK_PER_BUILD: "true"
    SE_VIDEO_FOLDER: "/builds"
    SE_VIDEO_FILE_NAME: 'chromevideo.mp4'
    DISPLAY_CONTAINER_NAME: browser

  before_script: 
    - python3 -m venv .venv
    - source .venv/bin/activate
    - pip3 install poetry
    - poetry install --only main
    - cd project/

  script:
    - python teste.py
  
  after_script:
    - mkdir -p ${CI_PROJECT_DIR}/evidency
    - mv -f /builds/chromevideo.mp4 ${CI_PROJECT_DIR}/evidency



  artifacts:
    when: always
    paths:
      - ${CI_PROJECT_DIR}/evidency
    expire_in: 1 week

Sempre o vídeo é gravado e mantém desta forma:

So, if I try for some reason to download it to run, the file simply doesn't open, which leads me to believe that the service (selenium/video) is not having time to correctly generate the video.
I did some research on something like:
SIGKILL/SIGTERM/SIGINT.

And from what I saw, it seems that when I send the SIGINT command which is composed of "kill -2 $PID", the container is closed as seen locally, reporting a "keyboard interrupt", but there is no way to send this command, so I imagine it would be otherwise.

[alexandremariano4]

Just for note, i don't know if this is relevant , my 'test.py' file where i run the script is like this

from selenium.webdriver import Remote, ChromeOptions
from time import sleep
browser_option = ChromeOptions()

#Selenium Grid
browser_option.page_load_strategy = 'eager'
browser_option.set_capability('browserName','chrome')
browser_option.set_capability('platformName','LINUX')


brw = Remote(
    command_executor='http://browser:4444/wd/hub',
    options=browser_option
)

brw.get('https://google.com.br')
brw.maximize_window()
print(brw.title)
brw.quit()
raise Exception

This code raises a Exception just for validate if his really saving a video with a condition that is a failure in my code

[pitervergara]

I had a similar issue in (selenium-video:ffmpeg-7.0.1-20240820) resulting in corrupted video. It seems that the problem is really with the shutdown of the video service container.

What I did as a workaround was to manually kill the video container inside the script step of my job, after my tests are finished:

  variables:
    FF_NETWORK_PER_BUILD: 1
    DISPLAY_CONTAINER_NAME: chrome
    SE_FRAME_RATE: 22
    SE_VIDEO_RECORD_STANDALONE: true
    FILE_NAME: chrome_${CI_COMMIT_REF_SLUG}_${CI_COMMIT_SHORT_SHA}.mp4
    VIDEO_FOLDER: /cache/videos

  services:
    - name: selenium/standalone-chrome:127.0-chromedriver-127.0-20240820
      alias: chrome
    - name: selenium-video:ffmpeg-7.0.1-20240820_force-recording 
      alias: chrome_video

  artifacts:
    paths:
      - ${CI_PROJECT_DIR}/${FILE_NAME}
      - /builds/sistemas/myproject/target/surefire-reports
    when: always

  script:
    - |
      # ...
      mvn $MAVEN_CLI_OPTS test || true

      # Stops the video container so the .mp4 is finished and writed to the disk
      VIDEO_SERVICE_CONTAINER_ID=$(docker ps  -f "label=com.gitlab.gitlab-runner.job.id=${CI_JOB_ID}" -f name=selenium-video -q)
      docker stop ${VIDEO_SERVICE_CONTAINER_ID} && docker rm ${VIDEO_SERVICE_CONTAINER_ID}

      cp -v ${VIDEO_FOLDER}/${FILE_NAME} ${CI_PROJECT_DIR}/${FILE_NAME}

It works for now, but the video actually has a long blackscreen at the start, because the container starts to record as soon as it starts. So it's definitlly not a long term solution.

[VietND96]

@pitervergara, can you try video image tag 4.24.0-20240830 to verify it can be resolved?