Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

jf audit fail to scan pip dependencies #197

Open
chelyshev opened this issue Sep 30, 2024 · 1 comment
Open

jf audit fail to scan pip dependencies #197

chelyshev opened this issue Sep 30, 2024 · 1 comment
Labels
bug Something isn't working

Comments

@chelyshev
Copy link

Describe the bug

if repo contain Python code with pip dependencies, xRay CLI failed to build dependency tree.

Current behavior

$ jf audit

16:18:26 [Debug] JFrog CLI version: 2.70.0
16:18:26 [Debug] OS/Arch: linux/amd64     
16:18:26 [Debug] Trace ID for JFrog Platform logs: dcba57f925c057e5
16:18:26 [Debug] Sending HTTP GET request to: https://<jfrog-server>/xsc/api/v1/system/version
16:18:26 [🔵Info] Log path: /home/<user>/.jfrog/logs/jfrog-cli.2024-09-30.16-18-26.2746758.log
💬The full scan results are available here: /tmp/jfrog.cli.temp.-1727708651-272837479

Vulnerable Dependencies                        
┌─────────────────────────────────────────────┐
│ ✨ No vulnerable dependencies were found ✨ │  
└─────────────────────────────────────────────┘
                                               
Secret Detection                               
┌──────────┬───────────────────┬─────────────┬─────────────────┐
│ SEVERITY │ FILE              │ LINE:COLUMN │ SECRET          │
├──────────┼───────────────────┼─────────────┼─────────────────┤
│ 🎃Medium │ ******** │ ***      │ ************ │
└──────────┴───────────────────┴─────────────┴─────────────────┘

Infrastructure as Code Vulnerabilities
┌────────────────────────────────────────────────────────────┐
│ ✨ No Infrastructure as Code vulnerabilities were found ✨ │
└────────────────────────────────────────────────────────────┘

Static Application Security Testing (SAST)
┌─────────────────────────────────────────────────────────────────────────┐
│ ✨ No Static Application Security Testing vulnerabilities were found ✨ │
└─────────────────────────────────────────────────────────────────────────┘
17:04:11 [Debug] Sending an error report to JFrog analytics...
17:04:11 [Debug] Sending HTTP GET request to: https://<jfrog-server>/xsc/api/v1/system/version
17:04:11 [Debug] failed to check availability of Xsc service:server response: 404 Not Found
404 page not found

Reporting to JFrog analytics is skipped...
17:04:11 [🔵Info] Trace ID for JFrog Platform logs: dcba57f925c057e5
17:04:11 [🚨Error] audit command in '/opt/repos/<repo-name>' failed:
[Thread 0]  Xray dependency tree scan request on 'pip' failed:
scanning pip dependencies failed with error: Get Dependencies Scan results...  executor timeout after 540 attempts with 5000 milliseconds wait intervals

$ cat /home//.jfrog/logs/jfrog-cli.2024-09-30.16-18-26.2746758.log

[Debug] Analytics metrics are disabled, skipping sending event request to XSC
[Debug] Sending HTTP GET request to: https://<jfrog-server>/xray/api/v1/system/version
[Debug] Usage Report: Sending info...
[Debug] Sending HTTP GET request to: https://<jfrog-server>/artifactory/api/system/version
[Debug] Sending HTTP GET request to: https://<jfrog-server>/xray/api/v1/entitlements/feature/contextual_analysis
[Debug] Artifactory response: 200
[Debug] JFrog Artifactory version is: 7.90.9
[Debug] Sending HTTP POST request to: https://<jfrog-server>/artifactory/api/system/usage
[Debug] Sending HTTP GET request to: https://<jfrog-server>/xray/api/v1/configuration/jas
[Debug] The path '/opt/repos/<repo-name>/.git/COMMIT_EDITMSG' is excluded
[Debug] The path '/opt/repos/<repo-name>/.git/FETCH_HEAD' is excluded
...
[Debug] The path '/opt/repos/<repo-name>/.gitignore' is excluded
[Debug] The path '/opt/repos/<repo-name>/.gitlab-ci.yml' is excluded
[Debug] The path '/opt/repos/<repo-name>/pytest.ini' is excluded
[Debug] The path '/opt/repos/<repo-name>/tests/test_func.py' is excluded
[Debug] mapped 1 working directories with indicators/descriptors:
{
  "/opt/repos/<repo-name>": [
    "/opt/repos/<repo-name>/requirements.txt"
  ]
}
[Debug] Detected 1 technologies at /opt/repos/<repo-name>: [pip].
[Info] Preforming 1 SCA scans:
[
  {
    "Target": "/opt/repos/<repo-name>",
    "Technology": "pip",
    "Descriptors": [
      "/opt/repos/<repo-name>/requirements.txt"
    ]
  }
]
[Debug] No pip.yaml configuration file was found. Resolving dependencies from pip default registry
[Info] Calculating Pip dependencies...
[Debug] Running /usr/bin/python3 -m venv venvdir
[Debug] Used "/usr/bin/python3" version: Python 3.12.4
[Debug] Failed running python venv: "/usr/bin/python3 -m venv venvdir" command failed: exit status 1 - Error: Command '['/tmp/jfrog.cli.temp.-1727705906-972550765/venvdir/bin/python3', '-m', 'ensurepip', '--upgrade', '--default-pip']' returned non-zero exit status 1.

[Debug] Running virtualenv -p /usr/bin/python3 venvdir
[Debug] Running python -m pip install .
[Debug] Used "python" version: Python 3.12.4
[Debug] Running python -m pip install -r requirements.txt
[Debug] Created 'Pip' dependency tree with 19 nodes. Elapsed time: 9.3 seconds.
[Debug] Unique dependencies list:
[
    "pypi://oauthlib:3.2.2",
    "pypi://pyyaml:6.0.2",
    "pypi://urllib3:2.2.3",
    "pypi://charset-normalizer:3.3.2",
    "pypi://websocket-client:1.8.0",
    "pypi://pip:24.2",
    "pypi://certifi:2024.8.30",
    "pypi://durationpy:0.7",
    "pypi://cachetools:5.5.0",
    "pypi://pyasn1-modules:0.4.1",
    "pypi://rsa:4.9",
    "pypi://kubernetes:31.0.0",
    "pypi://pyasn1:0.6.1",
    "pypi://python-dateutil:2.9.0.post0",
    "pypi://six:1.16.0",
    "pypi://google-auth:2.35.0",
    "pypi://requests:2.32.3",
    "pypi://idna:3.10",
    "pypi://requests-oauthlib:2.0.0"
  ]
[Info] [Thread 0] Running SCA scan for /opt/repos/<repo-name> vulnerable dependencies in /opt/repos/<repo-name> directory...
[Info] Scanning 19 pip dependencies...
[Debug] JFROG_CLI_RELEASES_REPO is not set
[Debug] 'JFROG_CLI_RELEASES_REPO' environment variable is not configured. The Analyzer Manager app will be downloaded directly from releases.jfrog.io if needed.
[Debug] Sending HTTP POST request to: https://<jfrog-server>/xray/api/v1/scan/graph?scan_type=dependency
[Debug] Sending HTTP HEAD request to: https://releases.jfrog.io/artifactory/xsc-gen-exe-analyzer-manager-local/v1/1.9.3/linux-amd64/analyzerManager.zip
[Info] Waiting for scan to complete on JFrog Xray...
[Debug] Sending HTTP GET request to: https://<jfrog-server>/xray/api/v1/scan/graph/d0c6dfb3-1feb-4899-50f8-4492df094287?include_vulnerabilities=true
[Debug] Get Dependencies Scan results... (Attempt 1)
[Debug] Artifactory response: 200
[Info] [Thread 2] Running IaC scan...
[Info] [Thread 1] Running secrets scan...
[Debug] Secrets scanner input YAML:
scans:
    - roots:
        - /opt/repos/<repo-name>
      output: /tmp/jfrog.cli.temp.-1727705906-2351225783/Secrets_1727705916/results.sarif
      type: secrets-scan
      skipped-folders:
        - '**/*.git*/**'
        - '**/*node_modules*/**'
        - '**/*target*/**'
        - '**/*venv*/**'
        - '**/*test*/**'

[Debug] IaC scanner input YAML:
scans:
    - roots:
        - /opt/repos/<repo-name>
      output: /tmp/jfrog.cli.temp.-1727705906-2351225783/IaC_1727705916/results.sarif
      type: iac-scan-modules
      skipped-folders:
        - '**/*.git*/**'
        - '**/*node_modules*/**'
        - '**/*target*/**'
        - '**/*venv*/**'
        - '**/*test*/**'

[Debug] Executing /home/<user>/.jfrog/dependencies/analyzerManager/analyzerManager sec /tmp/jfrog.cli.temp.-1727705906-2351225783/Secrets_1727705916/config.yaml
[Debug] Executing /home/<user>/.jfrog/dependencies/analyzerManager/analyzerManager iac /tmp/jfrog.cli.temp.-1727705906-2351225783/IaC_1727705916/config.yaml
[Debug] Sending HTTP GET request to: https://<jfrog-server>/xray/api/v1/scan/graph/d0c6dfb3-1feb-4899-50f8-4492df094287?include_vulnerabilities=true
[Debug] Get Dependencies Scan results... (Attempt 2)
[Info] [Thread 2] Found 0 IaC vulnerabilities
[Info] [Thread 2] Running SAST scan...
[Debug] Sast scanner input YAML:
scans:
    - roots:
        - /opt/repos/<repo-name>
      type: sast
      exclude_patterns:
        - '**/*.git*/**'
        - '**/*node_modules*/**'
        - '**/*target*/**'
        - '**/*venv*/**'
        - '**/*test*/**'

[Debug] Executing /home/<user>/.jfrog/dependencies/analyzerManager/analyzerManager zd /tmp/jfrog.cli.temp.-1727705906-2351225783/Sast_1727705925/config.yaml /tmp/jfrog.cli.temp.-1727705906-2351225783/Sast_1727705925/results.sarif
[Debug] Sending HTTP GET request to: https://<jfrog-server>/xray/api/v1/scan/graph/d0c6dfb3-1feb-4899-50f8-4492df094287?include_vulnerabilities=true
[Debug] Get Dependencies Scan results... (Attempt 3)
[Info] [Thread 1] Found 1 secrets vulnerabilities
[Debug] Sending HTTP GET request to: https://<jfrog-server>/xray/api/v1/scan/graph/d0c6dfb3-1feb-4899-50f8-4492df094287?include_vulnerabilities=true
[Debug] Get Dependencies Scan results... (Attempt 4)
[Info] [Thread 2] Found 0 SAST vulnerabilities
[Debug] Sending HTTP GET request to: https://<jfrog-server>/xray/api/v1/scan/graph/d0c6dfb3-1feb-4899-50f8-4492df094287?include_vulnerabilities=true
[Debug] Get Dependencies Scan results... (Attempt 5)
[Debug] Sending HTTP GET request to: https://<jfrog-server>/xray/api/v1/scan/graph/d0c6dfb3-1feb-4899-50f8-4492df094287?include_vulnerabilities=true
[Debug] Get Dependencies Scan results... (Attempt 6)
[Debug] Sending HTTP GET request to: https://<jfrog-server>/xray/api/v1/scan/graph/d0c6dfb3-1feb-4899-50f8-4492df094287?include_vulnerabilities=true
[Debug] Get Dependencies Scan results... (Attempt 7)
[Debug] Sending HTTP GET request to: https://<jfrog-server>/xray/api/v1/scan/graph/d0c6dfb3-1feb-4899-50f8-4492df094287?include_vulnerabilities=true
[Debug] Get Dependencies Scan results... (Attempt 8)
[Debug] Sending HTTP GET request to: https://<jfrog-server>/xray/api/v1/scan/graph/d0c6dfb3-1feb-4899-50f8-4492df094287?include_vulnerabilities=true
[Debug] Get Dependencies Scan results... (Attempt 9)
......
[Debug] Sending HTTP GET request to: https://<jfrog-server>/xray/api/v1/scan/graph/d0c6dfb3-1feb-4899-50f8-4492df094287?include_vulnerabilities=true
[Debug] Get Dependencies Scan results... (Attempt 540)
[Debug] Sending HTTP GET request to: https://<jfrog-server>/xray/api/v1/scan/graph/d0c6dfb3-1feb-4899-50f8-4492df094287?include_vulnerabilities=true
[Debug] Get Dependencies Scan results... (Attempt 541)
[Debug] [Thread 1]  We couldn't find any vulnerable dependencies. Skipping....
[Debug] Analytics metrics are disabled, skipping sending update event request to XSC

Reproduction steps

jf audit in repo with Python code, or run Frogbot in CI-pipeline

Expected behavior

No response

JFrog CLI-Security version

JFrog CLI version: 2.70.0, Frogbot version: 2.21.13

JFrog CLI version (if applicable)

2.70.0

Operating system type and version

ubuntu 24.04

JFrog Xray version

No response
@chelyshev chelyshev added the bug Something isn't working label Sep 30, 2024
@chelyshev chelyshev changed the title jfrog fail to scan pip dependencies jf audit fail to scan pip dependencies Sep 30, 2024
@hadarshjfrog
Copy link
Contributor

Hi @chelyshev - Thanks for reporting the issue. It does look like there's an availability issue with the xray server. We'll check internally and update ASAP

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@chelyshev @hadarshjfrog