Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[IDEA] feat: support PEP-770 - bundled (phantom) dependencies #831

Open
jkowalleck opened this issue Nov 15, 2024 · 1 comment
Open

[IDEA] feat: support PEP-770 - bundled (phantom) dependencies #831

jkowalleck opened this issue Nov 15, 2024 · 1 comment
Labels
enhancement New feature or request source: environment

Comments

@jkowalleck
Copy link
Member

jkowalleck commented Nov 15, 2024 •
edited
Loading

based on https://sethmlarson.dev/early-promising-results-with-sboms-and-python-packages
contact @sethmlarson

PEP: https://peps.python.org/pep-0770/
PEP discussion: https://discuss.python.org/t/pep-770-improving-measurability-of-python-packages-with-software-bill-of-materials/76308

goal

gather the declaration of bundled dependencies of a package, by reading its shipped SBOMs.

Warning

the PEP 770 is stil a draft, so it is unclear how declared shipped SBOMs may be detected ...

expected outcome:

  • bundled dependencies are listed as sub-components of their component in the SBOM result
  • each declared bundled dependency is present in the dependency graph-
  • dependency graph, if present, of such bundled dependencies is carried over into the SBOM result

followup

after implementing this, update the benchmark call andresults in https://github.com/psf/sboms-for-python-packages/tree/main/benchmark

example result

JSON based on a demo-SBOM for Pillow==11.1.0 https://gist.github.com/sethmlarson/9b87245c99147815e8e18901f4a10444

example JSON 
{
    "$schema": "http://cyclonedx.org/schema/bom-1.6.schema.json",
    "bomFormat": "CycloneDX",
    "specVersion": "1.4",
    "metadata": {
        "component": {
            "type": "application",
            "name": "my-app",
            "version": "0.13.37",
            "bom-ref": "my-app"
        }
    },
    "components": [
        {
            "type": "library",
            "bom-ref": "pillow==11.1.0",
            "name": "Pillow",
            "version": "11.1.0",
            "components": [
                {
                    "type": "library",
                    "bom-ref": "pillow==11.1.0|pkg:rpm/almalinux/[email protected]?distro=almalinux-8",
                    "name": "libXau",
                    "version": "1.0.9-3.el8",
                    "purl": "pkg:rpm/almalinux/[email protected]?distro=almalinux-8"
                },
                {
                    "type": "library",
                    "bom-ref": "pillow==11.1.0|pkg:rpm/almalinux/[email protected]?distro=almalinux-8",
                    "name": "jbigkit-libs",
                    "version": "2.1-14.el8",
                    "purl": "pkg:rpm/almalinux/[email protected]?distro=almalinux-8"
                },
                {
                    "type": "library",
                    "bom-ref": "pillow==11.1.0|pkg:rpm/almalinux/[email protected]_10?distro=almalinux-8",
                    "name": "libtiff",
                    "version": "4.0.9-33.el8_10",
                    "purl": "pkg:rpm/almalinux/[email protected]_10?distro=almalinux-8"
                },
                {
                    "type": "library",
                    "bom-ref": "pillow==11.1.0|pkg:rpm/almalinux/[email protected]?distro=almalinux-8",
                    "name": "libxcb",
                    "version": "1.13.1-1.el8",
                    "purl": "pkg:rpm/almalinux/[email protected]?distro=almalinux-8"
                },
                {
                    "type": "library",
                    "bom-ref": "pillow==11.1.0|pkg:rpm/almalinux/[email protected]?distro=almalinux-8",
                    "name": "openjpeg2",
                    "version": "2.4.0-5.el8",
                    "purl": "pkg:rpm/almalinux/[email protected]?distro=almalinux-8"
                },
                {
                    "type": "library",
                    "bom-ref": "pillow==11.1.0|pkg:rpm/almalinux/[email protected]?distro=almalinux-8",
                    "name": "libjpeg-turbo",
                    "version": "1.5.3-12.el8",
                    "purl": "pkg:rpm/almalinux/[email protected]?distro=almalinux-8"
                },
                {
                    "type": "library",
                    "bom-ref": "pillow==11.1.0|pkg:rpm/almalinux/[email protected]?distro=almalinux-8",
                    "name": "lcms2",
                    "version": "2.9-2.el8",
                    "purl": "pkg:rpm/almalinux/[email protected]?distro=almalinux-8"
                },
                {
                    "type": "library",
                    "bom-ref": "pillow==11.1.0|pkg:rpm/almalinux/[email protected]?distro=almalinux-8",
                    "name": "bzip2-libs",
                    "version": "1.0.6-26.el8",
                    "purl": "pkg:rpm/almalinux/[email protected]?distro=almalinux-8"
                },
                {
                    "type": "library",
                    "bom-ref": "pillow==11.1.0|pkg:rpm/almalinux/[email protected]?distro=almalinux-8",
                    "name": "libpng",
                    "version": "1.6.34-5.el8",
                    "purl": "pkg:rpm/almalinux/[email protected]?distro=almalinux-8"
                },
                {
                    "type": "library",
                    "bom-ref": "pillow==11.1.0|pkg:rpm/almalinux/[email protected]?distro=almalinux-8",
                    "name": "freetype",
                    "version": "2.9.1-9.el8",
                    "purl": "pkg:rpm/almalinux/[email protected]?distro=almalinux-8"
                },
                {
                    "type": "library",
                    "bom-ref": "pillow==11.1.0|pkg:rpm/almalinux/[email protected]_9.1?distro=almalinux-8",
                    "name": "libwebp",
                    "version": "1.0.0-9.el8_9.1",
                    "purl": "pkg:rpm/almalinux/[email protected]_9.1?distro=almalinux-8"
                },
                {
                    "type": "library",
                    "bom-ref": "pillow==11.1.0|pkg:rpm/almalinux/[email protected]_9.1?distro=almalinux-8",
                    "name": "libwebp",
                    "version": "1.0.0-9.el8_9.1",
                    "purl": "pkg:rpm/almalinux/[email protected]_9.1?distro=almalinux-8"
                },
                {
                    "type": "library",
                    "bom-ref": "pillow==11.1.0|pkg:rpm/almalinux/[email protected]_9.1?distro=almalinux-8",
                    "name": "libwebp",
                    "version": "1.0.0-9.el8_9.1",
                    "purl": "pkg:rpm/almalinux/[email protected]_9.1?distro=almalinux-8"
                }
            ],
        }
    ],
    "dependencies": [
        {
            "ref": "my-app",
            "dependsOn": [
                "pillow==11.1.0"
            ]
        },
        {
            "ref": "pillow==11.1.0",
            "dependsOn": [
                "pillow==11.1.0|pkg:rpm/almalinux/[email protected]?distro=almalinux-8",
                "pillow==11.1.0|pkg:rpm/almalinux/[email protected]?distro=almalinux-8",
                "pillow==11.1.0|pkg:rpm/almalinux/[email protected]_10?distro=almalinux-8",
                "pillow==11.1.0|pkg:rpm/almalinux/[email protected]?distro=almalinux-8",
                "pillow==11.1.0|pkg:rpm/almalinux/[email protected]?distro=almalinux-8",
                "pillow==11.1.0|pkg:rpm/almalinux/[email protected]?distro=almalinux-8",
                "pillow==11.1.0|pkg:rpm/almalinux/[email protected]?distro=almalinux-8",
                "pillow==11.1.0|pkg:rpm/almalinux/[email protected]?distro=almalinux-8",
                "pillow==11.1.0|pkg:rpm/almalinux/[email protected]?distro=almalinux-8",
                "pillow==11.1.0|pkg:rpm/almalinux/[email protected]?distro=almalinux-8",
                "pillow==11.1.0|pkg:rpm/almalinux/[email protected]_9.1?distro=almalinux-8",
                "pillow==11.1.0|pkg:rpm/almalinux/[email protected]_9.1?distro=almalinux-8",
                "pillow==11.1.0|pkg:rpm/almalinux/[email protected]_9.1?distro=almalinux-8"
            ]
        },
        {
            "ref": "pillow==11.1.0|pkg:rpm/almalinux/[email protected]?distro=almalinux-8"
        },
        {
            "ref": "pillow==11.1.0|pkg:rpm/almalinux/[email protected]?distro=almalinux-8"
        },
        {
            "ref": "pillow==11.1.0|pkg:rpm/almalinux/[email protected]_10?distro=almalinux-8"
        },
        {
            "ref": "pillow==11.1.0|pkg:rpm/almalinux/[email protected]?distro=almalinux-8"
        },
        {
            "ref": "pillow==11.1.0|pkg:rpm/almalinux/[email protected]?distro=almalinux-8"
        },
        {
            "ref": "pillow==11.1.0|pkg:rpm/almalinux/[email protected]?distro=almalinux-8"
        },
        {
            "ref": "pillow==11.1.0|pkg:rpm/almalinux/[email protected]?distro=almalinux-8"
        },
        {
            "ref": "pillow==11.1.0|pkg:rpm/almalinux/[email protected]?distro=almalinux-8"
        },
        {
            "ref": "pillow==11.1.0|pkg:rpm/almalinux/[email protected]?distro=almalinux-8"
        },
        {
            "ref": "pillow==11.1.0|pkg:rpm/almalinux/[email protected]?distro=almalinux-8"
        },
        {
            "ref": "pillow==11.1.0|pkg:rpm/almalinux/[email protected]_9.1?distro=almalinux-8"
        },
        {
            "ref": "pillow==11.1.0|pkg:rpm/almalinux/[email protected]_9.1?distro=almalinux-8"
        },
        {
            "ref": "pillow==11.1.0|pkg:rpm/almalinux/[email protected]_9.1?distro=almalinux-8"
        }
    ]
}
@jkowalleck jkowalleck changed the title [IDEA] feat: bundled (phantom) dependencies [IDEA] feat: bundled (phantom) dependencies - PEP770 Jan 9, 2025
@jkowalleck
Copy link
Member Author

jkowalleck commented Jan 9, 2025 •
edited
Loading

per PEP770, pyrthon packages could ship their own SBOM - in any format.
so either we woudl need to "understand" any format, and wuld transform it into CDX, o the component can be enhanced with the info given by its shipped SBOM,. ...

— OR —

we could see whether to put that component's SBOM as a "attachment"
what we could do today:
add an external reference of type "bom" to that component, and the url woud not be a http-url, but a data url.

the mime-type is disputable. maybe add some logic for proper detection?

e.g. compressed + base64ed the SBOM from the original ticket above: data:aapplication/zip;base64,eJztWF1v2jAUfedXVGiPjUOAsoKEhrRpr9vTVGmaKicYapoPKzErbOO/7zomCVACtkmqVC1P4HtzzvHRzcW+rb+tK/i0PyTeAwlwe3TVfuCcjWzbW3t+FJLpCkXx3JZh240Cy0EDJH+iRRKF7WuJAKGvURxgLjA+y4e/3GXRhBHvB4kTCg9A3EH9LBIQjqeYC2qpJV31ooABQMj3ltMQXzMiMDBjPvUwp7mGPCXEQZoSrC3IOoz+LnR0kNNDvY+HGWKbMZntQOTxTfptsxWfy0wg92eeVKLYp26M4/UJNkZ9P3oajx0HOahTtq3vadaJbR1/vETscdGK4s9v4h97nI9iFtjYD7BPw+XKBqg7vJxAEA2tHiL+7acpTXgcjfMc67aMKDNBgpRl7ZpR0JRls2Xsp8oNpT5D3Twnejl7Fy6dP1JuAWQy6SLHcvqGHu8iKThdcOkbrSG6WW4DFKez2aQvSyStkXunY1bRAknB6AMuo7JWFd04t1eeC+8j9G2oEfPeAShKvaPgMXJZRWyzHI4YCReMzLvwFkJ5WDeGHuc4So0jZ9J3WVlws3wGKKHa4svYjaBGblDPcrrmBV2AKdV1QWdU1xrSG2a7FySiUoaWsdcCQamotxwGBp8X2SxX3T+UdeV/tzgqDazuwNDdAknxaJdx6busIbpZbgMUC+cTcRHr9Y0btERRcrngMWoWKmKb5fAsJkTAincQ/rqHhhZnMGrdIiPS91hVbrNMBqgn4rL05etI0fcg36yUBZJix9jhMipnVdHvbr+7/Xbc3lv5VXDLwHahPSVwWp+S0KPk5Mhsf/p2MMiSIMm38Ogc63CIti+s0HV9hvv0LE5LQ4UzsJrHPzXPO2q84Nd6s63/OlfTzaXmo3uNZ9U6D2k19+/XBF9NdzRucBUSmkyQK9mm9iS1ElatyeLljPpTtkp2aTJnqoBYY/5yOZvJHKISc7Xu5Zczat9SK9mkdiN8U6zycN7atP4D7RY7SA==

— OR —

@stevespringett

Page 55 of the SBOM Guide has an example you might be interested in. The example is for a threat model, but changing the external reference to a bom would also be possible.

{
  "bomFormat": "CycloneDX",
  "specVersion": "1.6",
  "serialNumber": "urn:uuid:3e671687-395b-41f5-a30f-a58921a69b79",
  "version": 1,
  "components": [
    {
      "bom-ref": "acme-application",
      "type": "application",
      "name": "Acme Application",
      "version": "1.0.0",
      "externalReferences": [
        {
          "type": "threat-model",
          "url": "urn:cdx:3e671687-395b-41f5-a30f-a58921a69b79/1#acme-threatmodel"
        }
      ]
    },
    {
      "bom-ref": "acme-threatmodel",
      "type": "data",
      "name": "Acme Threat Model",
      "scope": "excluded",
      "data": [
        {
          "type": "other",
          "contents": {
            "attachment": {
              "encoding": "base64",
              "contentType": "application/pdf",
              "content": "VGhyZWF0IG1vZGVsIGdvZXMgaGVyZQ=="
            }
          }
        }
      ]
    }
  ]
}

asked in slack for alternatives: https://cyclonedx.slack.com/archives/CVA0G10FN/p1736440976834859

@jkowalleck jkowalleck mentioned this issue Jan 28, 2025
Open
60 tasks
@jkowalleck jkowalleck changed the title [IDEA] feat: bundled (phantom) dependencies - PEP770 [IDEA] feat: support PEP 770 - bundled (phantom) dependencies Jan 29, 2025
@jkowalleck jkowalleck changed the title [IDEA] feat: support PEP 770 - bundled (phantom) dependencies [IDEA] feat: support PEP-770 - bundled (phantom) dependencies Jan 29, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
enhancement New feature or request source: environment
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@jkowalleck