Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Unable to use MinGW instead of MSVC for building packages on Windows #2337

Open
VDovidaytis-HORIS opened this issue Oct 18, 2024 · 3 comments
Open

Unable to use MinGW instead of MSVC for building packages on Windows #2337

VDovidaytis-HORIS opened this issue Oct 18, 2024 · 3 comments
Labels
question

Comments

@VDovidaytis-HORIS
Copy link

Your question:

Hello!

I am trying to contribute the Lets-Plot package to conda-forge.
That project is based on Kotlin Native with Python API.

The issue I am facing is that conda-build defaults to using MSVC on Windows, but the library needs to be built with MinGW, because Kotlin Native supports only MinGW as a build target for the Windows platform.

Even after configuring meta.yaml, bld.bat, and setup.py to explicitly call MinGW, conda-build continues to invoke MSVC.

Steps taken:

  • Modified bld.bat to configure MinGW.
  • Adjusted setup.py to link to MinGW binaries.
  • Set appropriate environment variables.
  • Despite these modifications, conda-build continues to use MSVC.

Is there an official way to force conda-build to use MinGW for Windows builds? Or are there any known workarounds or future plans to support this?
@mfansler
Copy link
Member

Do you have a PR on staged-recipes? One need not have a fully worked out recipe to start a draft and let others see how it fails.

All R packages build through MinGW, so maybe have a look at their recipes. Recently we've been transitioning to UCRT, so some of the recipes might be in flux. For example, here's a recent submission without UCRT.

I believe it's mostly a matter of using compiler('m2w64_cxx') (for example) instead of compiler('cxx').

@VDovidaytis-HORIS
Copy link
Author

Hello! Thank you for the reply!

Do you have a PR on staged-recipes?

I have just made one:

conda-forge/staged-recipes#27974

For example, here's conda-forge/staged-recipes#27938.

I tried many things from that repository, but they didn't help. I also noticed that there is a problem with the CI build of that PR on Windows. However, I am not sure if it’s caused by MinGW problems.

I believe it's mostly a matter of using compiler('m2w64_cxx') (for example) instead of compiler('cxx').

When I try something like {{ compiler('m2w64_cxx') }} instead of {{ compiler('cxx') }}, conda starts searching for the m2w64_cxx_win-64 package, which doesn't exist. This issue occurs with any attempt to use m2w64_... instead of standard compiler options. I even tried to create a m2w64_cxx_win-64 package that links to the original m2w64_cxx package locally, but it didn’t help override the MSVC call.

I tried to build conda packages locally on my Windows machine by running the command (according to the documentation):

conda build recipes\lets-plot\ -m .\.ci_support\win64.yaml

I also tried running just conda build:

conda build recipes\lets-plot\

But there was no difference.

Build machine info:

Edition Windows 11 Pro
Version 23H2
Installed on    ‎18.‎01.‎2023
OS build    22631.4317
Experience  Windows Feature Experience Pack 1000.22700.1041.0

Conda build environment info:

active environment : test-conda-build
    active env location : D:\Util\miniconda3\envs\test-conda-build
            shell level : 2
       user config file : C:\Users\Valentin Dovidaytis\.condarc
 populated config files :
          conda version : 24.7.1
    conda-build version : not installed
         python version : 3.12.4.final.0
                 solver : libmamba (default)
       virtual packages : __archspec=1=skylake
                          __conda=24.7.1=0
                          __win=0=0
       base environment : D:\Util\miniconda3  (writable)
      conda av data dir : D:\Util\miniconda3\etc\conda
  conda av metadata url : None
           channel URLs : https://repo.anaconda.com/pkgs/main/win-64
                          https://repo.anaconda.com/pkgs/main/noarch
                          https://repo.anaconda.com/pkgs/r/win-64
                          https://repo.anaconda.com/pkgs/r/noarch
                          https://repo.anaconda.com/pkgs/msys2/win-64
                          https://repo.anaconda.com/pkgs/msys2/noarch
          package cache : D:\Util\miniconda3\pkgs
                          C:\Users\Valentin Dovidaytis\.conda\pkgs
                          C:\Users\Valentin Dovidaytis\AppData\Local\conda\conda\pkgs
       envs directories : D:\Util\miniconda3\envs
                          C:\Users\Valentin Dovidaytis\.conda\envs
                          C:\Users\Valentin Dovidaytis\AppData\Local\conda\conda\envs
               platform : win-64
             user-agent : conda/24.7.1 requests/2.32.3 CPython/3.12.4 Windows/11 Windows/10.0.22631 solver/libmamba conda-libmamba-solver/24.7.0 libmambapy/1.5.8 aau/0.4.4 c/. s/. e/.
          administrator : False
             netrc file : None
           offline mode : False

Conda environment packages:

# packages in environment at D:\Util\miniconda3\envs\test-conda-build:
#
# Name                    Version                   Build  Channel
archspec                  0.2.3              pyhd8ed1ab_0    conda-forge
attrs                     24.2.0             pyh71513ae_0    conda-forge
beautifulsoup4            4.12.3             pyha770c72_0    conda-forge
boltons                   24.0.0             pyhd8ed1ab_0    conda-forge
brotli-python             1.1.0           py312h275cf98_2    conda-forge
bzip2                     1.0.8                h2466b09_7    conda-forge
ca-certificates           2024.8.30            h56e8100_0    conda-forge
certifi                   2024.8.30          pyhd8ed1ab_0    conda-forge
cffi                      1.17.1          py312h4389bb4_0    conda-forge
chardet                   5.2.0           py312h2e8e312_2    conda-forge
charset-normalizer        3.4.0              pyhd8ed1ab_0    conda-forge
click                     8.1.7           win_pyh7428d3b_0    conda-forge
colorama                  0.4.6              pyhd8ed1ab_0    conda-forge
conda                     24.9.2          py312h2e8e312_0    conda-forge
conda-build               24.9.0          py312h2e8e312_0    conda-forge
conda-index               0.5.0              pyhd8ed1ab_0    conda-forge
conda-libmamba-solver     24.9.0             pyhd8ed1ab_0    conda-forge
conda-package-handling    2.4.0              pyh7900ff3_0    conda-forge
conda-package-streaming   0.11.0             pyhd8ed1ab_0    conda-forge
distro                    1.9.0              pyhd8ed1ab_0    conda-forge
filelock                  3.16.1             pyhd8ed1ab_0    conda-forge
fmt                       11.0.2               h7f575de_0    conda-forge
frozendict                2.4.6           py312h4389bb4_0    conda-forge
h2                        4.1.0              pyhd8ed1ab_0    conda-forge
hpack                     4.0.0              pyh9f0ad1d_0    conda-forge
hyperframe                6.0.1              pyhd8ed1ab_0    conda-forge
idna                      3.10               pyhd8ed1ab_0    conda-forge
importlib_resources       6.4.5              pyhd8ed1ab_0    conda-forge
jinja2                    3.1.4              pyhd8ed1ab_0    conda-forge
jsonpatch                 1.33               pyhd8ed1ab_0    conda-forge
jsonpointer               3.0.0           py312h2e8e312_1    conda-forge
jsonschema                4.23.0             pyhd8ed1ab_0    conda-forge
jsonschema-specifications 2024.10.1          pyhd8ed1ab_0    conda-forge
krb5                      1.21.3               hdf4eb48_0    conda-forge
libarchive                3.7.4                haf234dc_0    conda-forge
libcurl                   8.10.1               h1ee3ff0_0    conda-forge
libexpat                  2.6.3                he0c23c2_0    conda-forge
libffi                    3.4.2                h8ffe710_5    conda-forge
libiconv                  1.17                 hcfcfb64_2    conda-forge
liblief                   0.14.1               he0c23c2_2    conda-forge
libmamba                  1.5.10               h81425b0_1    conda-forge
libmambapy                1.5.10          py312h643a1bd_1    conda-forge
libsolv                   0.7.30               hbb528cf_0    conda-forge
libsqlite                 3.47.0               h2466b09_0    conda-forge
libssh2                   1.11.0               h7dfc565_0    conda-forge
libxml2                   2.12.7               h0f24e4e_4    conda-forge
libzlib                   1.3.1                h2466b09_2    conda-forge
lz4-c                     1.9.4                hcfcfb64_0    conda-forge
lzo                       2.10              hcfcfb64_1001    conda-forge
m2-conda-epoch            20230914               0_x86_64    conda-forge
m2-msys2-runtime          3.4.9.1              hd8ed1ab_4    conda-forge
m2-patch                  2.7.6.2              hd8ed1ab_4    conda-forge
markupsafe                3.0.2           py312h31fea79_0    conda-forge
menuinst                  2.1.2           py312h275cf98_1    conda-forge
more-itertools            10.5.0             pyhd8ed1ab_0    conda-forge
openssl                   3.3.2                h2466b09_0    conda-forge
packaging                 24.1               pyhd8ed1ab_0    conda-forge
pip                       24.2               pyh8b19718_1    conda-forge
pkginfo                   1.11.2             pyhd8ed1ab_0    conda-forge
pkgutil-resolve-name      1.3.10             pyhd8ed1ab_1    conda-forge
platformdirs              4.3.6              pyhd8ed1ab_0    conda-forge
pluggy                    1.5.0              pyhd8ed1ab_0    conda-forge
psutil                    6.0.0           py312h4389bb4_2    conda-forge
py-lief                   0.14.1          py312h275cf98_2    conda-forge
pybind11-abi              4                    hd8ed1ab_3    conda-forge
pycosat                   0.6.6           py312he70551f_0    conda-forge
pycparser                 2.22               pyhd8ed1ab_0    conda-forge
pysocks                   1.7.1              pyh0701188_6    conda-forge
python                    3.12.7          hce54a09_0_cpython    conda-forge
python-libarchive-c       5.1             py312h2e8e312_1    conda-forge
python_abi                3.12                    5_cp312    conda-forge
pytz                      2024.2             pyhd8ed1ab_0    conda-forge
pyyaml                    6.0.2           py312h4389bb4_1    conda-forge
referencing               0.35.1             pyhd8ed1ab_0    conda-forge
reproc                    14.2.4.post0         hcfcfb64_1    conda-forge
reproc-cpp                14.2.4.post0         h63175ca_1    conda-forge
requests                  2.32.3             pyhd8ed1ab_0    conda-forge
ripgrep                   14.1.0               ha073cba_1    conda-forge
rpds-py                   0.20.0          py312h2615798_1    conda-forge
ruamel.yaml               0.18.6          py312h4389bb4_1    conda-forge
ruamel.yaml.clib          0.2.8           py312h4389bb4_1    conda-forge
setuptools                75.1.0             pyhd8ed1ab_0    conda-forge
soupsieve                 2.5                pyhd8ed1ab_1    conda-forge
tk                        8.6.13               h5226925_1    conda-forge
tqdm                      4.66.5             pyhd8ed1ab_0    conda-forge
truststore                0.9.2              pyhd8ed1ab_0    conda-forge
tzdata                    2024b                hc8b5060_0    conda-forge
ucrt                      10.0.22621.0         h57928b3_1    conda-forge
urllib3                   2.2.3              pyhd8ed1ab_0    conda-forge
vc                        14.3                h8a93ad2_22    conda-forge
vc14_runtime              14.40.33810         hcc2c482_22    conda-forge
vs2015_runtime            14.40.33810         h3bf8584_22    conda-forge
wheel                     0.44.0             pyhd8ed1ab_0    conda-forge
win_inet_pton             1.1.0              pyh7428d3b_7    conda-forge
xz                        5.2.6                h8d14728_0    conda-forge
yaml                      0.2.5                h8ffe710_2    conda-forge
yaml-cpp                  0.8.0                h63175ca_0    conda-forge
zipp                      3.20.2             pyhd8ed1ab_0    conda-forge
zstandard                 0.23.0          py312h7606c53_1    conda-forge
zstd                      1.5.6                h0ea2cb4_0    conda-forge

@VDovidaytis-HORIS
Copy link
Author

I have just made one:

conda-forge/staged-recipes#27974

As I expected, the CI build on the Windows server crashed at the extension compilation step because MSVC was called instead of the MinGW compiler:

 creating build\temp.win-amd64-cpython-311\Release\bld\lets-plot_1729682723651\work\python-package\kotlin-bridge
  "C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Tools\MSVC\14.41.34120\bin\HostX64\x64\cl.exe" /c /nologo /O2 /W3 /GL /DNDEBUG /MD -ID:\bld\lets-plot_1729682723651\work\python-extension\build\bin\native\releaseStatic -ID:\bld\lets-plot_1729682723651\_h_env\include -ID:\bld\lets-plot_1729682723651\_h_env\Include "-IC:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Tools\MSVC\14.41.34120\include" "-IC:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Tools\MSVC\14.41.34120\ATLMFC\include" "-IC:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Auxiliary\VS\include" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.22621.0\ucrt" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\um" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\shared" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\winrt" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\cppwinrt" "-IC:\Program Files (x86)\Windows Kits\NETFXSDK\4.8\include\um" -ID:\bld\lets-plot_1729682723651\_h_env\Library\include -ID:\bld\lets-plot_1729682723651\_h_env\Library\include /d1trimfile:D:\bld\lets-plot_1729682723651\work /TcD:\bld\lets-plot_1729682723651\work\python-package\kotlin-bridge\lets_plot_kotlin_bridge.c /Fobuild\temp.win-amd64-cpython-311\Release\bld\lets-plot_1729682723651\work\python-package\kotlin-bridge\lets_plot_kotlin_bridge.obj
  lets_plot_kotlin_bridge.c
  D:\bld\lets-plot_1729682723651\work\python-package\kotlin-bridge\lets_plot_kotlin_bridge.c(47): warning C4090: 'function': different 'const' qualifiers
  "C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Tools\MSVC\14.41.34120\bin\HostX64\x64\link.exe" /nologo /INCREMENTAL:NO /LTCG /DLL /MANIFEST:EMBED,ID=2 /MANIFESTUAC:NO /LIBPATH:D:\bld\lets-plot_1729682723651\work\python-extension\build\bin\native\releaseStatic /LIBPATH:D:\bld\lets-plot_1729682723651\_h_env\libs /LIBPATH:D:\bld\lets-plot_1729682723651\_h_env /LIBPATH:D:\bld\lets-plot_1729682723651\_h_env\PCbuild\amd64 "/LIBPATH:C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Tools\MSVC\14.41.34120\ATLMFC\lib\x64" "/LIBPATH:C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Tools\MSVC\14.41.34120\lib\x64" "/LIBPATH:C:\Program Files (x86)\Windows Kits\NETFXSDK\4.8\lib\um\x64" "/LIBPATH:C:\Program Files (x86)\Windows Kits\10\lib\10.0.22621.0\ucrt\x64" "/LIBPATH:C:\Program Files (x86)\Windows Kits\10\\lib\10.0.22621.0\\um\x64" /LIBPATH:D:\bld\lets-plot_1729682723651\_h_env\Library\lib /LIBPATH:D:\bld\lets-plot_1729682723651\_h_env\Library\lib lets_plot_python_extension.lib stdc++.lib /EXPORT:PyInit_lets_plot_kotlin_bridge build\temp.win-amd64-cpython-311\Release\bld\lets-plot_1729682723651\work\python-package\kotlin-bridge\lets_plot_kotlin_bridge.obj /OUT:build\lib.win-amd64-cpython-311\lets_plot_kotlin_bridge.cp311-win_amd64.pyd /IMPLIB:build\temp.win-amd64-cpython-311\Release\bld\lets-plot_1729682723651\work\python-package\kotlin-bridge\lets_plot_kotlin_bridge.cp311-win_amd64.lib -static-libgcc -static -lbcrypt -lpthread -lz
  LINK : warning LNK4044: unrecognized option '/static-libgcc'; ignored
  LINK : warning LNK4044: unrecognized option '/static'; ignored
  LINK : warning LNK4044: unrecognized option '/lbcrypt'; ignored
  LINK : warning LNK4044: unrecognized option '/lpthread'; ignored
  LINK : warning LNK4044: unrecognized option '/lz'; ignored
  LINK : fatal error LNK1181: cannot open input file 'lets_plot_python_extension.lib'
  error: command 'C:\\Program Files\\Microsoft Visual Studio\\2022\\Enterprise\\VC\\Tools\\MSVC\\14.41.34120\\bin\\HostX64\\x64\\link.exe' failed with exit code 1181
  error: subprocess-exited-with-error
  Building wheel for lets-plot (setup.py): finished with status 'error'

The compiler('m2w64_cxx') setting was used in the recipe, but it was ignored by conda.

What steps should I try next to fix the Windows build?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
question
Milestone
No milestone
Development

No branches or pull requests
2 participants
@mfansler @VDovidaytis-HORIS