Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Ubuntu 16.04 failure to provision using enova-python. #4

Open
thomasdziedzic-pd opened this issue Feb 25, 2020 · 8 comments
Open

Ubuntu 16.04 failure to provision using enova-python. #4

thomasdziedzic-pd opened this issue Feb 25, 2020 · 8 comments

Comments

@thomasdziedzic-pd
Copy link

I have the following lines in my recipe to try to install python2:

node.default['poise-python']['options']['pip_version'] = '18.0'
include_recipe 'poise-python'

When I tried to provision a new host, I got the following error message:

[2020-02-25T23:10:12Z] 34.213.177.81   * python_runtime[2] action install[2020-02-25T23:10:12+00:00] INFO: Processing python_runtime[2] action install (poise-python::default line 26)

[2020-02-25T23:10:13Z] 34.213.177.81 

[2020-02-25T23:10:13Z] 34.213.177.81     * poise_languages_system[python2.7] action install[2020-02-25T23:10:13+00:00] INFO: Processing poise_languages_system[python2.7] action install (/var/chef/cache/cookbooks/poise-languages/files/halite_gem/poise_languages/system/mixin.rb line 32)

[2020-02-25T23:10:19Z] 34.213.177.81 [2020-02-25T23:10:19+00:00] INFO: apt_package[python2.7, python2.7-dev] installed ["python2.7-dev"] at ["2.7.12-1ubuntu0~16.04.9"]

[2020-02-25T23:10:19Z] 34.213.177.81 

[2020-02-25T23:10:19Z] 34.213.177.81       �[32m- install version 2.7.12-1ubuntu0~16.04.9 of package python2.7-dev�[0m

[2020-02-25T23:10:19Z] 34.213.177.81 �[0m    * apt_package[python2.7, python2.7-dev] action nothing[2020-02-25T23:10:19+00:00] INFO: Processing apt_package[python2.7, python2.7-dev] action nothing (/var/chef/cache/cookbooks/poise-languages/files/halite_gem/poise_languages/system/resource.rb line 144)

[2020-02-25T23:10:19Z] 34.213.177.81  (skipped due to action :nothing)

[2020-02-25T23:10:19Z] 34.213.177.81   

[2020-02-25T23:10:19Z] 34.213.177.81 �[0m[2020-02-25T23:10:19+00:00] INFO: [python_runtime[2]] distutils installer - only valid on ubuntu for python 3

[2020-02-25T23:10:19Z] 34.213.177.81   * python_runtime_pip[2] action install[2020-02-25T23:10:19+00:00] INFO: Processing python_runtime_pip[2] action install (/var/chef/cache/cookbooks/poise-python/files/halite_gem/poise_python/python_providers/base.rb line 140)

[2020-02-25T23:10:26Z] 34.213.177.81 

[2020-02-25T23:10:26Z] 34.213.177.81     �[32m- Bootstrapping pip latest from https://bootstrap.pypa.io/get-pip.py�[0m

[2020-02-25T23:10:26Z] 34.213.177.81 �[0m  * python_package[setuptools] action install[2020-02-25T23:10:26+00:00] INFO: Processing python_package[setuptools] action install (/var/chef/cache/cookbooks/poise-python/files/halite_gem/poise_python/python_providers/base.rb line 160)

[2020-02-25T23:10:27Z] 34.213.177.81 checking for ["setuptools"]

[2020-02-25T23:10:28Z] 34.213.177.81 

[2020-02-25T23:10:28Z] 34.213.177.81     �[0m

[2020-02-25T23:10:28Z] 34.213.177.81     ================================================================================�[0m

[2020-02-25T23:10:28Z] 34.213.177.81     �[31mError executing action `install` on resource 'python_package[setuptools]'�[0m

[2020-02-25T23:10:28Z] 34.213.177.81     ================================================================================�[0m

[2020-02-25T23:10:28Z] 34.213.177.81     

[2020-02-25T23:10:28Z] 34.213.177.81 �[0m    Mixlib::ShellOut::ShellCommandFailed�[0m

[2020-02-25T23:10:28Z] 34.213.177.81     ------------------------------------�[0m

[2020-02-25T23:10:28Z] 34.213.177.81     Expected process to exit with [0], but received '1'

[2020-02-25T23:10:28Z] 34.213.177.81 �[0m    ---- Begin output of ["/usr/bin/python2.7", "-", "setuptools"] ----

[2020-02-25T23:10:28Z] 34.213.177.81 �[0m    STDOUT: 

[2020-02-25T23:10:28Z] 34.213.177.81 �[0m    STDERR: Traceback (most recent call last):

[2020-02-25T23:10:28Z] 34.213.177.81 �[0m      File "<stdin>", line 22, in <module>

[2020-02-25T23:10:28Z] 34.213.177.81 �[0m    ImportError: cannot import name InstallCommand

[2020-02-25T23:10:28Z] 34.213.177.81 �[0m    ---- End output of ["/usr/bin/python2.7", "-", "setuptools"] ----

[2020-02-25T23:10:28Z] 34.213.177.81 �[0m    Ran ["/usr/bin/python2.7", "-", "setuptools"] returned 1�[0m

[2020-02-25T23:10:28Z] 34.213.177.81     

[2020-02-25T23:10:28Z] 34.213.177.81 �[0m    Cookbook Trace:�[0m

[2020-02-25T23:10:28Z] 34.213.177.81     ---------------�[0m

[2020-02-25T23:10:28Z] 34.213.177.81     /var/chef/cache/cookbooks/poise-languages/files/halite_gem/poise_languages/command/mixin.rb:207:in `tap'

[2020-02-25T23:10:28Z] 34.213.177.81 �[0m    /var/chef/cache/cookbooks/poise-languages/files/halite_gem/poise_languages/command/mixin.rb:207:in `language_command_shell_out!'

[2020-02-25T23:10:28Z] 34.213.177.81 �[0m    /var/chef/cache/cookbooks/poise-languages/files/halite_gem/poise_languages/command/mixin.rb:222:in `block in language_command_mixin'

[2020-02-25T23:10:28Z] 34.213.177.81 �[0m    /var/chef/cache/cookbooks/poise-python/files/halite_gem/poise_python/resources/python_package.rb:343:in `pip_command'

[2020-02-25T23:10:28Z] 34.213.177.81 �[0m    /var/chef/cache/cookbooks/poise-python/files/halite_gem/poise_python/resources/python_package.rb:368:in `pip_outdated'

[2020-02-25T23:10:28Z] 34.213.177.81 �[0m    /var/chef/cache/cookbooks/poise-python/files/halite_gem/poise_python/resources/python_package.rb:235:in `check_package_versions'

[2020-02-25T23:10:28Z] 34.213.177.81 �[0m    /var/chef/cache/cookbooks/poise-python/files/halite_gem/poise_python/resources/python_package.rb:214:in `load_current_resource'

[2020-02-25T23:10:28Z] 34.213.177.81 �[0m    /var/chef/cache/cookbooks/poise/files/halite_gem/poise/helpers/notifying_block.rb:69:in `notifying_block'

[2020-02-25T23:10:28Z] 34.213.177.81 �[0m    /var/chef/cache/cookbooks/poise-python/files/halite_gem/poise_python/python_providers/base.rb:50:in `action_install'�[0m

[2020-02-25T23:10:28Z] 34.213.177.81     

[2020-02-25T23:10:28Z] 34.213.177.81 �[0m    Resource Declaration:�[0m

[2020-02-25T23:10:28Z] 34.213.177.81     ---------------------�[0m

[2020-02-25T23:10:28Z] 34.213.177.81     # In /var/chef/cache/cookbooks/poise-python/files/halite_gem/poise_python/python_providers/base.rb

[2020-02-25T23:10:28Z] 34.213.177.81 �[0m    

[2020-02-25T23:10:28Z] 34.213.177.81 �[0m    160:         python_package 'setuptools' do

[2020-02-25T23:10:28Z] 34.213.177.81 �[0m    161:           parent_python new_resource

[2020-02-25T23:10:28Z] 34.213.177.81 �[0m    162:           version setuptools_version if setuptools_version.is_a?(String)

[2020-02-25T23:10:28Z] 34.213.177.81 �[0m    163:         end

[2020-02-25T23:10:28Z] 34.213.177.81 �[0m    164:       end

[2020-02-25T23:10:28Z] 34.213.177.81 �[0m    

[2020-02-25T23:10:28Z] 34.213.177.81 �[0m    Compiled Resource:�[0m

[2020-02-25T23:10:28Z] 34.213.177.81     ------------------�[0m

[2020-02-25T23:10:28Z] 34.213.177.81     # Declared in /var/chef/cache/cookbooks/poise-python/files/halite_gem/poise_python/python_providers/base.rb:160:in `install_setuptools'

[2020-02-25T23:10:28Z] 34.213.177.81 �[0m    

[2020-02-25T23:10:28Z] 34.213.177.81 �[0m    python_package("setuptools") do

[2020-02-25T23:10:28Z] 34.213.177.81 �[0m      package_name "setuptools"

[2020-02-25T23:10:28Z] 34.213.177.81 �[0m      action [:install]

[2020-02-25T23:10:28Z] 34.213.177.81 �[0m      default_guard_interpreter :default

[2020-02-25T23:10:28Z] 34.213.177.81 �[0m      declared_type :python_package

[2020-02-25T23:10:28Z] 34.213.177.81 �[0m      cookbook_name "poise-python"

[2020-02-25T23:10:28Z] 34.213.177.81 �[0m      parent_python python_runtime[2]

[2020-02-25T23:10:28Z] 34.213.177.81 �[0m      timeout 900

[2020-02-25T23:10:28Z] 34.213.177.81 �[0m    end

[2020-02-25T23:10:28Z] 34.213.177.81 �[0m    

[2020-02-25T23:10:28Z] 34.213.177.81 �[0m    System Info:�[0m

[2020-02-25T23:10:28Z] 34.213.177.81     ------------�[0m

[2020-02-25T23:10:28Z] 34.213.177.81     chef_version=14.13.11

[2020-02-25T23:10:28Z] 34.213.177.81 �[0m    platform=ubuntu

[2020-02-25T23:10:28Z] 34.213.177.81 �[0m    platform_version=16.04

[2020-02-25T23:10:28Z] 34.213.177.81 �[0m    ruby=ruby 2.5.5p157 (2019-03-15 revision 67260) [x86_64-linux]

[2020-02-25T23:10:28Z] 34.213.177.81 �[0m    program_name=/opt/chef/bin/chef-client

[2020-02-25T23:10:28Z] 34.213.177.81 �[0m    executable=/opt/chef/bin/chef-client�[0m

[2020-02-25T23:10:28Z] 34.213.177.81     

[2020-02-25T23:10:28Z] 34.213.177.81 �[0m[2020-02-25T23:10:28+00:00] INFO: Running queued delayed notifications before re-raising exception

[2020-02-25T23:10:28Z] 34.213.177.81   �[0m

[2020-02-25T23:10:28Z] 34.213.177.81   ================================================================================�[0m

[2020-02-25T23:10:28Z] 34.213.177.81   �[31mError executing action `install` on resource 'python_runtime[2]'�[0m

[2020-02-25T23:10:28Z] 34.213.177.81   ================================================================================�[0m

[2020-02-25T23:10:28Z] 34.213.177.81   

[2020-02-25T23:10:28Z] 34.213.177.81 �[0m  Mixlib::ShellOut::ShellCommandFailed�[0m

[2020-02-25T23:10:28Z] 34.213.177.81   ------------------------------------�[0m

[2020-02-25T23:10:28Z] 34.213.177.81   python_package[setuptools] (/var/chef/cache/cookbooks/poise-python/files/halite_gem/poise_python/python_providers/base.rb line 160) had an error: Mixlib::ShellOut::ShellCommandFailed: Expected process to exit with [0], but received '1'

[2020-02-25T23:10:28Z] 34.213.177.81 �[0m  ---- Begin output of ["/usr/bin/python2.7", "-", "setuptools"] ----

[2020-02-25T23:10:28Z] 34.213.177.81 �[0m  STDOUT: 

[2020-02-25T23:10:28Z] 34.213.177.81 �[0m  STDERR: Traceback (most recent call last):

[2020-02-25T23:10:28Z] 34.213.177.81 �[0m    File "<stdin>", line 22, in <module>

[2020-02-25T23:10:28Z] 34.213.177.81 �[0m  ImportError: cannot import name InstallCommand

[2020-02-25T23:10:28Z] 34.213.177.81 �[0m  ---- End output of ["/usr/bin/python2.7", "-", "setuptools"] ----

[2020-02-25T23:10:28Z] 34.213.177.81 �[0m  Ran ["/usr/bin/python2.7", "-", "setuptools"] returned 1�[0m

[2020-02-25T23:10:28Z] 34.213.177.81   

[2020-02-25T23:10:28Z] 34.213.177.81 �[0m  Cookbook Trace:�[0m

[2020-02-25T23:10:28Z] 34.213.177.81   ---------------�[0m

[2020-02-25T23:10:28Z] 34.213.177.81   /var/chef/cache/cookbooks/poise-languages/files/halite_gem/poise_languages/command/mixin.rb:207:in `tap'

[2020-02-25T23:10:28Z] 34.213.177.81 �[0m  /var/chef/cache/cookbooks/poise-languages/files/halite_gem/poise_languages/command/mixin.rb:207:in `language_command_shell_out!'

[2020-02-25T23:10:28Z] 34.213.177.81 �[0m  /var/chef/cache/cookbooks/poise-languages/files/halite_gem/poise_languages/command/mixin.rb:222:in `block in language_command_mixin'

[2020-02-25T23:10:28Z] 34.213.177.81 �[0m  /var/chef/cache/cookbooks/poise-python/files/halite_gem/poise_python/resources/python_package.rb:343:in `pip_command'

[2020-02-25T23:10:28Z] 34.213.177.81 �[0m  /var/chef/cache/cookbooks/poise-python/files/halite_gem/poise_python/resources/python_package.rb:368:in `pip_outdated'

[2020-02-25T23:10:28Z] 34.213.177.81 �[0m  /var/chef/cache/cookbooks/poise-python/files/halite_gem/poise_python/resources/python_package.rb:235:in `check_package_versions'

[2020-02-25T23:10:28Z] 34.213.177.81 �[0m  /var/chef/cache/cookbooks/poise-python/files/halite_gem/poise_python/resources/python_package.rb:214:in `load_current_resource'

[2020-02-25T23:10:28Z] 34.213.177.81 �[0m  /var/chef/cache/cookbooks/poise/files/halite_gem/poise/helpers/notifying_block.rb:69:in `notifying_block'

[2020-02-25T23:10:28Z] 34.213.177.81 �[0m  /var/chef/cache/cookbooks/poise-python/files/halite_gem/poise_python/python_providers/base.rb:50:in `action_install'�[0m

[2020-02-25T23:10:28Z] 34.213.177.81   

[2020-02-25T23:10:28Z] 34.213.177.81 �[0m  Resource Declaration:�[0m

[2020-02-25T23:10:28Z] 34.213.177.81   ---------------------�[0m

[2020-02-25T23:10:28Z] 34.213.177.81   # In /var/chef/cache/cookbooks/poise-python/recipes/default.rb

[2020-02-25T23:10:28Z] 34.213.177.81 �[0m  

[2020-02-25T23:10:28Z] 34.213.177.81 �[0m   26: python_runtime '2' if node['poise-python']['install_python2']

[2020-02-25T23:10:28Z] 34.213.177.81 �[0m  

[2020-02-25T23:10:28Z] 34.213.177.81 �[0m  Compiled Resource:�[0m

[2020-02-25T23:10:28Z] 34.213.177.81   ------------------�[0m

[2020-02-25T23:10:28Z] 34.213.177.81   # Declared in /var/chef/cache/cookbooks/poise-python/recipes/default.rb:26:in `from_file'

[2020-02-25T23:10:28Z] 34.213.177.81 �[0m  

[2020-02-25T23:10:28Z] 34.213.177.81 �[0m  python_runtime("2") do

[2020-02-25T23:10:28Z] 34.213.177.81 �[0m    action [:install]

[2020-02-25T23:10:28Z] 34.213.177.81 �[0m    updated true

[2020-02-25T23:10:28Z] 34.213.177.81 �[0m    updated_by_last_action true

[2020-02-25T23:10:28Z] 34.213.177.81 �[0m    default_guard_interpreter :default

[2020-02-25T23:10:28Z] 34.213.177.81 �[0m    subresources [python_virtualenv[/opt/pd-analytics/shared/python-env-2], python_runtime_pip[2], python_package[setuptools], python_package[wheel], python_package[virtualenv]]

[2020-02-25T23:10:28Z] 34.213.177.81 �[0m    declared_type :python_runtime

[2020-02-25T23:10:28Z] 34.213.177.81 �[0m    cookbook_name "poise-python"

[2020-02-25T23:10:28Z] 34.213.177.81 �[0m    recipe_name "default"

[2020-02-25T23:10:28Z] 34.213.177.81 �[0m    get_pip_url "https://bootstrap.pypa.io/get-pip.py"

[2020-02-25T23:10:28Z] 34.213.177.81 �[0m    pip_version true

[2020-02-25T23:10:28Z] 34.213.177.81 �[0m    setuptools_version true

[2020-02-25T23:10:28Z] 34.213.177.81 �[0m    version "2"

[2020-02-25T23:10:28Z] 34.213.177.81 �[0m    virtualenv_version true

[2020-02-25T23:10:28Z] 34.213.177.81 �[0m    wheel_version true

[2020-02-25T23:10:28Z] 34.213.177.81 �[0m  end

[2020-02-25T23:10:28Z] 34.213.177.81 �[0m  

[2020-02-25T23:10:28Z] 34.213.177.81 �[0m  System Info:�[0m

[2020-02-25T23:10:28Z] 34.213.177.81   ------------�[0m

[2020-02-25T23:10:28Z] 34.213.177.81   chef_version=14.13.11

[2020-02-25T23:10:28Z] 34.213.177.81 �[0m  platform=ubuntu

[2020-02-25T23:10:28Z] 34.213.177.81 �[0m  platform_version=16.04

[2020-02-25T23:10:28Z] 34.213.177.81 �[0m  ruby=ruby 2.5.5p157 (2019-03-15 revision 67260) [x86_64-linux]

[2020-02-25T23:10:28Z] 34.213.177.81 �[0m  program_name=/opt/chef/bin/chef-client

[2020-02-25T23:10:28Z] 34.213.177.81 �[0m  executable=/opt/chef/bin/chef-client�[0m

This might be related to #3 (comment) since they had an error in the CI during setuptools install as well.

Thanks for your time!
@thomasdziedzic-pd
Copy link
Author

I can reproduce the issue with the local kitchen tests using kitchen test Python3and2-pip18-ubuntu-1604, I will attach the full log,
Python3and2-pip18-ubuntu-1604.log

The relevant message from the log is at:

       [2020-02-26T15:30:52+00:00] FATAL: Mixlib::ShellOut::ShellCommandFailed: python_runtime[3] (poise-python::default line 25) had an error: Mixlib::ShellOut::ShellCommandFailed: python_package[setuptools] (/tmp/kitchen/cache/cookbooks/poise-python/files/halite_gem/poise_python/python_providers/base.rb line 160) had an error: Mixlib::ShellOut::ShellCommandFailed: Expected process to exit with [0], but received '1'
       ---- Begin output of ["/usr/bin/python3.5", "-", "setuptools"] ----
       STDOUT:
       STDERR: Traceback (most recent call last):
         File "<stdin>", line 17, in <module>
       ImportError: No module named 'pip.commands'

       During handling of the above exception, another exception occurred:

       Traceback (most recent call last):
         File "<stdin>", line 22, in <module>
       ImportError: cannot import name 'InstallCommand'
       ---- End output of ["/usr/bin/python3.5", "-", "setuptools"] ----
       Ran ["/usr/bin/python3.5", "-", "setuptools"] returned 1

which is the same error message that I get when trying to use the cookbook.

@thomasdziedzic-pd
Copy link
Author

I applied the patch located at shareaholic/poise-python@929195a and it seems to have made the job succeed.

@mpescetto
Copy link

mpescetto commented Feb 26, 2020 •
edited
Loading

Hi @thomasdziedzic-pd thanks for reporting the issue. If you have time to submit a PR that would be amazing, otherwise I can work on integrating your changes to this cookbook sometime next week. Also, you might want to remove your public IP from the logs in the description if you care about such things.

@thomasdziedzic-pd
Copy link
Author

thomasdziedzic-pd commented Feb 26, 2020 •
edited
Loading

. If you have time to submit a PR that would be amazing

I'm trying to figure out some permissions issue with forking cookbooks, if I manage to figure it out I will try and submit one!

Also, you might want to remove your public IP from the logs in the description if you care about such things.

The machine was already decommissioned by the time I posted the logs.

@thomasdziedzic-pd
Copy link
Author

As a follow up to:

I'm trying to figure out some permissions issue with forking cookbooks, if I manage to figure it out I will try and submit one!

I wasn't able to resolve this so I wont be submitting a PR unfortunately.

@pescetto
Copy link
Contributor

@thomasdziedzic-pd super sorry for the delay on this. I'll follow up by end of month. My apologies.

@ju2wheels
Copy link

Here is my first round fix for this:

https://github.com/tvision-insights/enova-python/blob/1.8.11/files/halite_gem/poise_python/resources/python_package.rb

I overlayed ndpgroup/poise-python changes over your generated cookbook in addition to fixing this bug, but the hack script changes in that should be able to be grabbed without also pulling in the other changes.

@mpescetto mpescetto mentioned this issue May 22, 2020
Closed
@ju2wheels
Copy link

I pushed another fix for the hack script as 1.8.12 on our fork. I have manually tested this version of the hack script against all pip versions from 8.1.1 to 20.3.3 using Python 2.7 to address a few more bugs on pip 19.3.x and higher.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@ju2wheels @pescetto @thomasdziedzic-pd @mpescetto