Skip to content
This repository has been archived by the owner on Jan 10, 2023. It is now read-only.

appcfg.py doesn't exist #294

Open
jdwall opened this issue Dec 3, 2016 · 8 comments
Open

appcfg.py doesn't exist #294

jdwall opened this issue Dec 3, 2016 · 8 comments

Comments

@jdwall
Copy link
Contributor

jdwall commented Dec 3, 2016

Hi,

I deployed a DO Droplet to give this a shot. I was able to get through all of the installation, until, the step where you are to interact with appcfg.py which doesn't exist. I feel this should be obvious, but, I am totally lost as to where it could be.

root@python-2gb-nyc3-01:/home/PerfKitExplorer# bash compile.sh
** Compilation initializing.
* Clean out the existing deployment content.
[01:04:15] Using gulpfile /home/PerfKitExplorer/gulpfile.js
[01:04:15] Starting 'third_party'...
[01:04:15] Finished 'third_party' after 38 ms
[01:04:15] Starting 'common'...
[01:04:16] Finished 'common' after 11 ms
[01:04:16] Starting 'prod'...
[01:04:16] Finished 'prod' after 11 ms
[01:04:16] Starting 'default'...
[01:04:16] Finished 'default' after 12 μs

** Compilation complete.
root@python-2gb-nyc3-01:/home/PerfKitExplorer# appcfg.py --oauth2 update deploy
appcfg.py: command not found

Even in /home/PerfKitExplorer/deploy/ there isn't a appcfg.py. Using find I couldn't find that file either. Did I mess up something which would have created appcfg.py or is the documentation need to be updated? Pulled from master and am up to commit 504ab84.

root@python-2gb-nyc3-01:~# gcloud -v
Google Cloud SDK 136.0.0
bq 2.0.24
bq-nix 2.0.24
core 2016.11.30
core-nix 2016.11.07
gcloud
gsutil 4.22
gsutil-nix 4.18

root@python-2gb-nyc3-01:~# python --version
Python 2.7.12

root@python-2gb-nyc3-01:~# java -version
java version "1.7.0_95"
OpenJDK Runtime Environment (IcedTea 2.6.4) (7u95-2.6.4-3)
OpenJDK 64-Bit Server VM (build 24.95-b01, mixed mode)

root@python-2gb-nyc3-01:~# lsb_release -a
No LSB modules are available.
Distributor ID:	Ubuntu
Description:	Ubuntu 16.04.1 LTS
Release:	16.04
Codename:	xenial
@tedsta
Copy link
Contributor

tedsta commented Dec 5, 2016

Have you installed the App Engine SDK for Python? If so, you may need to create symlinks in /usr/bin or similar, or modify your path. See here

@jdwall
Copy link
Contributor Author

jdwall commented Dec 5, 2016

I did install the App Engine SDK for Python. I destroyed the Droplet in the time since. I will redo the steps again this afternoon or tomorrow and report back my findings. Thanks in advance, @tedsta

@jdwall
Copy link
Contributor Author

jdwall commented Dec 6, 2016 •
edited
Loading

Hi @tedsta! I realized now, in retrospect, that I failed to provide the $PATH so the appcfg.py piece is fixed. However, now I am finding myself that I can't deploy the app to App Engine as there are limits of 10,000 files for new deployments? It's a bummer the limits employed disallow me from deploying until I send in a Google form. :(

2016-12-06 21:40:16,787 DEBUG    root            (gcloud.app.deploy) INVALID_ARGUMENT: This deployment has too many files. New versions are limited to 10000 files for this app.
Traceback (most recent call last):
  File "/usr/lib64/google-cloud-sdk/lib/googlecloudsdk/calliope/cli.py", line 740, in Execute
    resources = args.calliope_command.Run(cli=self, args=args)
  File "/usr/lib64/google-cloud-sdk/lib/googlecloudsdk/calliope/backend.py", line 1678, in Run
    resources = command_instance.Run(args)
  File "/usr/lib64/google-cloud-sdk/lib/surface/app/deploy.py", line 53, in Run
    return deploy_util.RunDeploy(args, app_create=True)
  File "/usr/lib64/google-cloud-sdk/lib/googlecloudsdk/command_lib/app/deploy_util.py", line 418, in RunDeploy
    all_services)
  File "/usr/lib64/google-cloud-sdk/lib/googlecloudsdk/command_lib/app/deploy_util.py", line 257, in Deploy
    endpoints_info)
  File "/usr/lib64/google-cloud-sdk/lib/googlecloudsdk/api_lib/app/appengine_api_client.py", line 114, in DeployService
    self.client.apps_services_versions.Create, create_request)
  File "/usr/lib64/google-cloud-sdk/lib/googlecloudsdk/api_lib/app/api/requests.py", line 67, in MakeRequest
    raise err() if err else exc
HttpException: INVALID_ARGUMENT: This deployment has too many files. New versions are limited to 10000 files for this app.
2016-12-06 21:40:16,788 ERROR    root            (gcloud.app.deploy) INVALID_ARGUMENT: This deployment has too many files. New versions are limited to 10000 files for this app.

I wasn't able to get appcfg.py to deploy the app, so I used gcloud app deploy. I had to comment out application and version for it to go through. I am not sure if that affects anything here, but, wanted to provide you context just in case it does.

[root@centos-2gb-nyc3-01 PerfKitExplorer]# appcfg.py --oauth2 update deploy --noauth_local_webserver

Note: the --oauth2 flag is now the default and can be omitted.

09:46 PM Application: REDACTED-perfkit-explorer; version: beta
09:46 PM Host: appengine.google.com
09:46 PM Starting update of app: REDACTED-perfkit-explorer, version: beta
09:46 PM Getting current resource limits.
09:46 PM Scanning files on local disk.
Could not guess mimetype for client/third_party/ui-grid/ui-grid.eot.  Using application/octet-stream.
Could not guess mimetype for client/third_party/ui-grid/ui-grid.svg.  Using application/octet-stream.
Could not guess mimetype for client/third_party/jquery/jquery.min.map.  Using application/octet-stream.
Could not guess mimetype for client/third_party/bootstrap-ui/fonts/glyphicons-halflings-regular.eot.  Using application/octet-stream.
Could not guess mimetype for client/third_party/bootstrap-ui/fonts/glyphicons-halflings-regular.svg.  Using application/octet-stream.
Could not guess mimetype for client/third_party/angular-material/angular-material.scss.  Using application/octet-stream.
Could not guess mimetype for client/third_party/angular/angular-animate.min.js.map.  Using application/octet-stream.
Could not guess mimetype for client/third_party/angular/angular-aria.min.js.map.  Using application/octet-stream.
Could not guess mimetype for client/third_party/angular/angular-sanitize.min.js.map.  Using application/octet-stream.
Could not guess mimetype for client/third_party/angular/angular.min.js.gzip.  Using application/octet-stream.
Could not guess mimetype for client/third_party/angular/angular.min.js.map.  Using application/octet-stream.
Could not guess mimetype for client/components/widget/data_viz/gviz/gviz-charts.json.  Using application/octet-stream.
Error 404: --- begin server output ---
This application does not exist (project_id=u'REDACTED-perfkit-explorer'). To create an App Engine application in this project, run "gcloud beta app create" in your console.
--- end server output ---
[root@centos-2gb-nyc3-01 PerfKitExplorer]#

@tedsta
Copy link
Contributor

tedsta commented Dec 6, 2016 •
edited
Loading

That is a bummer. I've pinged the original developer internally and will post back here with any updates.

@jmuharsky
Copy link
Contributor

It looks like venv (virtual environment for testing) is getting copied into the deploy folder. If you remove the deploy/server/venv folder before deployment, you should be well under the 10k file limit. I'll look at fixing this step shortly.

@jdwall
Copy link
Contributor Author

jdwall commented Dec 6, 2016 •
edited
Loading

@jmuharsky I am not seeing a folder called venv. Any idea on where it might be hiding?

[root@centos-2gb-nyc3-01 server]# pwd
/root/projects/PerfKitExplorer/deploy/server
[root@centos-2gb-nyc3-01 deploy]# tree /root/projects/PerfKitExplorer/deploy/
/root/projects/PerfKitExplorer/deploy/
├── appengine_config.py
├── app.yaml
├── client
│   ├── components
│   │   └── widget
│   │       └── data_viz
│   │           └── gviz
│   │               └── gviz-charts.json
│   ├── perfkit_scripts.js
│   ├── perfkit_styles.css
│   ├── perfkit_templates.js
│   └── third_party
│       ├── angular
│       │   ├── angular-animate.js
│       │   ├── angular-animate.min.js
│       │   ├── angular-animate.min.js.map
│       │   ├── angular-aria.js
│       │   ├── angular-aria.min.js
│       │   ├── angular-aria.min.js.map
│       │   ├── angular.js
│       │   ├── angular.min.js
│       │   ├── angular.min.js.gzip
│       │   ├── angular.min.js.map
│       │   ├── angular-mocks.js
│       │   ├── angular-sanitize.js
│       │   ├── angular-sanitize.min.js
│       │   └── angular-sanitize.min.js.map
│       ├── angular-material
│       │   ├── angular-material.css
│       │   ├── angular-material.js
│       │   ├── angular-material.min.css
│       │   ├── angular-material.min.js
│       │   └── angular-material.scss
│       ├── angular-ui-router
│       │   ├── angular-ui-router.js
│       │   └── angular-ui-router.min.js
│       ├── bootstrap-ui
│       │   ├── css
│       │   │   └── bootstrap.min.css
│       │   ├── fonts
│       │   │   ├── glyphicons-halflings-regular.eot
│       │   │   ├── glyphicons-halflings-regular.svg
│       │   │   ├── glyphicons-halflings-regular.ttf
│       │   │   ├── glyphicons-halflings-regular.woff
│       │   │   └── glyphicons-halflings-regular.woff2
│       │   ├── ui-bootstrap-tpls.js
│       │   └── ui-bootstrap-tpls.min.js
│       ├── codemirror
│       │   ├── codemirror.css
│       │   ├── codemirror.js
│       │   └── mode
│       │       ├── javascript
│       │       │   └── javascript.js
│       │       └── sql
│       │           └── sql.js
│       ├── jquery
│       │   ├── jquery.min.js
│       │   └── jquery.min.map
│       ├── showdown
│       │   ├── ng-showdown.js
│       │   ├── ng-showdown.min.js
│       │   ├── showdown.js
│       │   └── showdown.min.js
│       └── ui-grid
│           ├── ui-grid.css
│           ├── ui-grid.eot
│           ├── ui-grid.js
│           ├── ui-grid.min.css
│           ├── ui-grid.min.js
│           ├── ui-grid.svg
│           ├── ui-grid.ttf
│           └── ui-grid.woff
├── config
│   ├── big_query_v2_rest.json
│   ├── credentials.json
│   ├── data_source_config.json
│   └── data_source_config_mock.json
├── index.yaml
├── __init__.py
└── server
    ├── perfkit
    │   ├── common
    │   │   ├── big_query_client.py
    │   │   ├── big_query_client_test.py
    │   │   ├── big_query_result_pivot.py
    │   │   ├── big_query_result_pivot_test.py
    │   │   ├── big_query_result_util.py
    │   │   ├── big_query_result_util_test.py
    │   │   ├── credentials_lib.py
    │   │   ├── credentials_lib_test.py
    │   │   ├── data_source_config.py
    │   │   ├── data_source_config_test.py
    │   │   ├── datetime_util.py
    │   │   ├── datetime_util_test.py
    │   │   ├── gae_big_query_client.py
    │   │   ├── gae_cloud_sql_client.py
    │   │   ├── gae_test_util.py
    │   │   ├── http_util.py
    │   │   ├── http_util_test.py
    │   │   ├── __init__.py
    │   │   └── mock_big_query_client.py
    │   ├── explorer
    │   │   ├── handlers
    │   │   │   ├── base.py
    │   │   │   ├── dashboard.py
    │   │   │   ├── dashboard_test.py
    │   │   │   ├── data.py
    │   │   │   ├── data_test.py
    │   │   │   ├── explorer_config.py
    │   │   │   ├── explorer_config_test.py
    │   │   │   ├── __init__.py
    │   │   │   ├── pages.py
    │   │   │   ├── pages_test.py
    │   │   │   └── templates
    │   │   │       ├── compare.html
    │   │   │       ├── dashboard-admin.html
    │   │   │       ├── explorer.html
    │   │   │       ├── page-base.html
    │   │   │       └── third-party-scripts.html
    │   │   ├── __init__.py
    │   │   ├── model
    │   │   │   ├── dashboard_fields.py
    │   │   │   ├── dashboard.py
    │   │   │   ├── dashboard_test.py
    │   │   │   ├── error_fields.py
    │   │   │   ├── explorer_config.py
    │   │   │   ├── explorer_config_test.py
    │   │   │   ├── __init__.py
    │   │   │   └── settings.py
    │   │   ├── samples_mart
    │   │   │   ├── explorer_method.py
    │   │   │   ├── explorer_method_test.py
    │   │   │   ├── __init__.py
    │   │   │   ├── label_manager.py
    │   │   │   ├── label_manager_test.py
    │   │   │   ├── product_labels.py
    │   │   │   └── product_labels_test.py
    │   │   └── util
    │   │       ├── explorer_config_util.py
    │   │       ├── explorer_config_util_test.py
    │   │       ├── __init__.py
    │   │       └── user_validator.py
    │   ├── ext
    │   │   ├── cloudsql
    │   │   │   ├── handlers
    │   │   │   │   ├── cloudsql_config.py
    │   │   │   │   ├── cloudsql_config_test.py
    │   │   │   │   └── __init__.py
    │   │   │   ├── __init__.py
    │   │   │   └── models
    │   │   │       ├── cloudsql_config.py
    │   │   │       ├── cloudsql_config_test.py
    │   │   │       └── __init__.py
    │   │   └── __init__.py
    │   ├── __init__.py
    │   └── test_util.py
    └── third_party
        ├── apiclient
        │   ├── channel.py
        │   ├── discovery.py
        │   ├── errors.py
        │   ├── http.py
        │   ├── __init__.py
        │   ├── mimeparse.py
        │   ├── model.py
        │   ├── sample_tools.py
        │   └── schema.py
        ├── dateutil
        │   ├── easter.py
        │   ├── __init__.py
        │   ├── parser.py
        │   ├── relativedelta.py
        │   ├── rrule.py
        │   ├── tz.py
        │   ├── tzwin.py
        │   └── zoneinfo
        │       ├── __init__.py
        │       └── zoneinfo-2010g.tar.gz
        ├── httplib2
        │   ├── cacerts.txt
        │   ├── __init__.py
        │   ├── iri2uri.py
        │   └── socks.py
        ├── __init__.py
        ├── oauth2client
        │   ├── anyjson.py
        │   ├── appengine.py
        │   ├── client.py
        │   ├── clientsecrets.py
        │   ├── crypt.py
        │   ├── django_orm.py
        │   ├── file.py
        │   ├── gce.py
        │   ├── __init__.py
        │   ├── keyring_storage.py
        │   ├── locked_file.py
        │   ├── multistore_file.py
        │   ├── old_run.py
        │   ├── tools.py
        │   ├── util.py
        │   └── xsrfutil.py
        └── uritemplate
            └── __init__.py

40 directories, 163 files
[root@centos-2gb-nyc3-01 deploy]#

@jmuharsky
Copy link
Contributor

Hrm... apparently this was chaff from previous deployment work. I'm not sure why you're seeing that issue -- there clearly aren't anywhere near 10k files.

@michaelsmit
Copy link

Reviewing the detailed log for gcloud app deploy (for me, /home/$USER/.config/gcloud/logs/), it looks like it is trying to upload every file in PerfKitExplorer. Not sure why and didn't try to debug.

appcfg.py wasn't on my $PATH either, but I was able to deploy by finding the file and running it using its full path:
/home/$USER/google-cloud-sdk/platform/google_appengine/appcfg.py

Looks like the deployment process has changed since the README was written...

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@jmuharsky @tedsta @michaelsmit @jdwall