Skip to content

Commit

Permalink
Merge branch 'fix_installer' into 'main'
Browse files Browse the repository at this point in the history
[READY TO REVIEW] - Fix installer issues and add remove unnecessary commands.

See merge request wwso-cross-industry-prototyping/arcade/arcade!63
  • Loading branch information
William Sia committed Oct 3, 2024
2 parents 261b8eb + f1b4b62 commit 3bbf781
Show file tree
Hide file tree
Showing 33 changed files with 2,177 additions and 1,313 deletions.
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -199,3 +199,5 @@ infrastructure/src/stacServer/lambdas/api.zip
infrastructure/src/stacServer/lambdas/ingest.zip

stac-server

**/package-lock.json
10 changes: 9 additions & 1 deletion .vscode/settings.json
Original file line number Diff line number Diff line change
Expand Up @@ -21,5 +21,13 @@
"lib"
],
"formatFiles.runOrganizeImports": true,
"formatFiles.excludePattern": "*.log,*.tsbuildinfo"
"formatFiles.excludePattern": "*.log,*.tsbuildinfo",
"[shellscript]": {
"editor.defaultFormatter": "foxundermoon.shell-format"
},

"files.exclude": {
"**/*.js": true,
"**/*.map": true
}
}
88 changes: 27 additions & 61 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,9 @@ To dive deep into AGIE components, please visit the following links:
- [UI Module](typescript/packages/apps/ui)

### Success

Upon successful deployment of this guidance, you will be able to do the following:

- Enroll polygons (areas of interest, farm fields, geographical areas of land) for processing and analysis
- Schedule the frequency of polygon analysis, for each polygon, to support one-time and recurring automated processing
- Automate the searching, downloading, clipping, and processing of earth observation data into NDVI maps and meta data, per polygon
Expand All @@ -42,7 +44,6 @@ Upon successful deployment of this guidance, you will be able to do the followin
- Visualize processed imagery and meta data via an optional but integrated UI
- Integrate, customize, extend and augment existing workflows to achieve business goals


### Cost

You will need to pay for the AWS services used while running this guidance. As of September 2024, the cost for running this Guidance with the default settings in the US East (N. Virginia) is approximately $147.66 per month for analyzing 1000 polygons every 5 days as illustrated in the sample table below.
Expand All @@ -60,22 +61,17 @@ You will need to pay for the AWS services used while running this guidance. As o
| US East (N. Virginia) | AGIE Modules Lambda | AWS Lambda | $0 | Average size of each request (50 KB) Requests (50,000 per month) |
| Total | | | $147.66 | |


## Prerequisites for Deployment

In order to deploy `AGIE` from your local workstation, you need to install the following dependencies:

- Active AWS Account
- [Node.js 20.x](https://nodejs.org/en/download/package-manager)
- [AWS Command Line Interface](https://aws.amazon.com/cli/)
- [Docker](https://docs.docker.com/engine/install/)
- [Rush.js](https://rushjs.io/)
- Node.js 20.x
- [pnpm 9.0.2](https://www.npmjs.com/package/pnpm/v/9.0.2)
- [tsx](https://www.npmjs.com/package/tsx/v/4.18.0)
- It may be necessary to bootstrap your AWS environment. Bootstrapping is the process of preparing your AWS environment for usage with the AWS Cloud Development Kit (AWS CDK). Before you deploy a CDK stack into an AWS environment, the environment must first be bootstrapped. For more information about bootstrapping, please review the following:
- https://docs.aws.amazon.com/cdk/v2/guide/bootstrapping.html
- https://docs.aws.amazon.com/cdk/v2/guide/bootstrapping-env.html
- NOTE: AWS AGIE includes an optional UI module. The optional and integrated UI module includes functionality provided by Amazon Location Service. As of 7/24, Amazon Location Service is not available in certain AWS Regions, including us-west-1. Please review the most current list of Region availability of Amazon Location Service to ensure the optional UI module can be deployed within a Region that meets your company requirements.

> NOTE: AWS AGIE includes an optional UI module. The optional and integrated UI module includes functionality provided by Amazon Location Service. As of 7/24, Amazon Location Service is not available in certain AWS Regions, including us-west-1. Please review the most current list of Region availability of Amazon Location Service to ensure the optional UI module can be deployed within a Region that meets your company requirements.
## Deployment

Expand All @@ -84,61 +80,44 @@ In order to deploy `AGIE` from your local workstation, you need to install the f
Run the following command to clone the solution repository into your local workstation:

```shell
$ git clone https://github.com/aws-solutions-library-samples/guidance-for-automated-geospatial-insights-engine-on-aws
$ cd guidance-for-automated-geospatial-insights-engine-on-aws
$ export AGIE_FOLDER=$PWD
$ export CLI_FOLDER="$PWD/typescript/packages/apps/cli"
```

### 2. Bundle the artifact from `Stac-server` repository

AGIE conforms to the [STAC specification](https://github.com/radiantearth/stac-spec/blob/master/item-spec/item-spec.md) and generates [STAC item](https://github.com/radiantearth/stac-spec/blob/master/item-spec/item-spec.md) as a result of the analysis. It stores these analysis results in [Stac-server](https://github.com/stac-utils/stac-server). To make the deployment simpler, we port the infrastructure definition from `serverless` framework to `cdk`. Run the following command to bundle `Stac-server` NodeJS application codes which are referred by the [StacServer](infrastructure/src/stacServer/stacServer.stack.ts) stack

```shell
$ cd $AGIE_FOLDER
$ git clone --depth 1 --branch v3.8.0 https://github.com/stac-utils/stac-server.git
$ cd stac-server && npm install && npm run build
$ cp dist/api/api.zip $AGIE_FOLDER/infrastructure/src/stacServer/lambdas && cp dist/ingest/ingest.zip $AGIE_FOLDER/infrastructure/src/stacServer/lambdas

git clone https://github.com/aws-solutions-library-samples/guidance-for-automated-geospatial-insights-engine-on-aws
cd guidance-for-automated-geospatial-insights-engine-on-aws
export AGIE_FOLDER=$PWD
export CLI_FOLDER="$PWD/typescript/packages/apps/cli"
```

### 3. Build the solution
### 2. Build the solution

Run the following command to install the solution dependencies and build it:

```shell
$ cd $AGIE_FOLDER
$ rush update
$ rush build
cd $AGIE_FOLDER
rush update
rush build
```

### 4. Deploy the solution
### 3. Deploy the solution

Setup some environment variables which will be referenced by rest of the commands. Replace the following variables with the actual value.

| Name | Description |
|----------------------------------|-----------------------------------------------------------------------------------------------------------------------|
| AGIE_ADMINISTRATOR_EMAIL | Administrator's email. |
| AGIE_ADMINISTRATOR_MOBILE_NUMBER | Administrator's phone number (including the area code, e.g. +61xxxxxx). |
| AWS_REGION | AGIE's AWS Region deployment. |
| ENVIRONMENT | Environment (.e.g dev, stage or prod). You can have multiple environment deployed in the same AWS account and region. |


```shell
$ export AGIE_ADMINISTRATOR_EMAIL=<AGIE_ADMINISTRATOR_EMAIL>
$ export AGIE_ADMINISTRATOR_MOBILE_NUMBER=<AGIE_ADMINISTRATOR_MOBILE_NUMBER>
$ export ENVIRONMENT=<ENVIRONMENT>
$ export AWS_REGION=<AWS_REGION>
export ENVIRONMENT=<ENVIRONMENT>
export AWS_REGION=<AWS_REGION>
```

Run the following command to start the installation of AGIE:
The installer will guide you through a list of configurable settings that you can apply to your environment deployment. Run the following command to start the installation of AGIE:

```shell
$ cd $AGIE_FOLDER/infrastructure
$ npm run cdk -- deploy --concurrency=10 --require-approval never -c environment=$ENVIRONMENT -c administratorEmail=$AGIE_ADMINISTRATOR_EMAIL -c administratorPhoneNumber=$AGIE_ADMINISTRATOR_MOBILE_NUMBER --all
cd $CLI_FOLDER
bin/run.js install -e $ENVIRONMENT -r $AWS_REGION
```

### 5. Deployment Validation
### 4. Deployment Validation

To validate the deployment, open the CloudFormation console and verify the status of the following stacks

Expand All @@ -152,28 +131,18 @@ To validate the deployment, open the CloudFormation console and verify the statu
8. agie-\<ENVIRONMENT\>-engine
9. agie-\<ENVIRONMENT\>-scheduler

### 6. Deploy the UI
### 5. Set your initial password

Run the following the bundle the Web UI assets and upload it to the web artifact buckets:

```shell
$ cd $AGIE_FOLDER/typescript/packages/apps/ui
$ npm run deploy
```

### 7. Set your initial password

Run the following command to set the password for the administrator user (replace the `<PASSWORD>` with your own) :
Run the following command to set the password for the administrator user (replace the `ADMINISTRATOR_EMAIL` and `<PASSWORD>` with your own) :

```shell
export USER_POOL_ID=$(aws ssm get-parameter --name "/agie/$ENVIRONMENT/shared/cognitoUserPoolId" --query "Parameter.Value" --output text)

aws cognito-idp admin-set-user-password --user-pool-id $USER_POOL_ID --username $AGIE_ADMINISTRATOR_EMAIL --password <PASSWORD> --permanent
aws cognito-idp admin-set-user-password --user-pool-id $USER_POOL_ID --username <ADMINISTRASTOR_EMAIL> --password <PASSWORD> --permanent
```

Now that you have everything set up, visit [walkthrough](docs/walkthrough.md) to get started on working with `AGIE`.


## Next Steps

- Instead of relying on the container provided by AGIE, you can integrate your own custom image processing implementation into the application. This approach allows you to have greater control over the image processing functionality and tailor it to your specific requirements.
Expand All @@ -185,12 +154,13 @@ Now that you have everything set up, visit [walkthrough](docs/walkthrough.md) to
When deleting resources, please follow these steps:

```shell
$ cd $AGIE_FOLDER/infrastructure
$ npm run cdk -- destroy --all
cd $CLI_FOLDER
bin/run.js remove -e $ENVIRONMENT -r $AWS_REGION
```

Also, we've configured some resources not to be deleted automatically. Please follow these steps to delete them manually:

1. **Open the AWS Management Console**: Go to the AWS Management Console (https://console.aws.amazon.com) and sign in with your AWS credentials.
1. **Open the AWS Management Console**: Go to the AWS Management Console (<https://console.aws.amazon.com>) and sign in with your AWS credentials.

2. **Navigate to the S3 Service**: In the AWS Management Console, search for the "S3" service or select it from the list of services.

Expand All @@ -211,7 +181,3 @@ Also, we've configured some resources not to be deleted automatically. Please fo
8. **Confirm the Bucket Deletion**: A confirmation dialog will appear, asking you to type the bucket name to confirm the deletion. Type the bucket name exactly as shown and click the "Delete bucket" button to proceed.

9. **Repeat for Other Buckets**: If you have multiple S3 buckets created by your CDK deployment that you want to delete, repeat steps 4-8 for each bucket.




Loading

0 comments on commit 3bbf781

Please sign in to comment.