Elastic Map Reduce (EMR)

Overview

gAgent installation on EMR allows seamless integration with plug-and-play support, and leverages the EMR bootsrap actions from either the AWS console or the AWS CLI.

Installation Instructions

  1. 1.
    From the Amazon EMR Console: You can choose to specify a bootstrap action while creating a cluster. When creating a cluster, choose Advanced Options and then navigate to General Cluster Settings. Under Bootstrap Action, select Configure and Add under Advanced Options.
  2. 2.
    From the AWS CLI: You can pass references to bootstrap action scripts in AWS EMR by adding the --bootstrap-actions parameter when creating the cluster using the create-cluster command. It will look like this:
--bootstrap-actions Path="s3://mybucket/filename",Args=[arg1,arg2]
To install gAgent on your EMR cluster you need the following arguments:
  • Client ID: A unique customer license key that is required for the agent's authentication.
  • Service ID: A customer-defined service identifier used to correlate all the agents that are installed in the same customer service.
  • EMR Cluster Name: The name of the EMR cluster you would like to install the gAgent on.
  • Download Bucket: The name of the S3 bucket containing the bootstrap scripts.
AWS CLI
AWS Console
When creating a cluster you can add gAgent installation as a bootstrap action using the following command. Please note to choose the relevant cluster name, fill in the appropriate Client ID and Service ID, and preserve the Client= and Server= prefixes.
aws emr create-cluster --name "Test cluster" ...
--bootstrap-actions
"Path=s3://download.granulate.io/granulate_generic_env_wrapper.sh,Args=[<https://s3.amazonaws.com/download.granulate.io/granulate_run_gagent.sh,CLIENT=><CLIENT_ID>,SERVICE=<SERVICE_ID>]"
  1. 1.
    Open the Amazon EMR console at https://console.aws.amazon.com/elasticmapreduce/.
  2. 2.
    Choose Create cluster.
  3. 3.
    Click Go to advanced options.
  4. 4.
    In Create Cluster - Advanced Options, Steps 1 and 2 choose the options as desired. Proceed to Step 3: General Cluster Settings.
  5. 5.
    Under Bootstrap Actions, select Configure and add, choose Custom Actions, and fill in the following:
    1. 1.
      Name: Granulate gAgent
    2. 2.
      Script Location: s3://download.granulate.io/granulate_generic_env_wrapper.sh
    3. 3.
      Optional Arguments (note the line breaks between arguments):
Pay attention to replace the <CLIENT_ID> and <SERVICE_ID> with the appropriate values and preserve the CLIENT= and SERVER= prefixes.
6. The filled form should look like this:
7. Click Add.
8. Proceed to create the cluster. Your bootstrap action(s) will be performed after the cluster has been provisioned and initialized.
Last modified 6mo ago
Copy link
On this page
Overview
Installation Instructions