Elastic Map Reduce (EMR)

Overview

Granulate gAgent installation on EMR allows seamless integration with plug-and-play support leveraging the EMR Bootsrap actions. The gAgent EMR installation enables deploying Granulate agent on an EMR cluster nodes by incorporating Granulate's bash CLI installation to the EMR boostrap actions.

Installation

Granulate gAgent can be added to an EMR cluster Bootstrap actions from either the AWS console or the AWS CLI.
  1. 1.
    Amazon EMR Console - you can optionally specify a bootstrap action while creating a cluster. When creating a cluster you can choose Advanced Options and then navigate to General Cluster Settings and then under Bootstrap Action select Configure and add under the Advanced Options.
  2. 2.
    AWS CLI - you can pass references to bootstrap action scripts to Amazon EMR by adding the --bootstrap-actions parameter when you create the cluster using the create-cluster command.
1
--bootstrap-actions Path="s3://mybucket/filename",Args=[arg1,arg2]
Copied!
To install Granulate gAgent on your EMR cluster you would need the following arguments:
  • Client ID - A unique customer license key that is required for the agent's authentication.
  • Service ID - A customer-defined service identifier used to correlate all the agents that are installed in the same customer service
  • EMR Cluster Name- The name of the EMR cluster you would like to install the gAgents on.
  • Download Bucket - The name of the S3 bucket containing the bootstrap scripts
AWS CLI
AWS Console
When creating a cluster you can add Granulate gAgent installation as a bootstrap action using the following command. Please note to choose the relevant cluster name and fill in the appropriate Client ID and Service ID.
1
aws emr create-cluster --name "Test cluster" ...
2
--bootstrap-actions Path="s3://<Granulate Bucket>/granulate_run_gagent.sh" ,Args=[<Client ID>,<Serivce ID>]
3
Copied!
  1. 1.
    Open the Amazon EMR console at https://console.aws.amazon.com/elasticmapreduce/.
  2. 2.
    Choose Create cluster.
  3. 3.
    Click Go to advanced options.
  4. 4.
    In Create Cluster - Advanced Options, Steps 1 and 2 choose the options as desired and proceed to Step 3: General Cluster Settings.
  5. 5.
    Under Bootstrap Actions select Configure and add, choose Custom Actions, fill in the path with Granulate gAgent installation, your Client ID, Service ID, and then choose Add.
  6. 6.
    Proceed to create the cluster. Your bootstrap action(s) will be performed after the cluster has been provisioned and initialized.
While the cluster's master node is running, you can connect to the master node and see the log files that the bootstrap action script generated in the /mnt/var/log/bootstrap-actions/1 directory.
Last modified 6mo ago
Copy link