Searching...

Matching results

    How to use the Firehose Connector

    Integrate and extend your AirVantage platform

    This overview describes what you can do with the AirVantage Firehose Cloud Connector via a simple use case: Connecting your system to AWS Kinesis Firehose and accessing the raw data directly in the data store you specified (S3, redshift, Elasticsearch Service, etc.).

    1 / Principles

    2 / Configuration

    3 / Get messages

    Principles

    This section explains when and why to use this cloud connector, what it is, and the available options. To learn more about Kinesis Firehose see the AWS documentation web site .

    Introduction

    The Kinesis Firehose Cloud Connector provides the follow capabilities:

    • Message notifications from AirVantage
    • Transform records with AWS Lambda
    • Robust messaging for applications
    • High-volume messaging
    • Direct storage in a developer-selected destination: the developer just has to read data from the selected storage (i.e., no stream handling)

    Parameters

    When you connect AirVantage to Kinesis Firehose Streams, you have to define the type of notifications you want: operations, data, alerts, or usages.

    Configuration

    This section explains how to configure your client to be robust, reliable, and secure. Please read this section carefully and provide the correct information to enable the end point on the AirVantage side.

    Prerequisites

    Set up an Amazon Web Services account here .

    Introduction

    The role describes the rights (called policies) allowed for an AWS account on a specific object (here, a Kinesis Firehose).

    In the schema on the right, we created an airvantageKinesisProducer role applied to an AirVantage AWS account for a given company defined by the company uid in the SecretKey.

    Allowed policies are those which allow push messages in the stream and the last parameter defines the Kinesis Firehose to use.

    Configuration Process

    Step 1. Create a Kinesis Firehose in the customer’s AWS account, into which AirVantage will publish.

    The source must be set to Direct Put to allow AirVantage to push data directly in a Firehose.

    Be careful! There is a charge for using Kinesis Firehose resources. So before creating your stream, calculate the number of shards you need according to the quantity of data you handle and estimate your corresponding bill.

    When creating a Firehose, AWS creates a role called firehose_delivery_role for you. Don’t use it with any external account: firehose_delivery_role gives access to Redshift, S3, or ElasticSearch.

    Step 2. Create a Role in the customer’s AWS account:

    • The Role type must be Cross-Account Access > 3rd party AWS account.
    • The Account ID must be 406826153012 (Sierra’s own AWS account ID).
    • The External ID must be the UID of the customer’s company in AirVantage.
    • In the role policy, click on Next and then Finish.
    • Edit the role to add an Inline policies: The role’s Policy should only allow Sierra to write into the Firehose (firehose:PutRecord, firehose:PutRecordBatch).

    Step 3. Supply the following information using the CRM and select Request Support:

    • Customer’s AWS Account ID (a 12-digit number).
    • Name of the Role you created for AirVantage.
    • Customer’s AWS Region ID (e.g., eu-west-1).
    • Name of the Firehose we should write into (not the full ARN).
    • Type of messages you want to receive:
      • Alert event: Any alert rule triggered will be send
      • Operation: Any operation state will be sent (creation, progress, success, or failed)
      • Usages: any SIM usage
      • New message sent by the device: any device incoming communication
        • Optionally, you can supply the data path to whitelist (only the values for this data will be sent in the data stream). To get the path for data, you can go to the timeline to view the data path in the tooltip as shown in the screenshot below:

    Test your setup

    You can check that the data received from AirVantage shows up in the stream monitoring provided by Amazon. Use the Put Record Requests graph to verify that your Firehose is connected to AirVantage. If no data is shown, you need to check your configuration.

    Get the messages

    To get the data, refer to the AWS documentation corresponding to the destination storage you have selected:

    TOP