Searching...

Matching results

    How to use the Firehose Connector

    Integrate and extend your AirVantage platform

    Here we give you an overview of what you can do with the AirVantage Firehose Cloud Connector via a simple use case: Connecting your system to AWS Kinesis Firehose and accessing the raw data directly in the data store you specified (S3, redshift, Elasticsearch Service, …).

    Principles

    This section explains when and why to use this Cloud connector, what it is and the available options. You can know more about Kinesis Firehose on AWS documentation web site

    Introduction

    The Kinesis Firehose Cloud Connector provides:

    • message notifications from AirVantage
    • Transform records with AWS Lambda
    • Robust messaging for applications
    • High volume messaging
    • Direct storage in a developer selected destination: developer just has to read data from the selected storage (no stream handling)

    Parameters

    When you connect AirVantage to Kinesis Firehose Streams, you have to define the type of notifications you want: operations, data, alerts or usages.

    Configuration

    This section explains how to configure your client to be robust, reliable and secure. Please read carefully this section in order to supply correct information to enable the end point on the AirVantage side.

    Prerequisites

    Set up an Amazon web services account here .

    Introduction

    The role describes the rights (called policies) allowed for an AWS account on a specific object (here a Kinesis Firehose).

    In the schema on the right, we created an airvantageKinesisProducer role applied to AirVantage AWS account for a given company defined by the company uid in the secretKey.

    Allowed policies are the ones allowing to push messages in the stream and the last parameter defines the Kinesis Firehose to use.

    Configuration Process

    Step 1. Create a Kinesis Firehose in the customer’s AWS account, into which AirVantage will publish.

    The source must be Direct Put in order to allow AirVantage to push data directly in the firehose.

    Be carefull ! There is a charge for using Kinesis Firehose resources. So before creating your stream, calculate the number of shards you need according to the quantity of data you handle and estimate your corresponding bill.

    When creating a Firehose, AWS creates a role called firehose_delivery_role for you. Don’t use it with any external account: firehose_delivery_role gives access to Redshift, S3 or ElasticSearch

    Step 2. Create a Role in the customer’s AWS account:

    • The Role Type should be Cross-Account Access > 3rd party AWS account.
    • The Account ID should be 406826153012 (Sierra’s own AWS account ID).
    • The External ID should be the UID of the customer’s company in AirVantage.
    • In the role policy, click on Next and then Finish
    • Edit the role in order to add an Inline policies: The role’s Policy should only allow Sierra to write into the Firehose (firehose:PutRecord, firehose:PutRecordBatch).

    Step 3. Supply the following information using the CRM and select Request Support:

    • Customer’s AWS Account ID (a 12-digit number).
    • Name of the Role you created for AirVantage.
    • Customer’s AWS Region ID (e.g. eu-west-1).
    • Name of the Firehose we should write into (not the full ARN).
    • Kind of messages you want to receive:
      • Alert event: any alert rule triggered will be send
      • Operation: any operation state will be sent (creation, progress, success or failed)
      • Usages: any SIM usage
      • New message sent by the device: any device incoming communication
        • Optionnaly you can supply the data path to whitelist the data (only the values for these data will be sent in the data stream). To get the data path for a data, you can go to the timeline to view the data path in the tooltip like in the screenshot below:

    Test your setup

    You can check that the data received from AirVantage shows up in the stream monitoring provided by Amazon. The graph that describes the number of Put Record Requests lets you ensure that your Firehose is connected to AirVantage. If no data is shown, you need to check your configuration.

    Get the messages

    To get the data, refer to AWS documentation according to the desination storage you have selected.

    TOP