Author: Chakradhar Danireddy
What is DynamoDB?
As you know there was growing demand for no-sql database DynamoDB was one among them was based on ‘key-value’ pair storage. DynamoDB provides single-digit millisecond performance have built in backup and restore can handle more than 10 trillion requests per day and can support peaks of more than 20 million requests per second, on top of this it provides low latency and high performance. DynamoDB provides on demand capacity mode where it scales automatically based on number of requests in given timespan.
DynamoDB mainly has Tables,Items and Attributes as we are accustomed of correlating to an Sql database at least initially before getting accustomed to no-sql database Item can be related to rows, Attributes can related to columns here there’s a catch dont completely relate it to an Sql database comparison is just for your understanding and correlation. DynamoDB doesn’t have any schema attributes and their data types may vary at each item level, but each item uniquely identified by Primary Key(Partition key)Â Â which is mandatory for each item and is predefined while creating a table. We can even have a composite primary key with SortKey let us understand them from below snippets.
Create table:
Mention Table name, partition name and sort key (define sort key in case if you require composite primary key).
Below table Test is created with Key1 and Key2 as primary,sort key respectively as mentioned earlier Key1,Key2 are mandatory attributes for each item. We can insert any number of attributes for each item with unique key1 and key2 dynamically.
In the below snippet  have inserted two items with different attributes containing unique Key1 and Key2. First item contains ‘Flatno’ and the second contains ‘contact’ but key1 and key2 are unique and mandatory. Likewise, you can insert any number of attributes varying at each item level.
I hope you got a good understanding of what dynamo db is before jumping into operations we are doing through mule please go through regarding data types below as it is key to understand operations we are doing later.
https://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_AttributeValue.html
DynamoDb Connector Mule:
If you have started connecting to DynamoDB through mule you should have already known that there was an readily available connector in mule, but nothing was documented clearly to correlate or how actually we need to need to send request even though we have very few resources online we find it difficult to correlate it with actual Amazon API. In This blog will try to make the correlation and understanding with respect to the actual Amazon API. Even though there are many operations we will focus on the operation which we mostly use please refer below Amazon API before getting started.
https://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_Operations_Amazon_DynamoDB.html
Let’s discuss on below operations in this blog:
- GetItem
- BatchPutItem
- DeleteItem
- Scan
GetItem:
This operation is used to get particular item based on Primary Key.
Request In dataweave:
%dw 2.0
output application/java
—
{
            Key1:{"S": "Test"},
            Key2:{"S": "Test"}
}
Connector Reference:
Above is a dataweave transformation we have done for GetRequest let us understand request in detail.
Key1- Represents Primary Key (Attribute Key)
Key2- Represents Sort Key (Attribute Key)
S- Represents datatype hope you have already gone through data types link referred above.
Test- Attribute value we have already inserted for primary and sort respectively into table.(Need to be mandatorily passed as string as we have defined data types as string while creating table above)
BatchPutItem:
Batch put item comes handy when we need to insert bulk records into DynamoDB even though each BatchPut item request will allow us to only insert 25 Items at a time we can leverage Mule batch processing and achieve desired throughput.
Note:Â Please refer AWS documentation for limitations of batch requests.
Request In dataweave:
%dw 2.0
output application/java
—
{
            "Test" : [Â
            {
                        "PutRequest": {
                                    "Key1" : {
                                                "S": "Test2"
                                    } as Object {class:"org.mule.extension.dynamodb.api.model.AttributeValue"},
                                    "Key2" : {
                                                "S": "Test2"
                                    } as Object {class:"org.mule.extension.dynamodb.api.model.AttributeValue"},
                                    "Telephone" : {
                                                "S": "8682748242433"
                                    } as Object {class:"org.mule.extension.dynamodb.api.model.AttributeValue"}
                        }
            }  as Object {class:"org.mule.extension.dynamodb.api.model.WriteRequest"},
           {
                        "PutRequest": {
                                    "Key1" : {
                                                "S": "Test3"
                                    } as Object {class:"org.mule.extension.dynamodb.api.model.AttributeValue"},
                                    "Key2" : {
                                                "S": "Test3"
                                    } as Object {class:"org.mule.extension.dynamodb.api.model.AttributeValue"},
                                    "Phone" : {
                                                "S": "132234456"
                                    } as Object {class:"org.mule.extension.dynamodb.api.model.AttributeValue"}
                        }
            }  as Object {class:"org.mule.extension.dynamodb.api.model.WriteRequest"}     Â
      ]
}
As you can see from the above Dataweave ‘Test’ is the name of table for which we are passing array of put requests, you can correlate the dataweave inside each PutRequest containing mandatory Primary Key,Sort key and additional attributes if any as per requirement and their respective values, datatypes defined.
Note : You can insert entries into multiple tables in the same BatchPutItem request.
Connector Reference:
Please find below highlighted entries inserted into the table.
DeleteItem:
Delete item request is used to delete particular items from the table.
Request in dataweave:(deleting record inserted previously in BatchPut Request):
%dw 2.0
output application/java
—
{
            Key1:{"S": "Test2" as String},
            Key1:{"S": "Test2" as String}
}
Connector Reference:
Scan:
Scan is similar to ‘Select *’ in Sql database where as Sql we will get all the records, but in this scenario you will get paginated results as it will return records in table based on limitation in URL shared below.  Scan will return lastEvaluatedKey in case of any records still available in database it depends on how you design your logic using paginated response.
Initial Scan Connector Snippet:
Incase of records are still present in the table refer below snippet for querying based on lastEvaluatedKey.
Please refer below url regarding scan limitations.
https://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_Scan.html