Labels

Aws (32) MySQL (20) Aurora (12) Rds (7) DynamoDB (6) Cloudformation (4) Python3 (4) Unix (3) MongoDB (1)

Tuesday, 6 June 2017

DynamoDB Local and Global Secondary Indexes

Main features

  • LSI allows another SORT key, SAME partition key
  • Only at TABLE CREATION
  • Max Five per table
  • LSI are Asynchronously, reads from LSI are eventually consistency
  • WCU and RCU Shared with the TABLE!!
  • Eventual or Immediate Consistency
A LSI can make quicker and cheaper process quering only for the data that we need. For instance if we are insterested in a attribute that not always is available, we can have a LSI with the SORT KEY on this attribute.

Cloudformation

What is to highlight - The LSI sort key, must be listed in the attributeDefinitions - Projection is a mandatory parameter - The all AttributeDefinitions are mandatory while you create a new Item
{
  "Resources" : {
    "table" : {
      "Type" : "AWS::DynamoDB::Table",
      "Properties" : {
    "TableName" : "weatherstation_data2",
    "ProvisionedThroughput" : {"ReadCapacityUnits" : 1, "WriteCapacityUnits" : 1},
    "KeySchema" :
    [
      {"AttributeName" : "station_id", "KeyType" : "HASH"},
      {"AttributeName" : "dateandtime", "KeyType" : "RANGE"}
    ],
    "AttributeDefinitions" :
    [
      {"AttributeName" : "station_id", "AttributeType" : "S"},
      {"AttributeName" : "dateandtime", "AttributeType" : "S"},
      {"AttributeName" : "rainfall", "AttributeType" : "N"}
    ],
    "LocalSecondaryIndexes" :
    [
      {
        "IndexName" : "station_id-rainfall-LSI",
        "KeySchema" :
        [
          { "AttributeName": "station_id", "KeyType" : "HASH" },
          { "AttributeName": "rainfall", "KeyType" : "RANGE" }
        ],
        "Projection" : {
          "NonKeyAttributes" : ["station_id","rainfall"],
          "ProjectionType" : "INCLUDE"
        }
      }
    ]
      }
    }
  }
}
  • They are SPARSE index Property AttributeDefinitions is inconsistent with the KeySchema of the table and the secondary indexes

Projection

Projection operation is to copy the attribute from the table to the index. Represents attributes that are copied (projected) from the table into an index. These are in addition to the primary key attributes and index key attributes, which are automatically projected.
What you gain: - You can a smaller table projected with the attirbute that you only need, you can decrease the size of an Item and retrieve and pay for small consume units. - It also gives you then possibility to sort for a different key - If you "project" an NonKeyAttribute in the LSI it would be stored both into the main table and into the index LSI. (Double cost though)
Retrieving an attribute... -If it is in the list of NotKeyAttribute, it will be retrieved from the INDEX -If it is NOT in the list of NotKeyAttribute, it will be retrieved from the TABLE
ProjectionType The set of attributes that are projected into the index:
  • KEYS_ONLY - Only the index and primary keys are projected into the index.
  • INCLUDE - Only the specified table attributes are projected into the index. The list of projected attributes are in NonKeyAttributes.
  • ALL - All of the table attributes are projected into the index.

Item Ccollections

They are items that have the same P-Key and S-Key defined to create the LSI. It is important because LSI is stored on a single partition There is a phisical size limit to 10GB

Best practices

  • Don't create local secondary indexes on attributes that you won't often query.
  • Unused indexes, however, contribute to increased storage and I/O costs
  • The cost of I/O operations required to maintain the indexes can be significant.

Choose Projections Carefully

  • Because local secondary indexes consume storage and provisioned throughput,
    you should keep the size of the index as small as possible.
  • Specify ALL only if you want your queries to return the entire table item but you want to sort the table by a different sort key.

Optimize Frequent Queries To Avoid Fetches

  • To get the fastest queries with the lowest possible latency, project all of the attributes that you expect those queries to return.
  • If you query an index, but the attributes that you requests are not projected, DynamoDB will fetch the requested attributes from the table. To do so, DynamoDB must read the entire item from the table, which introduces latency and additional I/O operations.

Take Advantage of Sparse Indexes

For any item in a table, DynamoDB will only write a corresponding index entry if the index sort key value is present in the item. If the sort key does not appear in every table item, the index is said to be sparse.

Global Secondary Indexes

  • Can be created at any time
  • No limited to table creation
  • A new P-Key and S-Key
  • All, Include or Keys_only projection options
  • GSI have dedicated RCU and WCU
  • Item changes are written Asynchronously to the GSI
  • you CAN'T demand strong consistent
  • No ItemCollection constraints
  • GSI are also SPARSE indexes ( only item that have the GSI' P-Key are stored in the index)
  • Limit of 5 GSI's per table

Documentation links