ode~To~CLOUD

All about few development ideas & learning!

Importance of Consistency levels in Azure CosmosDB

by Leave a Comment

When any data is posted to the database, we look for the save action from CosmosDB to ensure consistent reads.

Strong Consistency?

We are very sure that the data we are reading is up to date, but there is always a waiting time for acknowledge if all the replicas are up to date. With Strong consistency, however, we always read the updated data which results in higher latency and higher consistency but lower availability and lower performance as everybody is waiting to receive the data to read from somewhere.

Stale Reads? (Bounded Staleness)

Stale reads mean that data you are currently reading is out of date and is not updated. Till some point, it is okay but you need to define that how much out of date, data do you need. So basically, it is okay to have dirty reads, but only till the level where your specified out of date is in control. After the out of date specified by you, reads will not be accepted and Cosmos DB will switch to strong consistency not allowing the stale reads too.

We can specify two things here to define how old is the data which you call the stale data.
– Lag in time
– Update Operations

Lag in time gives you the data that is older than maybe ‘t’ seconds, and the update operations give you the information of the data that has been updated for more than ‘x’ many times.

Another concept that comes in when talking about consistency levels is Session Consistency.

Session Consistency?

This is default level of consistency set for any database. Session consistency maintains a session for all the writers of the database by maintaining a unique session key. This session key then ensures that the writers will always have guaranteed read of the correct and updated data in any of the replicas, never getting a stale version.
However, this will only be applicable to the writers and the other users may still face dirty reads.

Consistent Prefix

In consistent prefix, it is possible that you are getting dirty reads, but one thing is made sure of here, and this is that whatever data you are reading is the one that is updated in all the replicas, however this is possible it is not the most up to date version.
Also, if the same data gets updated multiple times, we will get only that version which has been replicated. That means, we wont get access to the versions not replicated yet. So this acts as a guarantee provided by Consistent prefix.

Eventual

In Eventual consistency level, it is possible that you get dirty reads and there is no guarantee of the replicated data being up to date. But the good part here is that there is no waiting time as we need not wait for any acknowledgement that all the replicas have been updated.

This one is the total opposite of Strong Consistency as, in here there is no guarantee at all about the up to date reads from the replicas.

Overview of Azure cosmos DB Documents & Running the queries

by Leave a Comment

In the previous articles, you learnt about how to create collections, what are the features of NoSQL databases, why Cosmos DB is called a NoSQL database. Now that we have these basic pre-requisites clear, it is time to go on to work with some data creating documents on Cosmos database.

We have already created a collection called Human. To learn how to create collections, refer to the previous article, “Overview of Azure Cosmos DB collections”.
Now when we expand the Human collection, we can see the option to create a document. This is where we put in all the data required for the document.

As we go on to create a document, this is the display to us. Now, when we create the documents and provide the JSON data, it creates a unique ID for every document when we save it. See image below for reference.

After putting in the data and saving the document, it updates the document with all the id and other attributes like rid, self, etc. We have created two documents and now let’s run the queries by going to the New SQL query option on the top left.

As we see in this image, we have put the query to select all the results which have this particular pin code and it has successfully given us the documents data which has this pin code. Another important thing to note here is that ‘c’ is being used as an alias for the collection name.

We can try different things by providing different properties to any document and then running queries.

Now, suppose the query goes something like this:

In this case, it returned only one document where family name is “Anirudh”.

A good thing about this schema-free database is that you can anytime return to it with more properties and update them in the document because there is no schema management and we can update the data whenever we need to. We can also update this using code, not just manually.

We did not incur any costs while creating all these documents and implementing them using queries. This was because we are using the local emulator. However, when we need to scale out our application and distribute it globally, we would have to use the real cosmos DB account and for that we will be charged the cost required. Deploying to the cosmos DB account, though, will be very simple by changing the connection string of the emulator to that of the Cosmos DB account.

In this example, we have manually written all the data inside the documents, however, we can code and implement this using Cosmos DB which is out of scope of this blog post. However, we can also create collections and documents through code implementation. 

Multi-model Cosmos DB?

by Leave a Comment

In the previous article ‘Creating collections and Documents in Cosmos DB’, we saw how we can create documents with the JSON data and how we can create it later. Referring to Cosmos DB as multi-model actually means that how you want to obtain your data in the results and that’s when you chose one data model over another. To choose different data models, we select any of the multiple available APIs. Here the job is for the developer to decide as to what type of data it is that is being processed and how should it be presented be it, in graph, table, document or columnar form.

However, one very important thing to keep note of, here is that Cosmos DB uses ARS (Atom record sequence) to store the data and choosing any of the data models doesn’t play any role here. 

The different data models that Cosmos DB provides to work with are:

  • Document data model that uses SQL API for JSON documents or MongoDB API which works with BSON documents, i.e. JSON documents with support for binary input as well. The MongoDB API also extends wire level support which makes it easier for MongoDB developers to expand their area of work and enjoy the performance with Cosmos DB.
  • Graph data models using the Gremlin API which provides a traversal language that helps build a relationship between the data and have access to more data with the help of those relationship among the data.
  • Columnar data models using the Cassandra API where in we define the schema and the data is set into the form of columns 
  • Key-value Data model which uses Table API wherein all the entities can have a different set of key-value pairs. The major use of Table API is that any of our data on the Azure Table Storage can be easily migrated on to Cosmos DB and can be worked well on.

However, which so ever API we choose, it hardly matters apart from the presentation of the data and they simply project our data in different data models. The data is simply stored under ARS. All of the data is actually present in the form of keys and values. Key and value pair is what Table API supports, but actually if we look at it technically, it all the relationship between these keys and values pairs. even for the graph and columnar data models, the data is actually having a key and a value for the key which is just projected differently! This is what ARS actually means, a layer for working with the key value pairs.

If we look at the graph data model which has vertices and edges, they are actually a pairs of keys and values only. Similarly, in the columnar data model, the columns containing the data are just the keys and values

Another added functionality is that within a database that we have created for one project and choosing one API, we are actually allowed to switch to different APIs that makes the whole work a lot easier. Suppose the project has been worked on using the SQL API, but then you want it to be projected in the graph data model, in that case, you will be able to switch to the Gremlin API and looks at the modified results.

Globally Distribute your data CosmosDB

by Leave a Comment

To start, you might know by now that CosmosDB creates a replica of all the data that we provide in the database. It starts by creating 4 replicas by default and then keeps on adding replicas of the data as it grows. The more are the replicas in the database, the more will be the data availability.

Now that we are clear with the concept of replicas in the CosmosDB, we must understand that to globally distribute the data, we look at the map and click on the location where we want to replicate the data to and save it. This replicates the whole data created in the local Azure CosmosDB to that location following some geo-fencing policies that are set like some countries do not allow the data to be replicated outside their own country. 

The other case is when a particular location searched for is not available, you can dynamically add or remove the region. There is another option called failover properties which means that you can set one desired Azure CosmosDB region for the data that you want to store but in case that region is unavailable, you dynamically add another region in place of that.

When accessing the data from the same Azure Data Center, the process of running and retrieving the data is super fast because that is where the whole data is located. However, if you try running the same data for some other location now, you will get results very slowly. To solve this problem, we replicate the data globally as follows:

After clicking the replicate data globally option, we chose the regions as the read regions and the write regions and click save. We can choose multiple regions as read and write regions. This replication process takes quite a long time and needs a wait time.

After setting this up, we also need to configure the order of regions and the fail over properties required to set up the regions. Now we need to establish a connection policy which says which region to try first for reading data. To do that, we write a simple code in our .cs file as follows:

var connectionPolicy= new ConnectionPolicy();
ConnectionPolicy.PreferredLocations.Add(ConfigurationManager.AppSettings[“PreferredRegion1”]);
ConnectionPolicy.PreferredLocations.Add(ConfigurationManager.AppSettings[“PreferredRegion2”]);

To reference this policy with the client, we use:

var client = new DocumentClient( new Uri(endpoint), masterKey, connectionPolicy)

After having done this, we need to acquaint ourselves with another concept called Dirty Read. 

What is Dirty Read?

We know that when the data is replicated 4 times as default in one Azure Datacenter, we need to make sure that the data we are reading at any given point of time is the latest version of that data. It is very much possible that the data that we are reading in any one of the total replicas produced, is not the latest version of the original data.
So, in case, we are reading the old version of the data in any of the replicas, this is then called Dirty Read.

So basically, there needs to be a consistency of data in all the different replicas that we have of the same data. Azure CosmosDB provides us five consistency levels ranging from Strong to eventual.

Strong here refers to:
– Higher Latency
– Strong Consistency
– Low Availability

whereas Eventual means:
– Low Latency
– Weaker Consistency
– Higher Availability

In case of Strong consistency, it is ensured there is no Dirty read but the cons here will be that data will be available for reading only when all the replicas are updated, resulting in increased wait time.

In Eventual, there is no guarantee of Dirty Read but no waiting time too.

However, the concept of consistency and dirty read comes into picture mainly when we start geo-replicating the data, because in one Azure data center the data moves quite quickly resulting in all the replicas getting updated very quickly, lessening the chances of Dirty reads.

Getting Started With Azure Cosmos DB

by Leave a Comment

After a short blog post on the technical overview of Azure Cosmos DB, we will put some light on what we mean when we say that Cosmos DB is a NoSQL database and that how has it evolved over time, in this article.

Azure Cosmos DB is Microsoft’s NoSQL database platform running on the cloud. A NoSQL database, in simple terms, is a distributed database that maintains replicas of the databases and provides it as an output to a no. of users. NoSQL databases have a variety of features like scale-out, no schema management etc. We will understand each one of these further. Another important thing is that NoSQL, unlike the name suggests, doesn’t mean that there is no us of SQL. In fact, it uses SQL to query the NoSQL databases.

One of the important features of NoSQL is horizontal partitioning that we do to scale out the throughput, and that is how Azure Cosmos DB manages the growth of huge data by server-side horizontal partitioning. Once the throughput required i.e. the units of information or processes that will pass through the system, is specified to Cosmos DB, it creates that no. of replicas and gives the performance

Among many other features of Cosmos DB, which is an evolution of DocumentDB, Microsoft’s NoSQL database that supports schema-free JSON documents, the best one is that it has multiple APIs that extend support to different data models like Tables, graphs, documents etc. This was not the same case with Document DB as it used to support only the document data models. Being schema-free doesn’t mean that there is no schema at all, but means that there is no schema management required.

Cosmos DB has multiple APIs (application programming interface) and supports multiple data models. We use different APIs to work with different data models depending on the type of structure that we want our data to be in. Document DB is used to support only JSON documents which is now known as the SQL API in Cosmos DB. For BSON (JSON documents with support for binary data) documents, Cosmos Db has a MongoDB compatible API which is called MongoDB API. To support the graph-based data models, Cosmos DB has Gremlin API, key-value databases using the Table API, and the columnar databases using the Cassandra API.

Another feature of Cosmos DB, i.e. the global distribution feature of Cosmos DB makes it accessible anywhere across the globe by replication on simply a click and mapping it on to the location where it was pointed to.

With the support for different data models, Cosmos Db is spreading all across and for you to try your stint with Cosmos DB, here are some steps to follow to get started with Azure Cosmos DB.

Step 1. Create or log in to your Microsoft account and set up your details. https://signup.live.com/


Step 2. Now that you have your Microsoft account ready, grab your Azure subscription at azure.microsoft.com

Step 3. You are good to go!

Azure Cosmos DB is very friendly, so in case, you don’t want to create a Microsoft account or an Azure subscription, there is a local emulator that can be downloaded and used without the need of a network connection. Emulator will help us run the Cosmos DB service just like any other service, but the only thing that would not be supported by the emulator would be global distribution.

We will talk more about what and how Emulator helps us work with Cosmos DB without the need of creating a Microsoft Azure account, in the next blog post.