Wondering what to do if you do not have an Azure subscription? Well, Azure Cosmos DB can be used to write and test applications locally using the Cosmos database. And to do that, we do not need to have an Azure subscription at all. We can simply download the local emulator for Cosmos DB and test our applications inside the system without the need of an Internet connection, the only downside being that we won’t be able to work with the feature of global distribution using the emulator.

To work with emulator, we just need two things:

• Master key
• Connection string of the emulator

The key can be found on the documentation on the Azure Cosmos DB emulator and the connection string plays an important role when it comes to connecting with the real Cosmos DB account. When we want to deploy our code on to the real account, we can simply change the connection string of the emulator to that of the real Cosmos DB account, and there we go!

Steps to install the local emulator:

• Download the local emulator from the browser and install it. You should get a screen like this. You can simply search to download the emulator on Google.

• Once it is installed, you’ll get the access to the portal and then comes the master key requirement which can be found in the Documentation link given within the emulator.
• Grab your master key, and you can then play around with the Cosmos DB emulator which is no less than a real Cosmos DB account.

Now let us try to understand the first screen that we get when we start the Emulator.

So here is the quick start page where the first option is URI, which is the identifier for your emulator link. If you see in the address of the browser, it is the same link as URI.

Next is the Primary key which is the unique identification of your account and this is what maps and saves what you do in your emulator from creating collections to working with documents. Well, we will come to that later.

Then comes the connection string which is going to help you deploy our project on to the real Cosmos DB account. This is Emulator connection string which will be then replaced by the Cosmos DB account connection string. Also, we have a separate connection string for MongoDB API users since MongoDB API support is like a wire level support for those who are already working with MongoDB and this string will help them deploy their project and work around with the Cosmos DB functionalities.

On the left side panel, if you see the next option is the Explorer, where we can create the collections and documents. Since the word collections is coming a lot into use, let us understand it as a container for all the huge data that we will use on Cosmos DB, for now. Now this container has different names for different APIs, like the SQL API calls it collection, whereas the Gremlin API will call it Graphs and so on.

The explorer tab look like this:

On the top left is the option to create a new collection that will be the container for your data.

In the next blog post, we will go into the details of how to start working with Cosmos DB creating collections, documents etc. 

Assuming that you have already created your Cosmos DB account or have installed the local Cosmos DB emulator, let us take the first step in our journey of working with Cosmos DB.

The first step is creating some data and for that, we need to have something that contains this data. This is how the concept of collections comes into picture. 

To create a collection, we get the option to create a New Collection on the data explorer tab which looks like this:

Step one is to provide the id for the database that we need to create and then provide the collection id. In case, we do not have multiple collections, then the data and collection are almost the same things. However, we can definitely have multiple collections inside the database.

Earlier Cosmos DB used to provide the feature to choose the storage capacity from two options:
– Fixed (10 GB) i.e. one physical partition
– Unlimited

But now, this feature has been deprecated as it scales down the elasticity. The minimum throughput has now been lowered to 400 RUs(Request Units). See the next image for reference.

Also, it is always better to avoid creating different fixed size collections when cosmos Db can do the whole work if we choose Unlimited storage capacity and all the partitions for multiple collections will be created.

Now since we let Cosmos DB do the partitioning for us, we do need to specify the partition key on the basis of which it will classify the data into different collections. It is used to group multiple partitions together.
So the format to do that is as shown in the image above, to use /address/zipcode. So all the data that will belong to a particular zip code will be allocated to one collection and as the data will grow later, Cosmos DB will assign it automatically by simply looking at the Partition key of that.

A simple example of this is here:

Next comes throughput, where in we request for the performance level that we need for the project. This performance level is measure in the form of RUs (Request Units). On the local emulator , any throughput value wouldn’t matter but on a real Cosmos DB account, we need to pay for the performance level we request for. That means that if the throughput value is low, we pay less cost for low performance. For greater value of throughput, there is high performance and therefore, high cost for the performance. For the same reason, it is preferred to create multiple collections inside the same database.

You can see we have provided maximum throughout, reason being, on the emulator, whatever no. of RUs you provide, it really doesn’t matter, so we can have it maximum.

After we have created the collection, it looks to be like:

We can see the database and the collection being created. When we expand the collection, there we get the option to create documents.

Creating the documents is an important furtherance to our work with Cosmos DB which we will discuss in the next blog post.

---

In this blog post, we will look at the resource model of Cosmos DB and then look at the resource properties, identifiers etc. which help to uniquely identify the documents.

In CosmosDB, every resource is actually a collection of resources. This works like, we have a Cosmos DB account which contains of databases which eventually contain data containers. These data containers are known as collections. The JSON documents are stored inside collections when working with SQL API.

Azure Cosmos DB account -> Databases -> Collections -> Documents -> Attachments

The collections further contain documents which when expanded can contain a variety of data. Documents contain attachments also like images etc. but when trying to use attachments for large data maybe BLOB (Binary large object), using attachments isn’t really a good option since it has a fixed limit of 2 gigabytes.

As an alternative Azure provides Azure BLOB storage to contain that BLOB data and the links to these BLOBs can then be stored as a properties in Cosmos DB account.

Documents -> Users -> Permissions

We also have users and permissions in Cosmos DB that is used for granular security. We have seen that we can use the master key to access all the resources, databases and collections across the data, but if we want access to specific data, we can use user permissions which grant the access only to particular resources.

---

Some of the Resource properties are :

id property

Every resource has an id property assigned to uniquely identify it. however, if we do not specify it, Cosmos DB uses a default created id for a particular resource.

Apart from the id property there are many more auto-generated properties prefixed by underscore that show up when we create the document in Cosmos DB. They are:

-rid property

Resource id. This is an id that is always set and can never change. Every resource has basically 2 IDs. The regular id which can be set and modified, and the resource id which cant be played around with.

_ts property

This is the timestamp property which holds the date and time for the last update status of the resource.

_etag property

Cosmos DB uses it internally to check for concurrency.This means when one user has made some changes, the etag property will change and the other user will get a prompt that this was modified recently by …

The etag property contains binary values so it always changes when the resource changes. This prevents one user from overwriting the other’s changes.

_self property

This property is the full path to the resource, nested resources and even the attachments.

Let us look at how it looks in the Cosmos DB.

So there are three way to identify a document.

• self link
• ID-based routing
• URI

Self Link

Collection of all your resource ids. Its starts with dbs which is databases followed by the resource ids and then has /colls/ which stands for collections followed by collections resource id, then /docs for documents and finally ends with a slash to refer to the documents and resources contained within.

ID-based routing

if we do not recall all the resource ids of the resources that we created, it is better to use ID-based routing which takes simply the ID which we can easily refer to and use. It also doesn’t have a terminating slash. We can simply use the collection and database ID and then the document ID.

URI

It is best to use URI which uses URIConnectionFactory to create a URI. It has its own syntax to create a URI and we do not need to construct one by ourselves. It has a create URI method for all the types of data. 

In case of document data, we use createDocumentUri(). It will take three parameters, database id, collection id and document id to create the URI for us.

We all work with data somewhere or the other, and migrating it from one place to another sounds like a hassle in itself.

In this blog post, we will see how easily we can migrate any of our data into Cosmos DB. To migrate your data from any source to Cosmos DB, you can use Data Migration Tool which is an open source utility available on Microsoft Download Center.

To download this tool from the Microsoft Download center, go the link below:Microsoft Download Center

Also, you can refer to the Azure documentation which provides the link to the download center. Another good thing with this being an open-source project is that its source code is available on GitHub.

To access the source code for this project on Github, go to the link below:Azure/azure-documentdb-datamigrationtool
Azure DocumentDB Data Migration Tool. Contribute to Azure/azure-documentdb-datamigrationtool development by creating an…github.com

This contains all the supported data sources from where you can import the data inside your CosmosDB project. Some of the supported sources for importing your data from are:

• SQL server
• Mongo DB
• Azure Table Storage
• CSV files
• Flash files
• JSON data

When we import the data from JSON files, it is simple to import them as they are but to import the data from relational databases, it is not very easy as it needs to be tweaked.

By this transforming, we mean that the normalized data in SQL server has to match up the de-normalized way of the data in the JSON.

But how do we import this data? Let us take a look at that.

To start the import of data from a SQL server into Cosmos DB, we take a sample SQL data from AdventureWorks.

What is AdventureWorks? In simple terms, a sample database for SQL server. Now inside AdventureWorks, pick up any of the views. Views? Another terminology!

Views are mainly a combination of related table entities. Open any of the views from the database and when you look at the data, each of the data is derived from different tables of that view. Select the query and using the id of the data, which is actually a string that will be used as the data import identifier into the data migration tool.

Inside the data migration tool, you then define connection strings, indexing policies etc. to connect the data from your sample SQL server to the Data migration Tool which eventually will import all the data into your Cosmos DB account provided, you have put in the URI, connection string and other required details of your Cosmos DB account.

This article is just a brief introduction of Data Migration Tool. However, installing the data migration tool and getting it into Cosmos DB using a sample SQL server is a task in itself. In the next blog post, we will see the installation of Data migration tool and import of data from SQL server into Cosmos DB.

Summary

In this blog post, we started off by discussing the role of migration of data from one tool to another, followed by the access to data migration tool, an open source utility which allows us to import data from many data sources into Cosmos DB. We also saw how when getting to bringing in data from a SQL server, AdventureWorks proves to be very helpful by the provision of sample SQL server.

In the previous article, we saw the five different types of consistency levels ranging from Strong to Eventual. In Cosmos DB, to set the consistency level, we start by setting the default one from all the available consistency levels for every account entirely. This default consistency level will be applied to all the operations inside the accounts under the databases. This will be set till the time we change it and even if we do, we can go back and activate it any time.

To set the consistency level, we go to Explorer -> Settings -> Default Consistency.

As you can see it is set to Default Consistency by default. You can set any of these consistency levels from here.

Suppose, you choose Bounded staleness, you will see the options to set the acceptable level of freshness, which means by setting the maximum lag of operations i.e. the no. of operations by which you are lagging behind and the maximum lag in terms of days, hours, minutes, and seconds.

It looks like this:

We can set these control values and in the above picture, we are setting the no. of operations by which are allowed to lag behind as 100 and time to be 5 seconds and not more than that. We definitely do not want stale results more than these set control properties.

Another thing that we can do is override the default consistency level for each particular query but by only weakening the consistency level. This means that if your default consistency is session, then you can set the consistency level per query to only consistent prefix or eventual. Not the higher ones.

Similarly, if the set consistency for the entire account is Strong, you can set the consistency per query to any of the consistency levels because all other consistency levels are weaker than this one.

This option is better to use when we set it to Strong for no dirty reads with higher latency and then we can set it to lower consistency levels for particular queries for which we want the data as quickly as possible and it is totally fine if it is stale data too.

Let us see how do we do this.

Step 1. Use the .NET sdk, go to the DocumentClient class

Step 2. In that class, choose the overloaded constructor that accepts a consistency level enum.

It should look like:

new DocumentClient(new Uri(endpoint), masterKey, connectionPolicy, ConsistencyLevel._______________)

This blank can have any of the five available consistency levels:
– BoundedStaleness
– ConsistentPrefix
– Eventual
– Session
– Strong