In my previous article I gave an introduction to Azure Blob Storage. Now that you know all the wondrous things about Blob Storage, the question is how do you get some of it? I’m going to assume here that you have an Azure account. If you don’t have one, you can go to http://azure.microsoft.com/ and click on the “Try for Free” button to sign up for a free trial account.
To start with Blob Storage, you need to set up an Azure storage account. First, log into your Azure account on the current portal at http://manage.windowsazure.com/. Click on ‘Storage’ on the left-hand side to access your storage accounts.
When I do this on my new Azure account, it says I have no storage accounts, and prompts me to create one. You can also click the big +NEW button on the bottom left-hand side of the screen, and select DATA SERVICES, then STORAGE, then QUICK CREATE. You will then be prompted for three fields.
For the URL, fill in a name for your storage account. This has to be unique amongst all of the storage accounts in Azure. If it is not unique, it will show a red circle with an exclamation mark in it, and won’t let you create the storage account until you fix it. When it’s okay, it will show a green circle with a checkmark in it.
The next field is LOCATION/AFFINITY GROUP. Click on the dropdown and select the region closest to you. I recommend ignoring “affinity group”. These are discouraged for new accounts in favor of region. For more information about affinity groups and regions, check out this article by Neil Mackenzie.
For REPLICATION, Locally Redundant is the least expensive option and is more than adequate for our purposes. For more information on the different kinds of replication, please read my previous article.
After making your selections, click the check mark to create your storage account, then wait a bit until the status is “Online”. Congratulations, you have your very own storage account and can now fill it with Blobs.
How do I put Blobs in my new storage account?
There are several free products you can use to upload Blobs to your storage account. There is the Azure Storage Explorer on Codeplex and the CloudBerry Explorer. Cerebrata also has a simple product called Azure Explorer that you can use to manage Blob Storage. You can also use the Storage Client Library to create your own client; this will be covered in the next article. In addition to these options there is some limited functionality in Visual Studio’s Server Explorer and in the Azure portal as well. The Azure Storage team put together this list of storage explorers if you want to more information about the various products.
I’ve been working with Azure since 2010, and have found that I can’t live without Cerebrata’s Azure Management Studio (AMS). I know that sounds suspiciously like product placement since this blog is supported by Cerebrata (part of Red Gate), but if you’ve seen me speak or read my blog entries, you will find that I frequently use this in my demos and blog articles, so that’s what I’m going to be using here. You can use any application that works for you, or try out the free trial if you want to follow along specifically.
Set up Cerebrata’s Azure Management Studio
Now that I’ve created a storage account, I’m going to show how to set it up in AMS. Then we will see what’s in the storage account, upload files, and download files. Click on the storage account name; it should show the Dashboard. If it goes to the Quick Start page, click on through to the Dashboard. At the bottom of the page, there are three options.
Click Delete to delete a storage account. “Think long and hard before you delete a storage account, because there is no going back.” -Voice of Experience (Coincidentally, this sounds a lot like my voice.)
“Manage domain” allows you to set up a custom domain, so that rather than using myneatstoragename.blob.core.windows.net, you can use something like myneatstoragename.myneatcompany.com.
To access the storage account you need the name of the storage account and one of the access keys. Click MANAGE ACCESS KEYS to see this screen:
Each storage account has two access keys, which makes it easy to change the credentials of the storage account when needed. Let’s say you have several applications that use the primary access key. When you need to change the key used, you go through and change all of them to the secondary access key, verify that everything works, and then regenerate the primary access key so the old value no longer works.
Click on the Copy icon next to the primary access key to copy the key into the buffer. (Don’t waste your time trying to use the key displayed above; the keys are actually much longer than text displayed!)
When you first run AMS, it will prompt you to download your publishing settings. You don’t need to do this simply to access Blob Storage, so feel free to skip that step. Now you should see this:
Add your new storage account by clicking on “Add Storage Account Connection…”. You will see this screen:
You can run Azure applications locally using the compute and storage emulators. When you do this, the storage emulator uses a local SQL Server database for storage and surfaces the data to you as Blobs, Queues, and Tables. (Hence the use of the word “emulation”). To use this local storage, you set the connection string to “UseDevelopmentStorage=true”. If you want to view that development storage, you can add that account by clicking “Add the Development Storage account…” on this screen.
We want to use a regular storage account, so click “Add a Windows Azure Storage account…”.
Fill in your storage account name and paste the account key that you copied into the Windows clipboard from the portal. The other settings are fine as they are. Click OK to add the storage account connection. Now you will see it on the left-hand side under the default connection group (My Connection Group).
You can see all of the bits of storage – Tables, Queues, and Blob Containers. If you open Blob Containers, you’ll find you don’t have any. You need to add one or more containers before you upload any files. You can think of this as a top-level folder for your Blobs. Think a bit about the name of the container, because it is displayed in the URL used to access the file. It might be okay to name it plants-vs-zombies if you’re going to store plants and zombies in it, but your company may not appreciate its use for customer data (like http:// mycompanystorage.blob.core.windows.net/plants-vs-zombies/report-for-ceo.pdf).
If you right-click on Blob Containers, you will see your options to create a new container. Note that if you have files that need to reside in the root, you can create a Root Blob Container, and any files placed in there will appear to be in the root. This is often used for cross domain files and robots.txt files. If you have a website that is all static content, you can host it in Blob Storage, at which point you may need one or more of those files in the root.
Right-click on Blob Containers and select “New Blob Container”. This dialog will appear.
This dialog will appear.
Fill in the name of your Blob container and select an ACL that defines the type of access available for that container.
- Private means only someone with the credentials for the storage account can access the container and the Blobs therein. Use this if you don’t want anybody to be able to access the Blobs without going through your application. You can access the files in private containers programmatically through a Storage Client Library using the storage account credentials, or with a URL that contains an expiring Shared Access Signature (SAS) token. The use of SAS will be covered in a later article in this series.
- Blob means the container is private but the Blobs are public. This means nobody can iterate through the container to get a list of the Blobs unless they have the credentials for the storage account. To access a Blob you have to have a URL that specifically points to the Blob.
- Container means the container is public and the Blobs are public, so if someone knows the container name, they can access all of the Blobs, and they can get a list and iterate through them.
Here’s an example where setting the ACL to Private is a good idea. Let’s say you have an application where the customer enters notes and some data, both of which are stored in a database. When the customer is finished, he hits a button to process the data which can take a few minutes. After the processing completes, it sends an automated e-mail to the customer with the notes and a URL to a file with details and results of the processing. The customer might want to share the notes, but not the file; if he forwards the e-mail to someone else, they would be able to retrieve the file if the container was public.
Instead, you can set the container to Private. When the user asks to download the file, generate a URL that points to the application which has the storage credentials needed to access the file, and have the application require authentication before letting the requestor download the file.
Another option would be to create a special token called an SAS token (Signed Access Signature), which is added to the URL. This token is created using the storage credentials to allow access to private files without knowing the storage credentials, and can have an expiration time.
If you set the ACL to Private and use a URL with an expiring SAS token, if someone uses Fiddler to retrieve the URL for later use, it will only work until the SAS token expires. We’ll cover SAS tokens in full in a future article.
Blob is the next level of security, and works fine if you don’t want someone to be able to pull everything out of the container, but can download whatever the specific URL targets. You don’t have to use SAS security tokens in the URLs to access the files.
For our demo here, select “Blob – Public read access for Blobs”, and hit Save. Now open Blob Containers on the Connection Group screen, and you will see your container there.
Upload files to Blob Storage
Click on the container name and it will show the container and files on the right-hand side. To upload files, click the Upload button on the top of the window and select File, or simply drag them into the window from your desktop. You will be given a standard Windows file dialog. Pick the files to upload and click Open. After uploading several images, this is what I see in my container:
To upload a folder, select Upload and then Folder. The standard Windows Browse for Folder Dialog is displayed. Select the folder to be uploaded and click OK. You can also open Windows Explorer and drag a folder from there into the container displayed in AMS to upload it. In either case, all files in all subfolders will also be uploaded, so you probably don’t want to select C:\Program Files\ !
After selecting and uploading two folders full of files, I now see this:
To download one or more files, select the files you want to download and click the Download button at the top of the screen. You will get the standard Windows Browse for Folder dialog. Pick the folder you want to download to and click OK. It will download the files to the selected location. You can also drag files and folders out of AMS onto your desktop or into Windows Explorer, and it will download the files and put them where specified. If you select a folder here, it will download the files in the folder and all files in subfolders and maintain the directory structure.
Containers, folders, and file names
The above image displays the Blobs I have uploaded in the same format as a Windows Explorer screen. This is not how the data is actually stored, though. The container is the actual top-level folder; all files must go into a container. The subfolders are really part of the actual Blob name; the application parses the Blob names for the “folder” names and displays them hierarchically like Windows Explorer.
In my example, there isn’t really a folder called NatGeo or one called paper. The “folder” is part of the actual Blob name. Let’s look at NatGeo.
This looks like there is a folder called imagefiles, then a folder called NatGeo under it, and a bunch of image files in that folder. In actuality, you have a folder (container) called imagefiles with a bunch of Blobs in it that have relative paths in the Blob names.
For example, the actual name of Ireland.jpg is NatGeo/Ireland.jpg. The actual name of Tiger.jpg is NatGeo/Tiger.jpg.
You can see the list of files in the way that they are stored rather than the hierarchical representation. This is called a “flat listing”. In AMS, there are two icons on the bottom right-hand corner of the window. These toggle back and forth between hierarchical view (left) and flat listing view (right).
To see the flat listing, click on the icon on the right. (If you want to change the default view, click Tools/Options/Blob, and select “File – No Hierarchy”, then save the changes.)
The flat listing of the files in my container looks like this:
Note that what looked like a folder before is now displayed as part of the Blob name.
These are the actual Blob names. Most applications parse the Blob names and show you the files in the storage account in a hierarchical fashion because that’s what most people relate to, but it’s important to understand the actual structure. When you use the Storage Client Library to retrieve a list of Blobs, the Blob names will include the “folders”, and you can append them to the container to get the actual URL to the file. The URL looks like this:
For one of the files above, the URL would be as follows:
(I put in the spaces after http:// so they won’t render as clickable links in your browser.)
If you want to retrieve all of the files in a “folder”, you wouldn’t use something like Directory.GetFiles in .NET. Instead, you would retrieve all of the files in the container that start with the “folder name”, like “paper/”.
If you download the files from the flat-listing view, the “folders” in the Blob names are ignored. It will download all of the files into the same download directory.
In this article, I showed you how to get started with Blob Storage by setting up a new storage account. While you can use any of several available products, I used the Cerebrata Azure Management Studio to show how to add a container and upload files to it from the local machine. I also explained how to download files to the local machine. Lastly, I explained the real structure of Blob Storage, and showed how to see a list of files in a hierarchical structure or a flat listing. In the next article, I’m going to show you how to use the .NET Storage Client Library to programmatically access and manage Blob Storage.