Windows Azure Blog
Microsoft Cloud Computing Platform
Microsoft Store
  • Home
  • Windows Azure Team Blog
You are here : Windows Azure Blog » OakLeaf Systems » Windows Azure and Cloud Computing Posts for 6/19/2012+

Windows Azure and Cloud Computing Posts for 6/19/2012+

Posted On Tuesday, June 19, 2012 By rss. Under OakLeaf Systems    

A compendium of Windows Azure, Service Bus, EAI & EDI,Access Control, Connect, SQL Azure Database, and other cloud-computing articles. image222

image433

Note: This post is updated daily or more frequently, depending on the availability of new articles in the following sections:

  • Windows Azure Blob, Drive, Table, Queue and Hadoop Services
  • SQL Azure Database, Federations and Reporting
  • Marketplace DataMarket, Social Analytics, Big Data and OData
  • Windows Azure Service Bus, Active Directory, and Workflow
  • Windows Azure VM Role, Virtual Network, Connect, RDP and CDN
  • Live Windows Azure Apps, APIs, Tools and Test Harnesses
  • Visual Studio LightSwitch and Entity Framework v4+
  • Windows Azure Infrastructure and DevOps
  • Windows Azure Platform Appliance (WAPA), Hyper-V and Private/Hybrid Clouds
  • Cloud Security and Governance
  • Cloud Computing Events
  • Other Cloud Computing Platforms and Services

Azure Blob, Drive, Table, Queue and Hadoop Services

David Pallman reported Azure Storage Explorer 5 Preview 1 Now Available with Logging & Monitoring Support in a 6/17/2012 post:

imagePreview 1 of Azure Storage Explorer 5 is now available on CodePlex. This preview reveals the new look of version 5 and contains some new functionality, with more to come in successive updates. Today we’ll review the new logging and monitoring support which is included in Preview 1.

imageIf you’re not familiar with Azure Storage Explorer, it’s a community donation from Neudesic that lets you view and manage Windows Azure storage data. As far as I know, Azure Storage Explorer was the first such tool for Windows Azure: I wrote the first version in January 2009, and we’ve gone all the way up to version 4 which has been out for about a year and a half now. It’s high time we came out with another version, especially given the major update to the platform that took place earlier this month.

Azure Storage Explorer isn’t the most sophisticated tool in it’s class–there are some good robust commercial offerings such as Cloud Storage Studio from Cerebrata, part of Red Gate Software. However, it does an adequate job for most users and it is free, which makes it useful in classroom settings. We also include the source code which makes it a useful example of a storage application for learning. There have been over 50,000 downloads to date.

Goals for Version 5
For version 5, we wanted to first of all match the look of the new HTML-based Windows Azure portal. Secondly, we wanted to support some the new features in Windows Azure storage, including the ability to configure and view both logging and monitoring. Third we wanted to resolve reported issues and adopt good suggestions that users have submitted on CodePlex. We’ve got some of that done in Preview 1 with more to follow.

Logging Support
Windows Azure Storage accounts can now be configured to track logging information. Azure Storage Explorer lets you can configure logging of reads, writes, and deletes for blobs, queues, and tables as you prefer, along with a retention policy for the data.

When you view your storage account blob containers, you’ll notice a container named $logs if loggig is enabled for the account.

The blobs contain records showing storage access requests, in the form of textual data, separated by semicolons.

Monitoring Support
Windows Azure Storage accounts can also be configured to track metrics. Azure Storage Explorer lets you can configure the level of metrics for blobs, queues, and tables (off, minimal, or verbose) as you prefer, along with a retention policy for the data.

When you view your storage account tables, you’ll notice four tables named $MetricsCapacityBlob, $MetricsTransactionsBlob, $MetricsTransactionsTable, $MetricsTransactionsQueue.

The amount of data in the tables depends on whether you’ve enabled minimal or verbose metrics collection.

New User Interface
Like the new Windows Azure portal, Azure Storage Explorer 5 has a look inspired by the styling of Windows 8 Metro apps. In Preview 1, we’ve made some of the more fundamental changes including fonts, layout, and colors in the main window—but there’s more work to be done, including reworking dialogs and how we show the item detail view of blobs, messages, and entities.

I hope you enjoy this first preview of Azure Storage Explorer 5: and stay tuned for more.


Carl Nolan @carl_nolan described a Framework for .Net Hadoop MapReduce Job Submission Binary Output in a 6/15/2012 post:

imageTo end the week I decided to make a minor change to the “Generic based Framework for .Net Hadoop MapReduce Job Submission”.

I have been doing some work on creating a co-occurrence matrix for item recommendations. I was going to map the process to a MapReduce job(s), then came across the issue of how I would output the vector data from the reducer. In the current framework the reducer outputs the key/value data in a string format. This works fine for simple data but for a vector this quickly becomes problematic.

image_thumb11To resolve this I have enabled a parameter called “outputFormat”. The default output will be the usual string format; optionally specified with the parameter value “Text”. Additionally a parameter value of “Binary” is supported:

MSDN.Hadoop.Submission.Console.exe
-input "mobile/data" -output "mobile/querytimes"
-mapper "MSDN.Hadoop.MapReduceFSharp.MobilePhoneQueryMapper, MSDN.Hadoop.MapReduceFSharp"
-reducer "MSDN.Hadoop.MapReduceFSharp.MobilePhoneQueryReducer, MSDN.Hadoop.MapReduceFSharp"
-outputFormat Binary
-file "C:\Projects\Release\MSDN.Hadoop.MapReduceFSharp.dll"

When the output format is specified as binary the reducer value is output as a binary serialized version of the data, represented as a Base64 string. Reading the reduced output one can then easily serialize this object back into a .Net type:

  1. let Deserialize (value:string) =
  2. let bytes = Convert.FromBase64String(value);
  3. use stream = new MemoryStream(bytes)
  4. let formatter = new BinaryFormatter()
  5. formatter.Deserialize(stream)

Hopefully one will find this a lot simpler than performing string manipulations.


Brad Calder posted TechEd 2012: New Windows Azure Storage Features, Improved Manageability, and Lower Prices to the Windows Azure blog on 6/12/2012:

We are very excited to release multiple improvements to Windows Azure Storage. These include price reductions, new manageability features, and new service features for Windows Azure Blobs, Tables, and Queues.

imageJai Haridas will be presenting these features and more on Windows Azure Storage at Tech Ed 2012, so for more details please attend his talk today or view his talk online in a few days.

New Service Features

We’ve released a new version of the REST API, “2012-02-12”. We’ve updated the Java Storage Client library to reflect the new features. We’ve also released source code for a CTP of our .NET storage client library. This version contains the following new features:

  • Shared Access Signatures (Signed URLs) for Tables and Queues – similar to the Shared Access Signature feature previously available for Blobs, this allows account owners to issue URL access to specific resources such as tables, table ranges, queues, blobs and containers while specifying granular sets of permissions. In addition, there are some smaller improvements to Shared Access Signatures for Blobs. Learn more: Introducing Table SAS (Shared Access Signature), Queue SAS and update to Blob SAS
  • Expanded Blob Copy – For Blobs, we now support copying blobs between storage accounts and copy blob (even within accounts) is performed as an asynchronous operation. This is available in the new version, but will only work if the destination storage account was created on or after June 7, 2012. Of course, Copy Blob operations within the same account will continue to work for all accounts. Learn more: Introducing Asynchronous Cross-Account Copy Blob
  • Improved Blob Leasing – Leasing is now available for blob containers, and allows infinite lease duration. In addition, lease durations between 15-60 seconds are also supported. Changing the lease id (in order to rotate the lease-id across your components) is now supported. Learn more: New Blob Lease Features: Infinite Leases, Smaller Lease Times, and More

Improved Manageability

Users of the Windows Azure Management Portal will benefit from the following improvements in managing their storage accounts. These portal improvements are detailed further in New Storage Features on the Windows Azure Portal post.

  • Introducing Locally Redundant Storage – Storage users are now able turn off geo-replication by choosing Locally Redundant Storage (LRS). LRS provides highly durable and vailable storage within a single location (sub region).
  • Choosing Geo Redundant Storage or Locally Redundant Storage – By default storage accounts are configured for Geo Redundant Storage (GRS), meaning that Table and Blob data is replicated both within the primary location and also to a location hundreds of miles away (geo-replication). As detailed in this blog post, using LRS may be preferable in certain scenarios, and is available at a 23-34% discount compared to GRS. The price of GRS remains unchanged. Please note that a one-time bandwidth charge will apply if you choose to re-enable GRS after switching to LRS. You can also learn more about geo-replication in Introducing Geo-replication for Windows Azure Storage.
  • Configuration of Storage Analytics – While our analytics features (metrics and logging) have been available since last summer, configuring them required the user to call the REST API. In the new management portal, users can easily configure these features. To learn more about metrics and logging, see Windows Azure Storage Analytics.
  • Monitoring Storage Metrics – Storage users can now also monitor any desired set of metrics tracked in your account via the management portal.


Pricing

As mentioned above, users can reduce costs by choosing to use Locally Redundant Storage. Furthermore, we are excited to announce that we are reducing the pricing for storage transactions from $0.01 per 10,000 transactions to $0.01 per 100,000, reducing transaction costs by 90%! Learn more: 10x Price Reduction for Windows Azure Storage Transactions.

Summary

We’ve introduced a number of improvements to Windows Azure Storage and we invite you to read about each of them in the referenced blog posts. As always, we welcome your feedback and hope you’ll enjoy these new features!


Brad Calder posted Introducing Locally Redundant Storage for Windows Azure Storage to the Windows Azure blog on 6/8/2012:

imageWe are excited to offer two types of redundant storage for Windows Azure: Locally Redundant Storage and Geo Redundant Storage.

Locally Redundant Storage (LRS) provides highly durable and available storage within a single location (sub region). We maintain an equivalent of 3 copies (replicas) of your data within the primary location as described in our SOSP paper; this ensures that we can recover from common failures (disk, node, rack) without impacting your storage account’s availability and durability. All storage writes are performed synchronously across three replicas in three separate fault domains before success is returned back to the client. If there was a major data center disaster, where part of a data center was lost, we would contact customers about potential data loss for Locally Redundant Storage using the customer’s subscription contact information.

Geo Redundant Storage (GRS) provides our highest level of durability by additionally storing your data in a second location (sub region) within the same region hundreds of miles away from the primary location. All Windows Azure Blob and Table data is geo-replicated, but Queue data is not geo-replicated at this time. With Geo Redundant Storage we maintain 3 copies (replicas) of your data in both the primary location and in the secondary location. This ensures that each data center can recover from common failures on its own and also provides a geo-replicated copy of the data in case of a major disaster. As in LRS, data updates are committed to the primary location before success is returned back to the client. Once this is completed, with GRS these updates are asynchronously geo-replicated to the secondary location. For more information about geo replication, please see Introducing Geo-Replication for Windows Azure.

Geo Redundant Storage is enabled by default for all existing storage accounts in production today. You can choose to disable this by turning off geo-replication in the Windows Azure portal for your accounts. You can also configure your redundant storage option when you create a new account via the Windows Azure Portal.

Pricing Details: The default storage is Geo Redundant Storage, and its current pricing does not change. The current price of GRS is the same as it was before the announced pricing change. With these changes, we are pleased to announce that Locally Redundant Storage is offered at a discounted price (23% to 34% depending upon how much data is stored) relative to the price of GRS. Note if you have turned off geo-replication and choose to enable geo-replication at a later time, this action will incur a one-time bandwidth charge to bootstrap your data from the primary to its secondary location. The amount of bandwidth charged for this bootstrap will be equal to the amount of data in your storage account at the time of bootstrap. The price of the bandwidth for the bootstrap is the egress (outbound data transfer) rates for the region (zone) your storage account is in. After the boostrap is done, there are no additional bandwidth charges to geo-replicate your data from the primary to the secondary. Also, if you use GRS from the start for your storage account, there is no boostrap bandwidth charge. For full details, please review the pricing details.

Some customers may choose Locally Redundant Storage for storage that does not require the additional durability of Geo Redundant Storage and want to benefit from the discounted price. This data typically falls into the categories of (a) non-critical or temporary data (such as logs), or (b) data that can be recreated if it is ever lost from sources stored elsewhere. An example of the latter is encoded media files that could be recreated from the golden bits stored in another Windows Azure Storage account that uses Geo Redundant Storage. In addition, some companies have geographical restrictions about what countries their data can be stored in, and choosing Locally Redundant Storage ensures that the data is only stored in the location chosen for the storage account (details on where data is replicated for Geo Redundant Storage can be found here).


<Return to section navigation list>

SQL Azure Database, Federations and Reporting

Cihan Biyikoglu (@cihangirb) reported Federations: What’s Next? Announcements from TechED 2012 in a 6/14/2012 post:

imageFederations have been available for 6 months in SQL Azure (now called Windows Azure SQL Database) as of today! In this post, I’ll first cover few of the improvements we have made to the SQL Database Federations and talk about a few of the announcement we made at Teched this week on what’s next for the technology.

image

RECENT IMPROVEMENT AVAILABLE TODAY

imageImproved Latency for USE FEDERATION: One of the recent changes that is key to performance of your scale-out systems is the improvements we made to the “USE FEDERATION” statement. With the improvement the latency of USE FEDERATION drastically improved. Internally with USE FEDERATION, we now have 2 types of caching.

  • Connection Pool to the Backend: As applications go back to a hot member, with the pooled connections, they won’t have to reestablish new connections from the GW to the DB nodes in the system and will reuse the existing pooled connection lowering latency for the app.
  • Caching of the Federation Map: USE FEDERATION is used for routing your connections to the given federation key value. This information reside in the root database but it is cached at the gateway layer in the system. This means the root database isn’t ever hit for executing the USE FEDERATION stmnt. This lowers latency and makes USE FEDERATION extremely efficient.

USE FEDERATION connection management extremely efficient for applications by allowing them to point to a single connection string (to a single endpoint name that the server scale out). This solves a huge connection management problem known as connection pool fragmentation. This is a nasty problem and with USE FEDERATION you never have to learn or be aware of the issue.

image

IMPROVEMENT COMING IN THE NEXT FEW MONTHS AND QUERTERS

Support for “Timestamp”, “Rowversion” and “Identity” on Reference Tables: With the upcoming update to SQL Azure, This will enable using IDENTITY property and timestamp data type on reference tables. Federated tables will still be restricted. This means your schema in the federation member can now have the following reference table without issues:

CREATE TABLE zipcodes(
id bigint identity primary key,
modified_date datetime2,
ts timestamp)

Manual Setup with Data Sync Service: Within the next few months we will also have manual setup enabled with Data Sync Services. Data Sync Service can be used for operation like synchronization of reference data between federation members and root or for moving your data in federations to SQL Server or to other Windows Azure SQL dbs.

Federation SWITCH Operation: With the improvements in federations we will make moving federation members in and out for the federation easy as well though an ALTER FEDERATION statement. With the SWITCH IN and OUT operation, it becomes easy to compose and decompose federations. Note: The syntax shown below is just a placeholder and the final version may be different.

image

Database Copy (DBCopy) Support for Federations: We will also enable DBCopy command for federation root or member database. You will be able to point to a federation member of a root with copy database command.

CREATE DATABSAE [Customers-100-200]
AS COPY OF [system-d6c763f4-eda2-427e-af9e-3a8fedd4a16c]

image

IMPROVEMENTS FURTHER OUT ON THE ROADMAP:

Disaster Recovery with Federations: There are a number of disaster recovery improvements we are working on with Windows Azure SQL Database. The general idea is to enable better local and geographic disaster recovery scenarios for your critical data. To simplify DR, we will make point-in-time-restore operation available that enable you to recover from user and admin errors. With Geo-DR, we will enable ability to make your data redundantly available in multiple data center and continuously keep them in sync. You can find more information about them on this session from Sasha: Business Continuity Solutions in Microsoft SQL Azure

Point-in-time-Restore support with Federations: As we make point in time restore available, we will also enable the ability to do point in time restores of federation data in the root or the members. Point in time restore technology allows recovering from user and admin mistakes like dropping tables or deleting a whole bunch of rows accidentally. PITR allows travel back into time, much like database RESTORE command. Provide a data and time for a database and we can restore a snapshot of it for the given time slot..

image

Geo Disaster Recovery: Again as we make the geo disaster recovery features available with Windows Azure SQL Database (SQL Azure). You will be able to make the members and root geo redundant with the capabilities. that is you will be ale to have your data in multiple data centers to protect against data center level failures.

image

Love to hear feedback on all of these features. If you have questions, simply email through the blog or post comments.


<Return to section navigation list>

MarketPlace DataMarket, Social Analytics, Big Data and OData

imageimage_thumb15_thumbNo significant articles today.


<Return to section navigation list>

Windows Azure Service Bus, Caching Active Directory and Workflow

John Shewchuk posted Reimagining Active Directory for the Social Enterprise (Part 2) to the Windows Azure blog on 6/19/2012:

imageMuch of the value of an identity management system derives from the ability to glue together a lot of people, devices, and applications, and to enable applications and administrators to understand and manage all these connections. So in many ways, this value is proportional to the number of “things” that are connected through the directory. The more applications that are connected the more value there is to the people who use the directory. And the more people who connect with the directory the more sense it makes to connect applications to the directory.

Now look at the opportunity that the cloud brings to this equation. While identity management solutions like Active Directory have been pretty successful managing things within organizations, now the cloud is opening the door to connect beyond the organization. The cloud is ushering in explosive growth in the number of applications available to organizations and in the collection of people and devices that can interact with these applications. Ubiquitous connectivity means that people and organizations could potentially use any application anywhere. And any application could potentially be used by anyone—people within the organization, partners, or customers. The “available market” for connecting to an organization’s directory is huge.

This increased scope is a profound change in potential value of the identity management system. But to take advantage of this potential, organizations must be able to function in a world where most of the connections will be to things that are no longer under their direct control. Applications will be running in the cloud and managed by software vendors, cloud platform providers, or other third parties—not within the organization’s data centers. The people won’t be managed by the organization; they will be customers and partners all operating with their own identity providers. And many of the people will be using mobile phones, tablets, and other consumer-oriented devices not issued by the organization.

So if organizations want to take advantage of the emerging opportunity to connect their identity management and applications to this growing number of external applications and people, they will need to make it easy to connect to their directory, enable new ways to create connections, and have new kinds of workflows, policies, and governance that can deal safely with external connections.

Let’s look at some of ways that we have been reimagining Active Directory to help organizations “connect” in this new world.

Windows Azure Active Directory Developer Preview

On June 7, as part of the spring release of Windows Azure, we announced the developer preview for Windows Azure Active Directory. The developer preview adds two major capabilities to the Windows Azure Active Directory service that we described in Part 1 of this post. First, it enables developers to connect to and use information in the directory through an easy-to-use REST interface. Second, it allows developers to connect to the organizational single-sign-on (SSO) capabilities of Windows Azure Active Directory—the same capabilities that are currently used by Microsoft Office 365, Windows Intune, and other Microsoft products.

The developer preview, which will be available soon, builds on capabilities that Windows Azure Active Directory is already providing to customers. These include support for integration with consumer-oriented Internet identity providers such as Google and Facebook, and the ability to support Active Directory in deployments that span the cloud and enterprise through synchronization technology.

Together, the existing and new capabilities mean a developer can easily create applications that offer an experience that is connected with other directory-integrated applications. Users get SSO across third-party and Microsoft applications, and information such as organizational contacts, groups, and roles is shared across the applications. From an administrative perspective, Windows Azure Active Directory provides a foundation to manage the life cycle of identities and policy across applications.

Let’s look at the new capabilities in more detail.

Connecting applications to information in the directory

In the Windows Azure Active Directory developer preview, we added a new way for applications to easily connect to the directory through the use of REST/HTTP interfaces.

An authorized application can operate on information in Windows Azure Active Directory through a URL such as:

https://directory.windows.net/contoso.com/Users(‘Ed@Contoso.com’)

Such a URL provides direct access to objects in the directory. For example, an HTTP GET to this URL will provide the following JSON response (abbreviated for readability):

{ “d”: {
"Manager": { "uri":
"https://directory.windows.net/contoso.com/Users(‘User…’)/Manager" },
"MemberOf": { "uri":
"https://directory.windows.net/contoso.com/Users(‘User…’)/MemberOf" },
"ObjectId": "90ef7131-9d01-4177-b5c6-fa2eb873ef19",
"ObjectReference": "User_90ef7131-9d01-4177-b5c6-fa2eb873ef19",
"ObjectType": "User",
"AccountEnabled": true,
"DisplayName": "Ed Blanton",
"GivenName": "Ed",
"Surname": "Blanton",
"UserPrincipalName": "Ed@contoso.com",
"Mail": "Ed@contoso.com",
"JobTitle": "Vice President",
"Department": "Operations",
"TelephoneNumber": "4258828080",
"Mobile": "2069417891",
"StreetAddress": "One Main Street",
"PhysicalDeliveryOfficeName": "Building 2",
"City": "Redmond",
"State": "WA",
"Country": "US",
"PostalCode": "98007" } }

This kind of Internet-friendly interface makes it easy for developers—building on any platform—to integrate their applications and Windows Azure Active Directory. Using standard HTTP requests, a developer can access any “thing” in the directory (for instance, users), and they can access relationships between things. Continuing with the example above, we can see that it is easy to access a user’s groups by using the URL:

https://directory.windows.net/contoso.com/Users(‘Ed@Contoso.com’)/MemberOf

Sending an HTTP GET to this URL would return the following JSON response (abbreviated for readability) that provides a list of groups for Ed:

…
"results": [
{ "__metadata": { … }
"ObjectId": "30a041bf-e43f-42d6-bec4-a24ce33d5d42",
"ObjectReference": "Group_30a041bf-e43f-42d6-bec4-a24ce33d5d42",
"ObjectType": "Group",
"DisplayName": "Vice Presidents",
"Mail": null
},
{ "__metadata": { … }
"ObjectId": "451758b1-a66e-4d74-b6ce-03c7ec2fee7e",
"ObjectReference": "Group_451758b1-a66e-4d74-b6ce-03c7ec2fee7e",
"ObjectType": "Group",
"DisplayName": "All Users",
"Mail": null
},
…

Because the directory interfaces are built using standard REST semantics, no special protocols or libraries are necessary to use the directory. The approach of using standard REST interfaces to operate over a graph containing entities (nodes) and relationships (arcs) between entities—often referred to as a graph interface—is very common on the Internet. For example, much of the information in Facebook is available in such a manner.

Last week at Microsoft TechEd North America 2012, Edward Wu discussed the directory graph interface. He talked about how, as the applications using the directory become more sophisticated, developers can build on this simple direct URL-based access to take advantage of the sophisticated filtering and metadata operations that are available via Open Data Protocol (OData) version 3.

Ed also noted that, in this early version of the preview, a few critical capabilities are missing. We recognize that developers will need write capabilities, notifications when data in the directory changes, and support for provisioning operations, and we are actively discussing with developers the best ways to provide these capabilities using REST semantics.

Thanks to the new directory graph interface, developers will find it straightforward to write applications that integrate with Windows Azure Active Directory and with other cloud solutions that operate with graph interfaces. For example, BuiltSteady has used the graph interfaces available in both Windows Azure Active Directory and Facebook to improve its application’s ability to intelligently assist users in task management scenarios that span work and personal experiences.

Connecting users to the directory

Above we discussed how the use of web technologies like REST/HTTP is making it much easier to connect applications to the directory. The cloud also provides an opportunity to make it easier to connect new people to the directory. The second major capability we are adding with the developer preview is providing developers the ability to connect their applications to same directory, and information about people, that is used by Office 365 and other Microsoft applications.

To put this new capability in context is useful to see that application developers building cloud-based applications are facing the same many-to-many dilemma that existed before the advent of directories—where each application essentially had to provide a custom application-specific system for identity management—except now the challenge is integrating the application with a variety of identity providers. This situation is depicted in the diagram below.

By using a shared directory, and having the directory directly support these identity providers, developers and administrators can reduce this challenge to a one-to-one integration. This approach is shown here.

Having a shared directory that enables this integration provides many benefits to developers, administrators, and users. If an application integrates with a shared directory just once—for one corporate customer, for example—in most respects no additional work needs to be done to have that integration apply to other organizations that use Windows Azure Active Directory. For an independent software vendor (ISV), this is a big change from the situation where each time a new customer acquires an application a custom integration needs to be done with the customer’s directory. With the addition of Facebook, Google, and the Microsoft account services, that one integration potentially brings a billion or more identities into the mix. The increase in the scope of applicability is profound.

One of the key technologies that Windows Azure Active Directory uses to connect to external identity providers is identity federation. Windows Azure Active Directory supports a range of different kinds of identity providers, including both consumer-oriented and enterprise-oriented identity providers.

To connect to consumer identities, Windows Azure Active Directory already integrates with Facebook, OpenID-based identity providers such as Yahoo! and Google, and the Microsoft account service that Chris Jones and Steven Sinofsky described in their recent post on the Building Windows 8 blog. Chris described “Microsoft account” as “our identity service for individuals who use Microsoft products and services.”

With the Windows Azure Active Directory developer preview, developers can enable SSO with Office 365 and other Microsoft applications. At TechEd last week Stuart Kwan talked about this in A Lap Around Windows Azure Active Directory. As Stuart describes we currently support WS-Federation to enable SSO between the application and the directory. We also see the SAML/P, OAuth 2, and OpenID Connect protocols as a strategic focus and will be increasing support for these protocols. Because integration with applications occurs over standard protocols, this SSO capability is available to any application running on any technology stack. In addition to Stuart’s talk, Vittorio Bertocci provided an in-depth presentation showing how to connect applications to the developer preview using Windows Identity Foundation.

Because Windows Azure Active Directory integrates with both consumer-focused and enterprise-focused identity providers, developers can easily support many new scenarios—such as managing customer or partner access to information—all using the same Active Directory–based approach that traditionally has been used for organizations’ internal identities. For example, in our discussions, developers in the pharmaceutical industry indicate they plan to use Windows Azure Active Directory to enable physicians to use a personal identity (for example, a Facebook or Google identity) when signing up to run clinical trials.

Connecting SaaS applications to the directory

In the diagram shown above, there are multiple directories involved in the relationship between the users and applications. The ability to connect together directories through federation is very powerful. Let’s look at how software developers can use this capability to build multi-tenant software-as-a-service (SaaS) applications.

As an example, let’s imagine that an ISV is building a customer relationship management (CRM) solution. As described above, this ISV can use Windows Azure Active Directory to help build the application. By using Windows Azure Active Directory Access Control, the ISV can create a common consistent view of users, rules, and other information across a range of identity providers including Facebook, Google, and Microsoft, as well as corporate customers running their own directories

The ISV’s customers can use a number of different directory solutions. Their directory could be Windows Server Active Directory with federation provided by Active Directory Federation Services. Or it could be a directory using PingFederate from Ping Identity to connect to the application.

Other customers might choose to use the Windows Azure Active Directory service as way to manage their users or to connect it to existing directory deployments. With the developer preview, an organization that is using Windows Azure Active Directory as part of it’s identity management solution can now grant permission to ISV’s application. This enables both single sign-on and the REST directory access described above.

In this case, notice that Windows Azure Active Directory can be used simultaneous by both the ISV—to help build the applications—and by the customers—to provide a cloud-based identity management solution for their organization.

In his TechEd presentation Vittorio Bertocci described how ISVs can build SaaS applications that can connect to multiple customers each using Windows Azure Active Directory. Specifically he showed how to use Window Identity Foundation extensibility to connect applications dynamically to the customers’ Windows Azure Active Directory deployment.

The opportunity for developers

The rapidly growing number of organizations using Windows Azure Active Directory and the large number of associated users who are available through the directory create opportunities for developers.

Integrating an application with Windows Azure Active Directory connects the application to the growing collection of applications that are already connected and to the large and growing collection of potential users. For users, this integration results in common experiences like single sign-on and shared context without the need to set up and maintain these connections. For the administrators of an organization, this translates to lower operating costs; consistent management of applications, including using the directory to define roles or disable an identity; and reduced risks. For software developers, these advantages make an application more relevant and valuable to users and administrators.

The combination of Windows Azure Active Directory enhancements to access the directory through new Internet-friendly protocols and integrate with both enterprise- and consumer-oriented identities will empower developers to use Windows Azure Active Directory to address new scenarios that go well beyond the “behind the firewall” role that identity management has historically played. We are excited to see how developers will take these new mechanisms and build new experiences and capabilities for organizations and users.

Because Windows Azure Active Directory supports industry-standard protocols, federation with it is no different from identity federation directly with an on-premises directory. Because applications interact with Windows Azure Active Directory through standard protocols, the applications can be cross-platform. Applications can run on the Windows operating system, Amazon Web Services, Google App Engine, iOS, and Android, and developers have their choice of many programming languages and runtimes.

In discussions with us, developers say integrating and keeping their applications federated with many different customer directories is one of their biggest challenges. Windows Azure Active Directory helps transform the application-to-directory federation challenge from a many-to-many problem for both customers and developers to a one-to-one integration. Windows Azure Active Directory provides a consistent, high-availability point of integration that reduces or eliminates many of the typical loose ends and anomalies that comes from one-off customer integrations. This results in substantially less work and testing for everyone, and much higher rates of successful deployment.

Where we are today and what’s next

In Part 1 of this post, we focused on how Microsoft is reimagining Active Directory as a cloud service. We discussed how the application of cloud architecture and economics is making it possible to bring the power of identity management to organizations of any size, with great ease of use, low cost, and high availability. As identity management becomes more available as a service, many more organizations will be able to take advantage of it.

Here in Part 2 of this post, we looked at how we are reimagining Active Directory and enabling new ways to connect to people, devices, and applications. The Windows Azure Active Directory developer preview offers new capabilities that enable integration with next-generation applications, mobile devices, consumer identities, and social networks.

The combination of widespread adoption and improved connectivity that identity management as a service offers will enable developers and organizations to tackle existing and emerging challenges. It will be an exciting time for the identity industry. We look forward to playing our part and working with others in shaping what this new world looks like.

Please look for Alex Simon’s blog post announcing availability of the developer preview soon.


Avkash Chauhan (@avkashchauhan) described Difference between Windows Azure Cache(Preview) and Windows Azure Shared Cache in a 6/18/2012 post:

imageWith the release of Windows Azure Caching (Preview) and there is already Windows Azure Shared Cache there are lots of question what these cache are and how to use them. I decide to write this article to provide the differences between them so users can understand it properly.

imageTo start, the main difference between Windows Azure Cache(Preview) and Windows Azure Share Cache is that Windows Azure Caching (Preview) is configured in within your role specific Virtual Machines. You have ability to either share a % of virtual machine resources from your existing Virtual Machine (co-located model) to use a Cache or just dedicate a full Virtual Machine as dedicated cache (Dedicated model). While Windows Azure Shared Cache is a dedicated cloud cache service configured for your application in cloud. From there you can actually understand how the implementation would differ in between two.

Windows Azure Cache (Dedicated and Co-located) Preview:

Windows Azure Cache (Preview) has the following advantages:

  • Pay no premium for caching. You pay only for the compute resources that host the cache.
  • Eliminates cache quotas and throttling.
  • Offers greater control and isolation.
  • Improved performance.
  • Automatically sizes caches when roles are scaled in or out. Effectively scales the memory that is available for caching up or down when role instances are added or removed.
  • Provides full-fidelity development time debugging.
  • Supports the memcache protocol.

This image explains Windows Azure Dedicated Cache (Preview):

This image explains Windows Azure Co-Located Cache (Preview):

How to Start Sample: https://www.windowsazure.com/en-us/develop/net/how-to-guides/cache/
FAQ: http://msdn.microsoft.com/en-us/library/windowsazure/hh914142
When you are using Windows Azure Cache (Dedicated and Co-located Cache) you must use the references located below (This is released as preview):
C:\Program Files\Microsoft SDKs\Windows Azure\.NET SDK\2012-06\ref\CachingPreview

Windows Azure Shared Cache:

Windows Azure Shared Caching enables you to easily provision a cache in the cloud, to be used from any applications or services that could benefit from caching. Here is an overview of Overview of Windows Azure Shared Caching: http://msdn.microsoft.com/en-us/library/windowsazure/hh697519

This image explains Windows Azure Shared Cache:

How to get started SAmple: http://msdn.microsoft.com/en-us/library/windowsazure/gg278346.aspx
When you are using Windows Azure Shared Cache then you should use references located below:
C:\Program Files\Microsoft SDKs\Windows Azure\.NET SDK\2012-06\ref\

FAQ: http://msdn.microsoft.com/en-us/library/windowsazure/hh697522


<Return to section navigation list>

Windows Azure VM Role, Virtual Network, Connect, RDP and CDN

My (@rogerjenn) Configuring a Windows Azure Virtual Network with a Cisco ASA 5505-BUN-K9 Adaptive Security Appliance preview, updated 6/19/2012, begins:

Contents: (This is a preview.)

  • imagePrerequisites
    • Supported VPN Appliances and Routers
    • The Cisco ASA 5505-BUN-K9 Appliance Used for this Tutorial
    • Upgrading to ASA v8.3(1) Software on Cisco ASA 5505s with Earlier Versions
    • Backing Up Configurations to local *.zip File
    • Resetting the ASA 5505 to Factory Default Configuration and Changing Its IP Address with the Command Line Interface (CLI)
    • Restoring a Configuration Backup
    • Troubleshooting ASDM Connection Problems on Multi-Homed PCs
  • Installing and Configuring a Cross-Premises Windows Azure Virtual Network

imageThis tutorial is intended as a guide for setting up a Windows Azure Virtual Network (WAVN) to support single sign-on of Remote Desktop Services (formerly Terminal Services) clients by Active Directory domain users and admins with the new Windows Azure Active Directory (WAzAD) feature. The tutorial doesn’t require prior network administration experience. The Windows Azure Team announced WAzAD preview on June 7, 2012 at the Meet Windows Azure conference in San Francisco.

imageThe following slide from Mark Russinovich’s Windows Azure Virtual Machines and Virtual Networks session on day 1 of TechEd North America 2012 describes the Virtual Network topology of a Fabrikam Events Manager sample hybrid cloud application with a Site-to-Site (S2S, cross-premises) VPN Tunnel created by a hardware VPN device:

image

Windows Azure Active Directory (WAzAD) provides authentication of users and administrators of the Windows Azure Web roles in the FrontEnd Subnet (10.3.1/0/24).

You can learn more about Windows WAzAD from the following resources:

  • John Shewchuk, Reimagining Active Directory for the Social Enterprise (Part 1)
  • John Shewchuk, Reimagining Active Directory for the Social Enterprise (Part 2)
  • Vittorio Bertocci and Stuart Kwan, A Lap Around Windows Azure Active Directory (video)

This post is based in part on the Windows Azure Team’s Create a Virtual Network for Cross-Premises Connectivity tutorial and uses some identical subnet names and addresses for clarity.

Prerequisites
  • A trial or paid Windows Azure subscription with Preview features enabled.
  • Remote Desktop Services configured in a Windows Server 2008 R2 instance of Small or larger size running as a Windows Azure Virtual Machine, as described in my earlier Installing Remote Desktop Services on a Windows Azure Virtual Machine running Windows Server 2012 RC post.
  • A local Windows Server 2003 R2 SP1 or later domain controller with Active Directory installed and running.
  • A hardware VPN appliance or router, preferably from the list of supported devices below, with the latest software upgrade, which is v8.3 for Cisco ASA 5500 series.
  • Java runtime v6.0 or higher installed (Java v7 Update 5 is used for this example)
  • The VPN device connected and configured at least to the point that tunnels can be configured by you or a network administrator
  • A Remote Desktop Services Per User or Per Device Client Access License (CAL) for each user of or device connected to the services.
Supported VPN Appliances and Routers

Creating a WAVN between a WAVM and an on-premises domain with an Active Directory domain controller requires a hardware VPN appliance. When this tutorial was written, only the following Cisco and Juniper VPN routers and gateways were supported with installation script templates that were written and tested by the Windows Azure team:

Cisco Systems OS Family Juniper Networks OS Family
ASA 5500 Series ASA 8.3 SRX 210 Router JunOS 11.2r6 or JunOS 10.4r9
ASR 1001 IOS 15.2 SRX 1400 Router JunOS 11.2r6 or JunOS 10.4r9
ASR 1004 IOS 15.2 J Series Routers JunOS 11.2r6 or JunOS 10.4r9
ASR 1006 IOS 12.2 ISG Series Routers ScreenOS 6.3r9 or ScreenOS 6.2r13
ISR 2921 IOS 15.0 [SSG Series Routers]  
ISR 3925 IOS 15.2    
ISR 3945 E IOS 15.0    

Note: The current version of the Supported VPN devices list doesn’t include Juniper’s SSG series routers.

The Cisco ASA 5505-BUN-K9 Appliance Used for this Tutorial

A Cisco ASA5505-BUN-K9 10-user Adaptive Security Appliance provides the hardware VPN device required by WAzAD in this tutorial. This device includes 10 bundled IPSec and two bundled SSL user sessions, and supports up to a total of 25 sessions. Cisco ASA 5500-series devices have been very popular in SOHO networks for several years. This device is available from Amazon.com affiliates and other network hardware sellers for about US$325.

image

I used Richard A. Deal’s Cisco ASA Configuration book (ISBN-10: 0071622691, McGraw-Hill, 2009, 718 pp.) as a reference when setting up OakLeaf’s ASA5505, which was purchased to test WAVN and later provide a VPN for the OakLeaf domain. Deal’s The Complete Cisco VPN Configuration Guide (ISBN-10: 1587052040, Cisco Press, 2005) is a bit pricey at US$67.40 from Amazon.com.

Note: Michael Dale posted a comparative Cisco ASA 5505 vs Juniper SSG 5 review in early 2008. Although the review is somewhat dated (Cisco software v7.2 and Juniper ScreenOS, replaced by JunOS), it is detailed and appears unbiased. The Juniper SSG5-SB 128MB Security Services Gateway is available from Amazon.com affiliates and other network hardware sellers for about US$490.

Mark Russinovich briefly showed the Home page of the Adaptive Security Device Manager (ADSM) Web UI for the ASA 5505 at 00:55:33 in his Windows Azure Virtual Machines and Virtual Networks session video archive:

image

Here’s the ASDM Java app’s configuration pane showing interfaces to seven subnets defined in the Virtual Networking portal:

image

Virtual Networking content begins at 00:48:09 into Mark’s session. …

The post continues with remainder of the topics from the table of contents above.


Venkat Gattamneni and Jason Chen posted Infrastructure as a Service Series: An Overview of Windows Azure Virtual Network to the Windows Azure blog on 6/19/2012:

imageWe announced the preview of Windows Azure Virtual Network on June 6. In this blog post, we wanted to take a deeper look at this brand new feature and explain it further in the context of other cross-premises networking features that already exist in Windows Azure.

Prior to the release of Windows Azure Virtual Network, you had a range of options to connect your on-premises IT environment with the public cloud. You were able to use SQL Data Sync for synchronizing databases, ServiceBus for application-level connectivity, and Windows Azure Connect for securely connecting machines at an IP level. Now, with Windows Azure Virtual Network, we’re enhancing our cross-premises connectivity stack further by allowing you to set up site-to-site connectivity, much like you’d set up a branch office network and connect your corporate network to it using VPN gateways.

Here’s a visual representation of our cross-premises connectivity stack with the release of the Virtual Network service:

With this new capability, you can now create a logically isolated private environment in Windows Azure, and connect it to your corporate datacenter using a secure VPN tunnel. Once set up, your isolated Windows Azure environment can function as a logical extension of your corporate network.

Create a Virtual Private Network in Windows Azure

You can create a private network (called a Virtual Network, or VNET for short) in the Windows Azure environment within which you’re able to define private IP address ranges. Within a VNET, you also have the choice of creating logical subnets and specifying a DNS that virtual machines will use. When virtual machines or role instances are launched inside a VNET or a subnet, they’re automatically assigned the IP address from the range you specify. A thing to note here is that VNETs are logically isolated from each other, so your private IP addresses do not collide with another customer’s private IP addresses even though they might be the same.

Creating a tunnel to your Virtual Network

Once you’ve created a VNET, you have the option to connect it securely to your on-premises network through a standard IPSEC VPN tunnel. If you choose to do this, a VPN gateway will automatically be provisioned for you in Windows Azure. Then, all you have to do is to configure your on-premises VPN gateway to finish setting up the tunnel.

With the functionality that Windows Azure Virtual Network provides, we think you’ll be able to address a variety of hybrid cloud scenarios like building ‘virtual’ extensions to your datacenter, or running some parts of your application in the Cloud and others in your local datacenter. For example, you can now domain join virtual machines running in Windows Azure to an on-premises AD, and you can run intranet-facing Sharepoint in Windows Azure.

“Great, so both Virtual Network and Connect allow me to create secure cross-premises IP level connections. What’s the difference?” you might ask. Here’s a quick video illustrating the difference between these two services:

Setting up and Managing a Virtual Network

You can create a VNET in Windows Azure through the management portal in a fairly simple manner. The following video explains how to setup a VNET, assign IP address ranges and then create a connection with your on-premises network.

We hope you’ll like the Windows Azure Virtual Network capability. Click here for more information and tutorials on creating and managing virtual networks in Windows Azure


<Return to section navigation list>

Live Windows Azure Apps, APIs, Tools and Test Harnesses

The Windows Azure Team released the Windows Azure Training Kit – June 2012 updated for the new “Spring Wave” feature previews on 6/18/2012:

imageThe Windows Azure Training Kit includes a comprehensive set of technical content including hands-on labs and presentations that are designed to help you learn how to use the latest Windows Azure features and services.

June 2012 Update

The June 2012 update of the Windows Azure Training Kit includes over 42 hands-on labs and 18 presentations. Some of the updates in this version include:

  • 12 new hands-on labs for Windows Azure Virtual Machines
  • 11 new hands-on labs for Windows Azure Web Sites
  • 2 new hands-on labs demonstrating Windows Azure with Windows 8 Metro-style applications
  • Several new hands-on labs for Node.js and PHP using Mac OS X
  • Updated content for the latest Windows Azure SDKs, tools, and new Windows Azure Management Portal
  • New and updated presentations designed to support individual sessions to a full 3 day training workshops

<Return to section navigation list>

Visual Studio LightSwitch and Entity Framework 4.1+

Beth Massi (@bethmassi) reported a Channel 9 Interview: Early Look at the Visual Studio LightSwitch HTML Client in a 6/12/2012 post:

imageI can’t tell you how excited I am that Jason Zander announced the LightSwitch HTML Client at TechEd yesterday! I have to admit it’s been really hard keeping my mouth shut about this :-) . Having the option to build companion clients for tablet devices without having to know HTML5 or JavaScript is a big deal. If you know data modeling, you can use LightSwitch in Visual Studio 2012 to build out OData services that can be used by a variety of clients. The next obvious step is to provide a development experience for building these clients.

image_thumb1So check it out, I posted a video on Channel 9 that gives us an early look at where the team is headed. In this interview, Joe Binder (Program Manager on the LightSwitch Team) walks through the design experience and shows us how easy it is to customize the styling and controls.

Channel 9 Interview: Early Look at the Visual Studio LightSwitch HTML Client

I recommend watching one of the high quality recordings here so you can see the demo better:

  • High Quality MP4 (iPad, PC)
  • High Quality WMV (PC, Xbox, MCE)

For more information, please see the team blog post announcing the LightSwitch HTML Client. You can also discuss with the team on the LightSwitch HTML Client Forum.

And stay tuned for more news as it unfolds on the LightSwitch Developer Center.

Check out Joe Binder’s Creating Screens with the LightSwitch HTML Client (Joe Binder) of 6/19/2012 and the LightSwitch Team’s Announcing the LightSwitch HTML Client! of 6/11/2012.


Beth Massi (@bethmassi) described LightSwitch IIS Deployment Enhancements in Visual Studio 2012 – Updated

imageI just updated my LightSwitch IIS Deployment Enhancements in Visual Studio 2012 post to coordinate with the Visual Studio 2012 Release Candidate and the Windows Azure SDK for .NET.

image_thumb1LightSwitch has a new publishing wizard in the new Azure SDK which makes it much easier to publish to Azure Cloud Services and also supports the new Azure Web Sites platform. For more info on Azure publishing see Brian’s post on the LightSwitch Team blog: Publishing LightSwitch Apps to Azure with Visual Studio 2012

Enjoy!


Return to section navigation list>

Windows Azure Infrastructure and DevOps

David Linthicum (@David Linthicum) asserted “Cloud deniers are becoming cloud supporters and conveniently forgetting their early rejection of cloud technology” in a deck for his Born-again cloud advocates finally see the light post to InfoWorld’s Cloud Computing blog:

imageLet me tell you, back in 2004, selling cloud computing in San Francisco was no easy task. Doors were slammed in my face more often than not, and I heard over and over again how dumb it was to assume that anyone would ever place their core data and business processes in Internet-linked systems.

imageThe fact is that many people in tech function like a kid’s soccer team: There is no core strategy. Instead, they chase the ball from place to place, hoping to get a whack at it.

In 2004, cloud computing (at least, IaaS and PaaS) was not well understood. There was little hype behind it and no soccer ball heading in my direction. Remember, it’s not a good idea until many others think it’s a good idea, and disappointingly, we have vastly more followers than leaders.

Fast-forward eight years, and many of those who slammed doors in my face, did not return phone calls, and outright told me cloud computing will never work have suddenly come around to the cloud way of thinking. Not only do they support cloud, they tell me they always did. I refer to these folks as "born-again cloud." They typically work for large technology providers, Global 2000 enterprises, or large consulting organizations. They speak in buzzwords. They survived the last set of layoffs. They’ve purchased at least five cloud computing books, mine included, that sit on the bookshelf behind their desk. They waited for the iPad 2.

Don’t get me wrong: It’s OK to be "born-again cloud." Many people accept technology evolutions late. Take, for example, the rise of the Web — entire corporate cultures needed to change before the Web was accepted in most enterprises. As long as you eventually move in the right direction, things work out. Moreover, you can adopt technology too early. I’d argue that in 2004 some maturation was still needed, albeit the cloud concept was sound.

However, I can’t help but wish that we keep an open mind about the next technology evolution when it begins and get religion earlier. We shouldn’t wait until everybody else does it. Oh, well — I’ll start building up a thick skin now.


David Pallman offered 10 Reasons to Take a Fresh Look at Windows Azure in a 6/15/2012 post:

Windows Azure has just been re-launched, and the new platform is extremely compelling. Here are 10 reasons to take a fresh look at the new Windows Azure:

1. Managed Web Sites
Window Azure has a special mode of use just for 2-tier web sites, called Windows Azure Web Sites. Both Microsoft’s SQL Database or MySQL databases can be used. WAWS web sites use a shared VM pool, and provisioning happens in just seconds, not minutes. Customers can promote to reserved VMs if they choose. WAWS web sites are fast, easy, and superbly managed.

2. Web Framework Support
Window Azure Web Sites also support common open source frameworks, including DotNetNuke, Drupal, Joomla, Orchard, and WordPress. Web developers can quickly provision web sites with their desired framework and get right to work.

3. Web Deployment Freedom
Web developers work in different ways, and there’s no single way to deploy. Windows Azure Web Sites support several popular methods of deployment, including Web Deploy, FTP, Git, and TFS. In addition, the deployment is conveniently always to a single "server", even when running multiple server VMs in the cloud. WAWS takes care of distributing new and updated deployment files to the individual VM instances.

4. IaaS Done Right
Window Azure now has Infrastructure-as-a-Service (IaaS) support, nicely complementing the Platform-as-a-Service (Paas) support that has been traditionally offered. And it’s not a "token" implementation: the IaaS support is rich and well-done. IaaS Virtual Machines are based on VHDs which are fully portable between cloud and on-prem. They are also persistent, making Windows Azure viable at last for single-server solutions and for running server products such as AD, SQL Server, or SharePoint Server.

5. Non-Microsoft OSs & Databases
Traditionally, Microsoft has offered platform services based on Windows Server and SQL Server. With the new platform there is now support for Linux virtual machines and MySQL databases. This means a broader range of software can now run in the Microsoft cloud.

6. VM Image Gallery
When creating Virtual Machines, you can select from a gallery of pre-configured images. For example, you can install Windows Server 2008 plain, or with SQL Server also installed. Better yet, you can add your own VM images to the gallery.

7. VM Composition
Window Azure virtual machines can be easily be provisioned, configured, and managed in the Windows Azure portal. Once you connect to and set up a VM, you can capture its disk to create a reusable image that is added to your VM image gallery.

8. Cloud Services
Window Azure may have some new modes of use for web sites and IaaS, but its traditional Platform-as-a-Service (PaaS) mode–now known as Cloud Services–not only remains, it’s getting new and updated services.
An expanded virtual networking capability can interface with enterprise VPN appliances to enable hybrid cloud scenarios where segments of your local network are joined to your cloud assets.

Windows Azure Media Services, a new service in limited preview right now, will allow uploading, transcoding, and delivery of media content such as video.
A new identity service is coming that will extend the identity federation we currently have in the Access Control Service with expanded capabilities.

9. The New Portal
Window Azure gets a huge usability boost in a cutting-edge new management portal. HTML-based, the new portal can also be used on mobile devices such as iPads and Windows 8 slates. The new portal also provides a view of built-in metrics for WAWS, Cloud Services, and Virtual Machines.

10. New Data Centers
Two new data centers have been added recently (West US and East US), bringing the number of Windows Azure data centers to 8 (four in the US, two in Europe, and two in Asia). In addition, there is supporting worldwide infrastructure including a 24-node edge cache CDN network.

There you have it — Windows Azure has it all. If you’ve formed an opinion about the Microsoft cloud in the past, it’s time to take another look: you’ll like what you see.

For a detailed look at what’s new in Windows Azure, see Reintroducing Windows Azure.


Henry Jerez posted Infrastructure as a Service Series: Support for Linux Virtual Machines on Windows Azure on 6/14/2012:

imageGreetings. It is my pleasure to write about the work that engineers in Windows Azure have been doing to enable our partners and customers to run and manage Linux workloads on top of Windows Azure.

As you might have heard by now, the new Virtual Machine capability currently under Preview allows you to run Windows Server as well as Linux virtual machines on Windows Azure.

Our goal in building support for Linux is to provide a first-class experience to our customers. We know that we cannot do this alone so we are partnering with terrific companies already delivering great products for enabling Linux in the cloud.

With the Preview version of this capability, this is a journey that has just begun for us. With this release you can see the early results of our work in this area. We understand that there is a lot of work still left to do and look forward to learning from your feedback in the Linux Virtual Machines forum.

Our Partners and Their Solutions

I encourage you to visit our partner solutions page where you can find more information about the capabilities provided by our partners.

These solutions range from advanced creation and management of images and virtual machines to complete stack images with web and developer stacks ready to deploy.

Learn More

You can learn more about how to get started with Linux on Windows Azure at the following links:

  • Managing Linux Virtual Machines
  • Common Tasks
  • How-To’s
  • Other Resources

Simon Munro (@simonmunro) explained The significance of Linux VMs on Windows Azure in a 6/7/2012 post:

imageOne of the most significant, highly anticipated, and worst kept secrets of the Windows Azure spring release is the inclusion of persistent VMs, with the notable addition of support for Linux on those VMs.

The significance of the feature is not that high architecturally — after all, Windows Azure applications that were specifically architected for Windows Azure run well already. The aspects that I find more significant are,

  • imageClosing the gap to AWS — It is has always been difficult to compare Windows Azure and AWS because of the IaaS bias of AWS versus Windows Azure. With the addition of persistent VMs, the two platforms can be better compared and better choices made.
  • Base understanding — Windows Azure is widely misunderstood, largely due to its PaaS nature. In the face of this misunderstanding, AWS as the de-facto choice, and the more common understanding of IaaS, has been easy. The addition of persistent VMs allows decision makers to go with something that is more familiar before branching out into some of the specific Windows Azure features (as customers moving to AWS tend to do).
  • Not just Windows — The inclusion of Linux is a big deal for Microsoft. Regardless of Microsoft’s own reasons, having first-class support of Linux breaks the perception that Windows Azure is Windows and .NET only. Support of Java, Node.js, Ruby and now Python under Windows Azure now has more credibility with the addition of Linux to the stable.
  • Architectural choices — I’ve never been a fan of running everything under the Windows Azure ‘role’ model. Running something like MongoDB or Solr in this way just seems wrong. The addition of persistent VMs now gives architects the chance to deploy technologies that work well under Linux, where there is better support and understanding of how they run. Building a solution with MongoDB running on Linux on Windows Azure is architecturally significant and very useful.
  • Enterprise comfort — Enterprises with legacy applications have struggled to make the move to Windows Azure and they are probably the largest drivers of the inclusion of persistent VMs (the ‘listening to our customers’ part of Microsoft). Regardless if it is a good idea or not to run SSIS or old-school SharePoint on a cloud platform, it is something that lots of people want to do. Enterprise customers can now run whatever they like, including Linux-based parts of their solutions.
  • Bring your stack — When the announcement of the spring release was made yesterday I was most interested to see the flurry of accompanying press releases. I saw news from RightScale, Cloudant, Opscode and 10Gen. These, and similar, organisations are the backbone of the cloud community and their support of Windows Azure (however extensive it may be) greatly increases the reach of Windows Azure into areas of the cloud playground where the cool kids are hanging out.

It will be interesting to see, over the coming weeks, how the markets and the clouderati respond to these announcements. It was a move that Microsoft had to make and they need to get the right messages about the changes out to the market in order to gain better traction of Windows Azure.


<Return to section navigation list>

Windows Azure Platform Appliance (WAPA), Hyper-V and Private/Hybrid Clouds

Bob Kelly announced Now Available: New Services and Enhancements to Windows Azure in a 6/7/2012 post to the Windows Azure blog:

imageToday, we are ushering in the new era of Hybrid Cloud, bringing together the best of on-premises and cloud computing. Virtual Machines, Virtual Network, and Web Sites are now available as new preview services for Windows Azure. SQL Reporting is now Generally Available to customers, a locally redundant storage option has been added to provide additional customer choice, and several enhancements to existing Windows Azure services are now live. These updates help customers build and bring their applications to the cloud in their own unique way.

Additionally, a number of pricing and metering updates have been made to increase overall value. These changes include graduated pricing for Network, CDN and Storage; preview pricing for Windows Server and Non-Windows VMs; and a 90% reduction in Storage and CDN transaction prices.

Full details are below, and please check back often for technical deep dive posts and other helpful information on these enhancements.

Here are the details:

Enhancements to Existing Services:

  • SQL Reporting—Now Generally Available with a fully backed SLA, SQL Reporting enables you to publish reports to the cloud or embed reports directly within on-premises applications that can be accessed via browser, mobile device or PC. Additional details available here.
  • Caching—“Caching Preview” offers a new tenancy model, new features and performance improvements. The Preview allows Caching to be deployed to web roles and to be co-located with other application components. Additionally, a dedicated cache tier can be created for one or more applications from multiple worker roles providing almost unlimited cache sizes and scale. The Preview adds many new features such as notifications, tags, regions and high availability to ensure that your cached data is resilient. Additional details available here.
  • Storage—As an additional choice to our existing Geo-Redundant Storage option, Locally Redundant Storage is now available for customers who don’t require geo-replication and are looking for reduced storage costs. Other storage updates include Blob Leases and Cross Storage Account Copy for blob storage, and Shared Access Signature (SAS) for Tables and Queues (with continued availability for blob storage). Additional details are available here.
  • Compliance— The SSAE 16 (SOC 1 Type 2) audit report is now available for Windows Azure core services. For more information, please visit the Windows Azure Trust Center.

New Services:

  • Windows Azure Virtual Machines— Virtual Machines give you application mobility, allowing you to move your virtual hard disks (VHDs) back and forth between on-premises and the cloud. Migrate existing workloads such as Microsoft SQL Server or Microsoft SharePoint to the cloud, bring your own customized Windows Server or Linux images, or select from a gallery. Compatible operating systems and images available in the online gallery include;
    • Windows Server
      • Windows Server 2008 R2
      • Windows Server 2008 R2 with SQL Server 2012 Eval
      • Windows Server 2012 RC
    • Linux:
      • OpenSUSE 12.1
      • CentOS 6.2
      • Ubuntu 12.04
      • SUSE Linux Enterprise Server 11 SP2

Virtual Machines can be accessed in the new Windows Azure Management Preview Portal or through Powershell, using the new Windows Azure SDK (June 2012). Command line tools are also available for development on a Mac or Linux.

To start using Virtual Machines, request access on the ‘Preview Features’ under the ‘account’ tab, after you log into your Windows Azure account. Don’t have an account? Sign-up for a free trial here.

  • Windows Azure Virtual Network— This new service provides you a simple way to create a private environment (called a virtual network or VNET for short) in Windows Azure and optionally connect it to your on-premises network using a VPN gateway. Within the virtual network you create, you have control over the network topology – for example, you can configure IP address ranges for the Virtual Machines or even specify your own DNS. For creating a secure connection with your corporate VPN gateway, the industry-standard IPSEC protocol is used. With the Windows Azure Virtual Network service, you can
    • Extend your on-premises datacenter by building “virtual” extensions in the public cloud
    • Provide a networking on-ramp for migrating existing (including legacy) apps and services to Windows Azure
    • Run “hybrid” apps that span the cloud and on-premises networks

To start using Virtual Networks, request access on the ‘Preview Features’ page under the ‘account’ tab, after you log into your Windows Azure account. Don’t have an account? Sign-up for a free trial here.

  • Windows Azure Web Sites—Build web sites and applications with this highly elastic solution that allows you to easily deploy with a few clicks.
    • Build modern applications using.NET, Node.js, or PHP, and connect with SQL Database (formerly known as SQL Azure) or MySQL (offered as a service on Windows Azure by ClearDB).
    • Easily deploy using built-in integration with Git and Team Foundation Service, along with FTP. Integration with Microsoft WebMatrix also allows you to easily pull down your web sites locally to make changes and then easily deploy back to Windows Azure.
    • Deploy popular open source web apps like WordPress, Joomla!, DotNetNuke, Umbraco, and Drupal to the cloud.

To start using Web Sites, request access on the ‘Preview Features’ page under the ‘account’ tab, after you log into your Windows Azure account. Don’t have an account? Sign-up for a free trial here.

Improved Tooling and Language Support:

  • Windows Azure Management Portal (Preview)– The Preview Portal features rich scenario-based user interfaces, real-time monitoring charts, diagnostics data, notifications and alerts to manage the health of your application, enabling easy deployment, configuration, monitoring and troubleshooting of your applications. The Preview Portal supports the following services: Cloud Services, Virtual Machines (Preview), Web Sites (Preview), Virtual Network (Preview), SQL Database and Storage.
  • New tools, language support, and SDK—Windows Azure SDK June 2012 includes new developer capabilities for writing code against the latest service improvements with updated support for Java, PHP, and .NET, and the addition of Python as a supported language on Windows Azure. Additionally, the SDK now provides 100% command line support for both Windows and Mac.
    • Windows Azure SDK for .NET
      • Added support for IISExpress in the Emulator
      • Added support for LocalDB in the Emulator
      • Added Dedicated Caching (Preview) functionality
        • Added Visual Studio support for Dedicated Caching (Preview) roles
      • Updated client libraries for Storage, Service Bus & Shared Caching
      • Added Visual Studio support for Service Bus
    • Windows Azure SDK for Java
      • Added service wrapper client libraries for Tables (Storage)
    • Windows Azure SDK for Node.js
      • Added Service Runtime client libraries
    • Windows Azure SDK for PHP
      • Added service wrapper client libraries for the following services
        • Storage (Tables, Queues & Blobs)
        • Service Bus
        • Service Runtime
      • Added packaging and tooling support for Windows Azure PowerShell cmdlets
    • Windows Azure SDK for Python
      • PowerShell cmdlets and Client Libraries (win, linux, mac).
      • Support for Django, the most popular Python web framework
      • Support for Windows Azure and Django in PTVS http://pytools.codeplex.com
      • Support for IPython (from win, linux, mac clients) to Virtual Machines (Linux or Windows)

Pricing Updates:

  • Preview Pricing for VMs. For the preview period, Windows Server and Linux VMs cost per hour is .013 for xSmall, .08 for Small, .16 for Medium, .32 for Large and .64 for XLarge.
  • Preview Pricing for SQL Server VMs. For the preview period, SQL Server 2012 Eval Edition is included at no additional charge. At GA, Images with SQL Server 2012 Web and Standard Editions will be available for an additional fee.
  • Graduated Pricing for Storage, CDN, and Network Egress. Customers using these services will automatically see their costs go down as their usage volume increases There is no need to do anything special, if you are a Pay As You Go customer, you will receive this benefit automatically.
  • Locally Redundant Storage. As mentioned above, we are adding a new option – Locally Redundant Storage. Locally Redundant Storage offers up to a 33% discount over our Geo-Redundant Storage option with the same graduated pricing tiers.
  • Price reduction for Storage and CDN transactions. Storage and CDN transactions have been reduced by 90% (from .01 for 10,000 transactions to .01 for 100,000 transactions).

For additional details on pricing, please visit our pricing page here.

Expanded Availability

  • Availability in New Countries— Availability of Windows Azure is being expanded to customers in 48 new countries, including Russia, South Korea, Taiwan, Turkey, Egypt, South Africa, and Ukraine. Roll-out will be complete later this month, making Windows Azure one of the most widely available cloud platforms in the industry with offerings in 89 countries and in 19 local currencies.

For a full list of countries, please click here.

Together, these updates offer customers greater flexibility for spanning on-premises and cloud servers, deliver powerful enhancements for cloud applications, and continue our commitment to make Windows Azure an open platform. We invite you to sign-up for these new previews and provide your feedback!


<Return to section navigation list>

Cloud Security and Governance

Steven Vidich described Security, Privacy & Compliance Update: Availability of SSAE 16 / ISAE 3402 Attestation in a 6/13/2012 post to the Windows Azure blog:

imageIn April 2012, we launched Windows Azure Trust Center (WATC) with the goal of providing customers and partners with easier access to regulatory compliance information. We indicated that WATC would be updated on a regular basis with additional compliance programs that Windows Azure is pursuing. As we continue to make progress on this, I’d like to share some updates with you.

image_thumbWindows Azure now publishes a detailed SOC 1 Type 2 report for the core features. The audit report is available to Enterprise Agreement (volume licensing) customers under a non-disclosure agreement. The audit was conducted in accordance with SSAE 16 and ISAE 3402 standards. For more information, please visit the Windows Azure Trust Center compliance page.

The scope of the audit covers the following Windows Azure features:

  • Cloud Services (includes Web and Worker roles)
  • Storage (includes Blobs, Queues, and Tables)
  • Networking (includes Traffic Manager and Windows Azure Connect)

The following additional features were launched after the examination review period but are subject to the same controls and processes that were tested in the audit:

  • Virtual Network
  • Virtual Machines

The SOC1 Type 2 audit report attests to the fairness of the presentation for Windows Azure service description. It also examines the suitability of the design and operating effectiveness of the controls to achieve the related control objectives.

We strive to take a leadership role when it comes to security, privacy, and compliance practices, and will continue to share updates with you through this blog.


<Return to section navigation list>

Cloud Computing Events

Eric D. Boyd (@EricDBoyd) reported What’s New in Windows Azure – Chicago Tour in a 6/19/2012 post:

imageOn the evening of June 6th, Microsoft published the Meet Windows Azure Fact Sheet that provided a summary of new and improved Windows Azure capabilities. The following day, the new Windows Azure capabilities were made available at Meet Windows Azure led by Microsoft CVP, Scott Guthrie. And last week, many great Windows Azure technical sessions were delivered at TechEd North America 2012. If you missed all of those amazing opportunities to learn more about What’s New in Windows Azure, and you live in the Chicagoland area, you’re in luck.

imageStarting tomorrow, I will begin a tour of Chicago User Groups presenting “What’s New in Windows Azure”. In this “What’s New in Windows Azure” tour, I’ll present an overview of the improvements and new Windows Azure capabilities that were just announced. I will visit a number of Chicago groups both in the Western suburbs and in the Downtown Chicago Loop, so if you want to learn more about the new features and capabilities in Windows Azure, RSVP for one of the following Chicagoland events.

Chicago Windows Azure User Group
Wednesday, June 20, 2012 @ 5:30 PM
Microsoft Office – Downers Grove
3025 Highland Pkwy, Suite 300
Downers Grove, IL
http://www.meetup.com/Chicagoazure/events/65587882/

Chicago Cloud Computing Meetup
Wednesday, June 27, 2012 @ 5:30 PM
Pariveda Solutions
550 West Washington Blvd, #410
Chicago, IL
http://www.meetup.com/The-Chicago-Cloud-Computing-Meetup-Group/events/65708372/

Software Development Community
Sunday, July 1, 2012 @ 1:00 PM
Microsoft Store in Oakbrook Center
49 Oak Brook Center
Oak Brook, IL
http://www.meetup.com/SoftDev/events/43412202/


Scott Guthrie (@scottgu) lets you Meet the New Windows Azure in a 6/7/2012 post to his ASP.NET blog:

imageToday we are releasing a major set of improvements to Windows Azure. Below is a short-summary of just a few of them:

New Admin Portal and Command Line Tools

imageToday’s release comes with a new Windows Azure portal that will enable you to manage all features and services offered on Windows Azure in a seamless, integrated way. It is very fast and fluid, supports filtering and sorting (making it much easier to use for large deployments), works on all browsers, and offers a lot of great new features – including built-in VM, Web site, Storage, and Cloud Service monitoring support.

image

The new portal is built on top of a REST-based management API within Windows Azure – and everything you can do through the portal can also be programmed directly against this Web API.

We are also today releasing command-line tools (which like the portal call the REST Management APIs) to make it even easier to script and automate your administration tasks. We are offering both a Powershell (for Windows) and Bash (for Mac and Linux) set of tools to download. Like our SDKs, the code for these tools is hosted on GitHub under an Apache 2 license.

Virtual Machines

Windows Azure now supports the ability to deploy and run durable VMs in the cloud. You can easily create these VMs using a new Image Gallery built-into the new Windows Azure Portal, or alternatively upload and run your own custom-built VHD images.

Virtual Machines are durable (meaning anything you install within them persists across reboots) and you can use any OS with them. Our built-in image gallery includes both Windows Server images (including the new Windows Server 2012 RC) as well as Linux images (including Ubuntu, CentOS, and SUSE distributions). Once you create a VM instance you can easily Terminal Server or SSH into it in order to configure and customize the VM however you want (and optionally capture your own image snapshot of it to use when creating new VM instances). This provides you with the flexibility to run pretty much any workload within Windows Azure.

image

The new Windows Azure Portal provides a rich set of management features for Virtual Machines – including the ability to monitor and track resource utilization within them.

image

Our new Virtual Machine support also enables the ability to easily attach multiple data-disks to VMs (which you can then mount and format as drives). You can optionally enable geo-replication support on these – which will cause Windows Azure to continuously replicate your storage to a secondary data-center at least 400 miles away from your primary data-center as a backup.

image

We use the same VHD format that is supported with Windows virtualization today (and which we’ve released as an open spec), which enables you to easily migrate existing workloads you might already have virtualized into Windows Azure. We also make it easy to download VHDs from Windows Azure, which also provides the flexibility to easily migrate cloud-based VM workloads to an on-premise environment. All you need to do is download the VHD file and boot it up locally, no import/export steps required.

Web Sites

Windows Azure now supports the ability to quickly and easily deploy ASP.NET, Node.js and PHP web-sites to a highly scalable cloud environment that allows you to start small (and for free) and then scale up as your traffic grows. You can create a new web site in Azure and have it ready to deploy to in under 10 seconds:

image

The new Windows Azure Portal provides built-in administration support for Web sites – including the ability to monitor and track resource utilization in real-time:

image

You can deploy to web-sites in seconds using FTP, Git, TFS and Web Deploy. We are also releasing tooling updates today for both Visual Studio and Web Matrix that enable developers to seamlessly deploy ASP.NET applications to this new offering. The VS and Web Matrix publishing support includes the ability to deploy SQL databases as part of web site deployment – as well as the ability to incrementally update database schema with a later deployment.

You can integrate web application publishing with source control by selecting the “Set up TFS publishing” or “Set up Git publishing” links on a web-site’s dashboard:

image

Doing do will enable integration with our new TFS online service (which enables a full TFS workflow – including elastic build and testing support), or create a Git repository that you can reference as a remote and push deployments to. Once you push a deployment using TFS or Git, the deployments tab will keep track of the deployments you make, and enable you to select an older (or newer) deployment and quickly redeploy your site to that snapshot of the code. This provides a very powerful DevOps workflow experience.

image

Windows Azure now allows you to deploy up to 10 web-sites into a free, shared/multi-tenant hosting environment (where a site you deploy will be one of multiple sites running on a shared set of server resources). This provides an easy way to get started on projects at no cost.

image

You can then optionally upgrade your sites to run in a “reserved mode” that isolates them so that you are the only customer within a virtual machine:

image

And you can elastically scale the amount of resources your sites use – allowing you to increase your reserved instance capacity as your traffic scales:

image

Windows Azure automatically handles load balancing traffic across VM instances, and you get the same, super fast, deployment options (FTP, Git, TFS and Web Deploy) regardless of how many reserved instances you use.

With Windows Azure you pay for compute capacity on a per-hour basis – which allows you to scale up and down your resources to match only what you need.

Cloud Services and Distributed Caching

Windows Azure also supports the ability to build cloud services that support rich multi-tier architectures, automated application management, and scale to extremely large deployments. Previously we referred to this capability as “hosted services” – with this week’s release we are now referring to this capability as “cloud services”. We are also enabling a bunch of new features with them.

Distributed Cache

One of the really cool new features being enabled with cloud services is a new distributed cache capability that enables you to use and setup a low-latency, in-memory distributed cache within your applications. This cache is isolated for use just by your applications, and does not have any throttling limits.

This cache can dynamically grow and shrink elastically (without you have to redeploy your app or make code changes), and supports the full richness of the AppFabric Cache Server API (including regions, high availability, notifications, local cache and more). In addition to supporting the AppFabric Cache Server API, it also now supports the Memcached protocol – allowing you to point code written against Memcached at it (no code changes required).

The new distributed cache can be setup to run in one of two ways:

1) Using a co-located approach. In this option you allocate a percentage of memory in your existing web and worker roles to be used by the cache, and then the cache joins the memory into one large distributed cache. Any data put into the cache by one role instance can be accessed by other role instances in your application – regardless of whether the cached data is stored on it or another role. The big benefit with the “co-located” option is that it is free (you don’t have to pay anything to enable it) and it allows you to use what might have been otherwise unused memory within your application VMs.

image

2) Alternatively, you can add “cache worker roles” to your cloud service that are used solely for caching. These will also be joined into one large distributed cache ring that other roles within your application can access. You can use these roles to cache 10s or 100s of GBs of data in-memory very effectively – and the cache can be elastically increased or decreased at runtime within your application:

image

New SDKs and Tooling Support

We have updated all of the Windows Azure SDKs with today’s release to include new features and capabilities. Our SDKs are now available for multiple languages, and all of the source in them is published under an Apache 2 license and and maintained in GitHub repositories.

image

The .NET SDK for Azure has in particular seen a bunch of great improvements with today’s release, and now includes tooling support for both VS 2010 and the VS 2012 RC.

We are also now shipping Windows, Mac and Linux SDK downloads for languages that are offered on all of these systems – allowing developers to develop Windows Azure applications using any development operating system.

image

Much, Much More

The above is just a short list of some of the improvements that are shipping in either preview or final form today – there is a LOT more in today’s release. These include new Virtual Private Networking capabilities, new Service Bus runtime and tooling support, the public preview of the new Azure Media Services, new Data Centers, significantly upgraded network and storage hardware, SQL Reporting Services, new Identity features, support within 40+ new countries and territories, and much, much more.

You can learn more about Windows Azure and sign-up to try it for free at http://windowsazure.com.

You can also watch a live keynote I’m giving at 1pm June 7th (later today) where I’ll walk through all of the new features. We will be opening up the new features I discussed above for public usage a few hours after the keynote concludes. We are really excited to see the great applications you build with them.

Hope this helps.


Amy Conklin recommended that you Watch ‘Learn Windows Azure’ Live from TechEd North America, June 11 in a 6/8/2012 post:

imageAt the Meet Windows Azure event yesterday, Scott Guthrie introduced the latest platform and language support and increased flexibility for architecting, building and deploying apps with Windows Azure. If you missed the live stream, you can watch the recorded version online at meetwindowsazure.com. Guest speakers included Jeff Lawson, cofounder and CEO of Twilio, as well as the team behind pottermore.com.

Get a more technical deep-dive into the Windows Azure preview – watch Learn Windows Azure, live from TechEd in Orlando, Florida (June 11, 8am PDT – 3pm PDT). Learn Windows Azure is a full day of Windows Azure-focused content delivered by some of Microsoft’s top technical leaders, including Scott Guthrie, Mark Russinovich, Quentin Clark and Bill Staples.

  • View the topics, speakers and schedule.
  • Save the Date. Add Learn Windows Azure details to your calendar.

Video archives are available at the addresses provided.


<Return to section navigation list>

Other Cloud Computing Platforms and Services

Jeff Barr (@jeffbarr) announced Apache HBase on Amazon EMR – Real-time Access to Your Big Data in a 6/12/2012 post to his Amazon Web Services blog:

imageAll Your Base
AWS has already given you a lot of storage and processing options to choose from, and today we are adding a really important one.

imageYou can now use Apache HBase to store and process extremely large amounts of data (think billions of rows and millions of columns per row) on AWS. HBase offers a number of powerful features including:

  • Strictly consistent reads and writes.
  • High write throughput.
  • Automatic sharding of tables.
  • Efficient storage of sparse data.
  • Low-latency data access via in-memory operations.
  • Direct input and output to Hadoop jobs.
  • Integration with Apache Hive for SQL-like queries over HBase tables, joins, and JDBC support.

image_thumb3_thumbHBase is formally part of the Apache Hadoop project, and runs within Amazon Elastic MapReduce. You can launch HBase jobs (version 0.92.0) from the command line or the AWS Management Console.

HBase in Action
HBase has been optimized for low-latency lookups and range scans, with efficient updates and deletions of individual records. Here are some of the things that you can do with it:

Reference Data for Hadoop Analytics – Because HBase is integrated into Hadoop and Hive and provides rapid access to stored data, it is a great way to store reference data that will be used by one or more Hadoop jobs on a single cluster or across multiple Hadoop clusters.

Log Ingestion and Batch Analytics – HBase can handle real-time ingestion of log data with ease, thanks to its high write throughput and efficient storage of sparse data. Combining this with Hadoop’s ability to handle sequential reads and scans in a highly optimized fashion, and you have a powerful tool for log analysis.

Storage for High Frequency Counters and Summary Data – HBase supports high update rates (the classic read-modify-write) along with strictly consistent reads and writes. These features make it ideal for storing counters and summary data. Complex aggregations such as max-min, sum, average, and group-by can be run as Hadoop jobs and the results can be piped back into an HBase table.

I should point out that HBase on EMR runs in a single Availability Zone and does not guarantee data durability; data stored in an HBase cluster can be lost if the master node in the cluster fails. Hence, HBase should be used for summarization or secondary data or you should make use of the backup feature described below.

You can do all of this (and a lot more) by running HBase on AWS. You’ll get all sorts of benefits when you do so:

Freedom from Drudgery – You can focus on your business and on your customers. You don’t have to set up, manage, or tune your HBase clusters. Elastic MapReduce will handle provisioning of EC2 instances, security settings, HBase configuration, log collection, health monitoring, and replacement of faulty instances. You can even expand the size of your HBase cluster with a single API call.

Backup and Recovery – You can schedule full and incremental backups of your HBase data to Amazon S3. You can rollback to an old backup on an existing cluster or you can restore a backup to a newly launched cluster.

Seamless AWS Integration – HBase on Elastic MapReduce was designed to work smoothly and seamlessly with other AWS services such as S3, DynamoDB, EC2, and CloudWatch.

Getting Started
You can start HBase from the command line by launching your Elastic MapReduce cluster with the –hbase flag :

$ elastic-mapreduce –create –hbase –name "Jeff’s HBase Cluster" –num-instances 2 –instance-type m1.large

You can also start it from the Create New Cluster page of the AWS Management Console:

When you create your HBase Job Flow from the console you can restore from an existing backup, and you can also schedule future backups:

Beyond the Basics
Here are a couple of advanced features and options that might be of interest to you:

You can modify your HBase configuration at launch time by using an EMR bootstrap action. For example, you can alter the maximum file size (hbase.hregion.max.filesize) or the maximum size of the memstore (hbase.regionserver.global.memstore.upperLimit).

You can monitor your cluster with the standard CloudWatch metrics that are generated for all Elastic MapReduce job flows. You can also install Ganglia at startup time by invoking a pair of predefined bootstrap actions (install-ganglia and configure-hbase-for-ganglia). We plan to add additional metrics, specific to HBase, over time.

You can run Apache Hive on the same cluster, or you can install it on a separate cluster. Hive will run queries transparently against HBase and Hive tables. We do advise you to proceed with care when running both on the same cluster; HBase is CPU and memory intensive, while most other MapReduce jobs are I/O bound, with fixed memory requirements and sporadic CPU usage.

HBase job flows are always launched with EC2 Termination Protection enabled. You will need to confirm your intent to terminate the job flow.

I hope you enjoy this powerful new feature!

PS – There is no extra charge to run HBase. You pay the usual rates for Elastic MapReduce and EC2.

Rumor has it that Apache Hadoop on Windows Azure will offer HBase before too long.


Anders Samuelsson described IAM roles for EC2 instances – Simplified Secure Access to AWS service APIs from EC2 in a 6/11/2012 post:

imageToday we are introducing AWS Identity and Access management (IAM) roles for EC2 instances, a new feature that makes it even easier for you to securely access AWS service APIs from your EC2 instances. You can create an IAM role, assign it a set of permissions, launch EC2 instances with the IAM role, and then AWS access keys with the specified permissions are automatically made available on those EC2 instances.

This short video illustrates that it is easy to get started: [Link missing.]

imageUntil now, you had to securely get your AWS access keys out to your EC2 instances, which could be challenging when managing large or elastically scaling fleets. You also needed to figure out how to implement security best practices such as regularly rotating your keys. IAM roles for EC2 instances now take care of both of these for you automatically.

IAM roles for EC2 instances are available to be used with:

  • ALL EC2 instance types
  • Linux and Windows instances
  • ALL AMIs
  • Amazon VPC
  • Spot and Reserved Instances
  • North America, South America, Europe, and Asia Pacific regions

Auto Scaling and AWS CloudFormation have also added integration for roles, so that they can start EC2 instances with IAM roles on your behalf, and GovCloud support will be coming soon.

Let’s take a look behind the scenes.

We have introduced a new IAM entity called a role. IAM roles allow applications in your EC2 instances to act on your behalf. Like an IAM user, you use the Access Policy Language to specify permissions. However unlike a user, a role cannot be used to directly call AWS service APIs. A role must be “assumed” by an entity – in this case an EC2 instance, in the future releases perhaps by an IAM user. To extend upon the “AWS Hotel” analogy that we referenced in a prior blog post to explain IAM users, a housekeeper named Sally might be provided an IAM user for her day-to-day responsibilities of cleaning guest rooms, but during yearly fire drills, she can assume the role of Safety Officer, which gives her different permissions (such as access to all rooms in the building, and use of walkie-talkies to coordinate with fire officials).

When you launch an EC2 instance with an IAM role, temporary AWS security credentials with permissions specified by the role will be securely provisioned to the instance and will be made available to your application via the EC2 Instance Metadata Service. The Metadata Service will make new temporary security credentials available prior to the expiration of the current active credentials, so that valid credentials are always available on the instance.

For enhanced security, the temporary security credentials are automatically rotated for you multiple times per day. If you are developing your application with the AWS SDK, all of this will be completely transparent to your application and you only need to make minor adjustments to your code to get started.

If you previously had something similar to the code below:

AWSCredentials creds = new BasicAWSCredentials(
"AKIAIOSFODNN7EXAMPLE",
"wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY");
CredentialProvider session = new STSSessionCredentialsProvider(creds);
AmazonDynamoDB dynamo = new AmazonDynamoDBClient(session);

With the latest AWS SDK that adds support for IAM roles, you can minimize this code to the following:

AmazonDynamoDB dynamo = new AmazonDynamoDBClient();

And the AWS SDK takes care of the rest! We have tried to remove as much “muck” as possible to enable you to just focus on developing your application. Make sure to visit the Working with Roles section in the Using IAM guide and Using IAM roles with Amazon EC2 Instances in the Amazon EC2 User Guide for additional information about this new and exciting feature.


<Return to section navigation list>

Technorati Tags: Windows Azure,Windows Azure Platform,Azure Services Platform,Azure Storage Services,Azure Table Services,Azure Blob Services,Azure Drive Services,Azure Queue Services,Azure Service Broker,Azure Access Services,SQL Azure Database,SQL Azure Federations,Open Data Protocol,OData,Cloud Computing,Visual Studio LightSwitch,LightSwitch,Amazon Web Services,AWS,HBase,Hadoop,Windows Azure Active Directory,Windows Azure Virtual Machines,Windows Azure Virtual Networks

http://oakleafblog.blogspot.com/2012/06/windows-azure-and-cloud-computing-posts.html

Share this:

  • Print
  • Email
  • Facebook
  • Twitter
  • Digg
  • Reddit
  • StumbleUpon
« Copying a virtual machine data drive in Microsoft Azure
Azure ACS – Should I encrypt ClaimTypes.NameIdentifier information? »
  • Categories
    • AppFabric Team Blog (13)
    • Channel 9 (440)
    • cloud development blog (42)
    • Cloudy in Seattle (10)
    • Convective (12)
    • Matias Woloski (15)
    • Nick Harris .NET – Enterprise Development with Azure, ASP .NET MVC and Windows Phone 7 (50)
    • OakLeaf Systems (427)
    • Scott Hanselman's Blog (29)
    • ScottGu (18)
    • SQL Azure Team Blog (29)
    • Stack Overflow Azure (7527)
    • Uncategorized (16)
    • Wade Wegner (19)
    • Windows Azure Developer Tools Team (25)
    • Windows Azure Marketplace DataMarket Blog (26)
    • Windows Azure Storage Team Blog (63)
    • Windows Azure Team Blog (563)
    • Windows Phone Developer Blog (56)
    • Zane Adam's blog (22)
  • Translator
    English flagItalian flagKorean flagChinese (Simplified) flagChinese (Traditional) flagPortuguese flagGerman flagFrench flagSpanish flagJapanese flagArabic flagRussian flagGreek flagDutch flagBulgarian flagCzech flagCroatian flagDanish flagFinnish flagHindi flagPolish flagRomanian flagSwedish flagNorwegian flagCatalan flagFilipino flagHebrew flagIndonesian flagLatvian flagLithuanian flagSerbian flagSlovak flagSlovenian flagUkrainian flagVietnamese flagAlbanian flagEstonian flagGalician flagMaltese flagThai flagTurkish flagHungarian flagBelarus flagIrish flagIcelandic flagMacedonian flagMalay flagPersian flag
  • Recent Posts
    • Error running tests on TFS Deploy
    • why I get an empty access token with azure app?
    • Everything I start from a worker role has no interface, why?
    • Create Azure Worker Role or Cloud Service in Code?
    • Windows Azure and Cloud Computing Posts for 6/17/2013+
  • Advertisements

  • RSS

    Windows Azure Blog

  • Twitter
  • Categories
    AppFabric Team Blog Channel 9 cloud development blog Cloudy in Seattle Convective Matias Woloski Nick Harris .NET - Enterprise Development with Azure, ASP .NET MVC and Windows Phone 7 OakLeaf Systems ScottGu Scott Hanselman's Blog SQL Azure Team Blog Stack Overflow Azure Uncategorized Wade Wegner Windows Azure Developer Tools Team Windows Azure Marketplace DataMarket Blog Windows Azure Storage Team Blog Windows Azure Team Blog Windows Phone Developer Blog Zane Adam's blog
Mocell WordPress Theme By MagPress.com
Thanks to Cat Lovers | Meet Locals | Florida Chat
Copyright © 2013. All Rights Reserved.
loading Cancel
Post was not sent - check your email addresses!
Email check failed, please try again
Sorry, your blog cannot share posts by email.