| A compendium of Windows Azure, Service Bus, EAI & EDI Access Control, Connect, SQL Azure Database, and other cloud-computing articles. |

Note: This post is updated daily or more frequently, depending on the availability of new articles in the following sections:
- Windows Azure Blob, Drive, Table, Queue and Hadoop Services
- SQL Azure Database and Reporting
- Marketplace DataMarket, Social Analytics and OData
- Windows Azure Access Control, Service Bus, and Workflow
- Windows Azure VM Role, Virtual Network, Connect, RDP and CDN
- Live Windows Azure Apps, APIs, Tools and Test Harnesses
- Visual Studio LightSwitch and Entity Framework v4+
- Windows Azure Infrastructure and DevOps
- Windows Azure Platform Appliance (WAPA), Hyper-V and Private/Hybrid Clouds
- Cloud Security and Governance
- Cloud Computing Events
- Other Cloud Computing Platforms and Services
Azure Blob, Drive, Table, Queue and Hadoop Services
CloudTweaks asserted “Open source “Big Data” cloud computing platform powers millions of compute-hours to process exabytes of data for Amazon.com, AOL, Apple, eBay, Facebook, foursquare, HP, IBM, LinkedIn, Microsoft, Netflix, The New York Times, Rackspace, Twitter, Yahoo!, and more” in an introduction to an Apache Hadoop v1.0: Open Source “Big Data” Cloud Computing Platform Powers Millions… news release on 1/4/2012:
The Apache Software Foundation (ASF), the all-volunteer developers, stewards, and incubators of nearly 150 Open Source projects and initiatives, today announced Apache™ Hadoop™ v1.0, the Open Source software framework for reliable, scalable, distributed computing. The project’s latest release marks a major milestone six years in the making, and has achieved the level of stability and enterprise-readiness to earn the 1.0 designation.
A foundation of Cloud computing and at the epicenter of “big data” solutions, Apache Hadoop enables data-intensive distributed applications to work with thousands of nodes and exabytes of data. Hadoop enables organizations to more efficiently and cost-effectively store, process, manage and analyze the growing volumes of data being created and collected every day. Apache Hadoop connects thousands of servers to process and analyze data at supercomputing speed.
“This release is the culmination of a lot of hard work and cooperation from a vibrant Apache community group of dedicated software developers and committers that has brought new levels of stability and production expertise to the Hadoop project,” said Arun C. Murthy, Vice President of Apache Hadoop. “Hadoop is becoming the de facto data platform that enables organizations to store, process and query vast torrents of data, and the new release represents an important step forward in performance, stability and security.
“Originating with technologies developed by Yahoo, Google, and other Web 2.0 pioneers in the mid-2000s, Hadoop is now central to the big data strategies of enterprises, service providers, and other organizations,” wrote James Kobielus in the independent Forrester Research, Inc. report, “Enterprise Hadoop: The Emerging Core Of Big Data” (October 2011).
Dubbed a “Swiss army knife of the 21st century” and named “Innovation of the Year” by the 2011 Media Guardian Innovation Awards, Apache Hadoop is widely deployed at organizations around the globe, including industry leaders from across the Internet and social networking landscape such as Amazon Web Services, AOL, Apple, eBay, Facebook, Foursquare, HP, LinkedIn, Netflix, The New York Times, Rackspace, Twitter, and Yahoo!. Other technology leaders such as Microsoft and IBM have integrated Apache Hadoop into their offerings. Yahoo!, an early pioneer, hosts the world’s largest known Hadoop production environment to date, spanning more than 42,000 nodes. [Microsoft emphasis added.]
“Achieving the 1.0 release status is a momentous achievement from the Apache Hadoop community and the result of hard development work and shared learnings over the years,” said Jay Rossiter, senior vice president, Cloud Platform Group at Yahoo!. “Apache Hadoop will continue to be an important area of investment for Yahoo!. Today Hadoop powers every click at Yahoo!, helping to deliver personalized content and experiences to more than 700 million consumers worldwide.”
“Apache Hadoop is in use worldwide in many of the biggest and most innovative data applications,” said Eric Baldeschwieler, CEO of Hortonworks. “The v1.0 release combines proven scalability and reliability with security and other features that make Apache Hadoop truly enterprise-ready.”
“Gartner is seeing a steady increase in interest in Apache Hadoop and related “big data” technologies, as measured by substantial growth in client inquiries, dramatic rises in attendance at industry events, increasing financial investments and the introduction of products from leading data management and data integration software vendors,” said Merv Adrian, Research Vice President at Gartner, Inc. “The 1.0 release of Apache Hadoop marks a major milestone for this open source offering as enterprises across multiple industries begin to integrate it into their technology architecture plans.”
Apache Hadoop v1.0 reflects six years of development, production experience, extensive testing, and feedback from hundreds of knowledgeable users, data scientists, systems engineers, bringing a highly stable, enterprise-ready release of the fastest-growing big data platform.
It includes support for:
- HBase (sync and flush support for transaction logging)
- Security (strong authentication via Kerberos)
- Webhdfs (RESTful API to HDFS)
- Performance enhanced access to local files for HBase
- Other performance enhancements, bug fixes, and features
- All version 0.20.205 and prior 0.20.2xx features
“We are excited to celebrate Hadoop’s milestone achievement,” said William Lazzaro, Director of Engineering at Concurrent Computer Corporation. “Implementing Hadoop at Concurrent has enabled us to transform massive amounts of real-time data into actionable business insights, and we continue to look forward to the ever-improving iterations of Hadoop.”
“Hadoop, the first ubiquitous platform to emerge from the ongoing proliferation of Big Data and noSQL technologies, is set to make the transition from Web to Enterprise technology in 2012,” said James Governor, co-founder of RedMonk, “driven by adoption and integration by every major vendor in the commercial data analytics market. The Apache Software Foundation plays a crucial role in supporting the platform and its ecosystem.”
Availability and Oversight
As with all Apache products, Apache Hadoop software is released under the Apache License v2.0, and is overseen by a self-selected team of active contributors to the project. A Project Management Committee (PMC) guides the Project’s day-to-day operations, including community development and product releases. Apache Hadoop release notes, source code, documentation, and related resources are available at http://hadoop.apache.org/.
About The Apache Software Foundation (ASF)
Established in 1999, the all-volunteer Foundation oversees nearly one hundred fifty leading Open Source projects, including Apache HTTP Server — the world’s most popular Web server software. Through the ASF’s meritocratic process known as “The Apache Way,” more than 350 individual Members and 3,000 Committers successfully collaborate to develop freely available enterprise-grade software, benefiting millions of users worldwide: thousands of software solutions are distributed under the Apache License; and the community actively participates in ASF mailing lists, mentoring initiatives, and ApacheCon, the Foundation’s official user conference, trainings, and expo.
For more information, visit http://www.apache.org/.
Source: Apache
<Return to section navigation list>
SQL Azure Database and Reporting
Avkash Chauhan (@avkashchauhan) recommended that you Assess your SQL Server to SQL Azure migration using SQL Azure Compatibility Assessment Tool by SQL Azure Labs in a 1/4/2012 post:
SQL Azure team announced today about the release of a new experimental cloud service, "SQL Azure Compatibility Assessment”. This tool is a created by SQL Azure Labs and if you are considering moving your SQL Server databases to SQL Azure, you can use this assessment service to check if your database schema is compatible with SQL Azure grammar. This service is very easy to use and does not require Azure account.
"SQL Azure Compatibility Assessment" is an experimental cloud service. It’s aimed at database administrators who are considering migrating their existing SQL Server databases to SQL Azure. This service is super easy to use with just a Windows Live ID. Here are the steps:
- Generate a .dacpac file from your database using SQL Server Data Tools (SSDT) CTP4. You can either run SqlPackage.exe or import the database into an SSDT project and then build it to generate a .dacpac. SQL Server 2005, 2008, 2008 R2, 2012 (CTP or RC0) are all supported.
- Upload your .dacpac to the "SQL Azure Compatibility Assessment" cloud service and receive an assessment report, which lists the schema objects that are not supported in SQL Azure and that need to be fixed prior to migration.
Learn more about Microsoft Codename "SQL Azure Compatibility Assessment" in this great video:
Learn Technical details about "SQL Azure Compatibility Assessment" tool:
SqlPackage.exe is a command line utility that automates the following database development tasks:
- Extraction: creating a database snapshot (.dacpac) file from a live SQL Azure database
- Publishing: incrementally updates a database schema to match the schema of a source database
- Report: creates reports of the changes that result from publishing a database schema
- Script: creates a Transact-SQL script that publishes a database schema
I intend to use this new tool after I bulk upload my 30+ GB tab-separated text file to SQL Server 2008 R2 in preparation for a move to SQL Azure federations.
<Return to section navigation list>
MarketPlace DataMarket, Social Analytics and OData
Joe McKendrick asserted “Growing interest in the cloud, and a desire for more vendor independence, may drive REST deeper into the enterprise BPM space” as a deck for his REST will free business process management from its shackles: prediction article of 1/3/2012 for ZDNet’s ServiceOriented blog:
The REST protocol has been known for its ability to link Web processes in its lightweight way, but at least one analyst sees a role far deeper in the enterprise. This year, we’ll see REST increasingly deployed to pull together business processes.
ZapThink’s Jason Bloomberg predicts the rise of RESTful business process management, which will enable putting together workflows on the fly, without the intervention of heavyweight BPM engines.
People haven’t really been employing REST to its full potential, Jason says. REST is more than APIs, it’s an “architectural style for building hypermedia applications,” which are more than glorified Web sites. REST is a runtime workflow, he explains. It changes the way we might look at BPM tools, which typically have been “heavyweight, integration-centric tools that typically rely upon layers of infrastructure.”
“With REST, however, hypermedia become the engine of application state, freeing us from relying upon centralized state engines, instead allowing us to take full advantage of cloud elasticity, as well as the inherent simplicity of REST.”
The shift will take time, beyond 2012. But with growing interest in the cloud, and a desire for more vendor independence, means its time for RESTful BPM.
As goes REST, so goes OData.
<Return to section navigation list>
Windows Azure Access Control, Service Bus and Workflow
Bruno Terkaly (@brunoterkaly) started a Service Bus series with Part 1-Cloud Architecture Series-Durable Messages using Windows Azure (Cloud) Service Bus Queues–Establishing your service through the Portal
Introduction
The purpose of this post is to explain and illustrate the use of Windows Azure Service Bus Queues.
This technology solves some very difficult problems. It allows developers to send durable messages among applications, penetrating network address translation (NAT) boundaries, , or bound to frequently-changing, dynamically-assigned IP addresses, or both. Reaching endpoints behind these types of boundaries in extremely difficult. The Windows Azure Service Bus Queuing technologies makes this challenge very approachable.
There are many applications for this technology. We will use this pattern to implement the CQRS pattern in future posts.
Well known pattern in computer science
These technologies refleofct well known patterns in computer science, such as the "Pub-Sub" or publish-subscribe pattern. This pattern allows senders of messages (Publishers) to send these messages to listeners (Subscribers) without knowing anything about the number of type of subscribers. Subscribers simply express an interest in receiving certain types of messages without knowing anything about the Publisher. It is a great example of loose coupling.
Publish-Subscribe Pattern: http://en.wikipedia.org/wiki/Publish/subscribe
Topics
Windows Azure also implements the concept of "Topics." In a topic-based system, messages are published to "topics" or named logical channels. Subscribers in a topic-based system will receive all messages published to the topics to which they subscribe, and all subscribers to a topic will receive the same messages. The publisher is responsible for defining the classes of messages to which subscribers can subscribe.
Getting Started at the portal
The next few screens will walk you through establishing a namespace at the portal.
Essential Download
To create a Service Bus Queue service running on Azure you’ll need to download the Azure SDK here:
Establishing a namespace for the service bus endpoint
Select “Service Bus, Access Control & Caching” as seen below.
Creating a new service bus endpoint
Click “New”
Providing a namespace, region. Selecting services.
The end result
Summary of information from portal
Thirumalai Muniswamy described Implementing Azure AppFabric Service Bus – Part 2 in a 1/4/2012 post to his Dot Net Twitter blog:
In my last post, I had explained about what Azure App Fabric is, the types and how Relaying Messaging works. From this post, we will see a real time example working with Windows Azure App Fabric – Relaying Messaging.
From the Azure training kit provided by Azure Team has lots of great materials to learn about each concepts of Azure. I hope this example also will add valuable learning material for those who starts implementing Azure App Fabric in their applications.
Implementation Requirement:
We are going to take an example of Customer Maintenance entry screen which has CURD (Create, Update, Read and Delete) operations on customer data. The service will be running in one of the on-premise server and exposed to public using Service Bus. The client Web Application will be running in another/same local system or in the cloud which consume the service to show the list of customers. The user can add a new customer, modify an existing customer by selecting from the customer list or even delete a customer from the list directly. The functionality is simple, but we are looking here how to implement with Service Bus and how the customer data flows from service to cloud and to client.
I am going to use Northwind database configured in my on-premise SQL Server for data storage. So make sure to have Northwind database for implementing this source code. (You can get the database script from here)Requirements
- You required to have Azure Subscription with App Fabric enabled.
- Visual Studio 2010 (for development and compilation)
- Install Windows Azure SDK for .NET (http://www.windowsazure.com/en-us/develop/net/)
Note: The Service Bus and Caching libraries are included as part of Windows Azure SDK from v1.6. So there is no need of installing additional setup for Azure App Fabric as we have done before.
Creating a Service Bus Namespace
Pls skip this step if you already have namespace created or know how to create.Step 1: Open Azure Management Portal using https://windows.azure.com and select the Service Bus, Access Control & Caching tab.
The portal will open Service panel in the left hand side.
Step 2: Select the Service Bus node and select the Subscription under which you required to create the Service Bus namespace.
Step 3: Press New menu button in the Service Namespace section. The portal will open Create a New Service Namespace popup window.
Step 4: Make sure Service Bus checkbox selected and provide the input as defined below.
- Enter the meaningful namespace in the Namespace textbox and verify the namespace availability with Check Availability.
Note:
- The namespace should not contain any special characters such as hyphen (_), dot (.) or any special characters. It can contain any alphanumeric letters and it should starts with only a characters.
- The Service Namespace name must be greater than 5 and less than 50 characters in length.
- As the namespace should be unique across all the service bus namespace created around the world, it would be better to follow some basic standard to form the namespace in the organization level.
For Example – Create a unique namespace at application level or business level. So, the namespace should be as defined below
<DomainExtention><DomainName><ApplicationName>
For Ex: ComDotNetTwitterSOP
The Service Bus endpoint URI would be
http://ComDotNetTwitterSOP.servicebus.windows.net
Once the Namespace created at application level, we can add other important names through hierarchies at the end of the service bus URI.
http://<DomainExtention><DomainName><ApplicationName>.servicebus.windows.net/<ServiceName>/<EnvironmentName>/<VersionNo>
- The Service Name will specify the name of the service
- The Environment Name will specify whether the namespace deployed for Test, Staging and Prod
- The Version No will specify the version number of the release such as V0100(for Version 1.0), V0102 (for Version 1.2), V01021203 (for V 1.2.1203) etc.,
So the fully qualified namespace URI for Sales Invoice would be
http://ComDotNetTwitterSOP.servicebus.windows.net/SalesInvoice/Test/V0100
This is an example of how a service bus service bus endpoint could be. Please decide as per your organization requirements.- Select the Country / Region (Good to select nearby where most of the consumer will reside and make the service also nearby the Country / Region selected).
- Make sure the Subscription selected is correct or select the correct one from the drop down list.
- Press the Create Namespace button.
The portal will create the namespace and list under the subscription selected. Initially the namespace status will be Activating… for some time and then will become Active.
Step 5: Select the namespace created, you can see properties panel in the right hand side. Press the View button under the Default Key heading from the properties.
The portal pops up a Default Key window with two values Default Issuer and Default Key. These two values are important to expose our service to cloud. So press the Copy to Clipboard button to copy to clipboard and save somewhere temporarily.
Note: When pressing Copy to Clipboard button, the system might popup Silverlight alert. You can press Yes to copy it to clip board.
Keep in mind
- Only one on-premise service can listen on a particular service bus endpoint address. (except NetEventRelayBinding, which is designed to listen multiple services)
- When attempting to use same address for multiple services, the first service only will take place. The remaining services will fail.
- An endpoint can use part of other endpoint, where as it can’t completely use other endpoint address.
For Ex: When an endpoint defined as http://ComDotnetTwitterSOP.servicebus.windows.net/SalesInvoiceReport/Test/V0100 which can’t used completely by other service bus endpoint. So defining http://ComDotnetTwitterSOP.servicebus.windows.net/SalesInvoiceReport/Test/V0100/01012012 endpoint is wrong. But below is accepted. http://ComDotnetTwitterSOP.servicebus.windows.net/SalesInvoiceReport/Test/V0102Further steps will be published on next post soon.
<Return to section navigation list>
Windows Azure VM Role, Virtual Network, Connect, RDP and CDN
Amit Kumar Agrawal posted Remote Desktop with Windows Azure Roles on 12/30/2011 (missed when published):
The Windows Azure SDK 1.3 and later adds the ability to use Remote Desktop Services to access Windows Azure roles. Visual Studio lets you configure Remote Desktop Services from a Windows Azure project. To enable Remote Desktop Services, you must create a working project that contains one or more roles and then publish it to Windows Azure.
Note:This ability to access a Windows Azure role is intended for troubleshooting or development only. The purpose of each virtual machine is to run a specific role in your Azure application and not to run other client applications.
To enable Remote Desktop Services follow these steps
- Open Solution Explorer, right-click the name of your project, and then click Publish.
- The Deploy Windows Azure project dialog box appears. At the bottom of the dialog box, click the Configure Remote Desktop connections link at the bottom.
- The Remote Desktop Configuration dialog box appears. checked the checkbox Enable connections for all roles.
Note :=>Visual Studio is designed to enable or disable Remote Desktop Services for all roles in your project. However, it will write remote desktop configuration information for each role. If you manually modify this information to disable Remote Desktop Service for some roles and not others, Visual Studio will no longer be able to modify the configuration and will display a dialog box that communicates this information.- You can select an existing certificate from the drop-down list or you can create a new one.
Note:=>The certificates that are needed for a remote desktop connection are different to the certificates that are used for other Windows Azure operations. The remote access certificate must have a private key.- To create a new certificate select <create> from the drop-down list.
The Create Certificate dialog box appears.
- Type a friendly name for the new certificate and then click OK
- To upload this certificate to the Windows Azure Platform Management portal, click View.
- Click the Details tab.
- To copy this certificate to a file, click Copy to File
- To export a private key for this certificate, select Yes, export the private keyand then click Next.
- To select the default export file format, click Next
- To protect this private key using a password, type a password. Confirm this password and then click Next.
- To obtain the path for this certificate file to use to upload the certificate, click Browse and copy the path shown in the Save As dialog box. Type the name of the file for this certificate in File name. Click Save, then click Next.
- To create this file, click Finish.
- Using the Windows Azure Platform Management portal, upload the certificate for the hosted service that you will connect to with Remote Desktop Services.
Note:=>If you attempt to deploy and you have not uploaded your Remote Desktop certificate to Windows Azure, you will receive an error message and your deployment will fail.- Into the Remote Desktop Configuration dialog box, type a Username and a Password.
Note:=>If the password does not meet the complexity requirements, a red icon will appear next to the password text box. A password that contains a combination of capital letters, lower case letters, and numbers or symbols will pass the complexity requirements.- Choose an account expiration date. The expiration date will automatically block any remote desktop connections when the date passes.
- Click OK. A number of settings are added to the .cscfg and .csdef files to enable Remote Access Services.
- If you are ready to publish your Windows Azure application, in the Deploy Windows Azure Project dialog box, click OK. If you are not ready to publish, click Cancel. Your Remote Desktop configuration will still be saved, and you can publish your application at a later date.
- Once you have published your project to Windows Azure, log on to the Management Portal, and click Hosted Services, Storage Accounts & CDN in the lower right hand corner of the screen.
- Click Hosted Services to see the hosted services currently running.
- Select your role in the Management portal. In the Remote Access group, ensure that the Enable check box is selected and the Configure button is enabled. This shows that the deployment is enabled for Remote Desktop.
- Select the role instance that you want to connect to, which should enable the Connect button in the Management Portal interface.
- Click Connect. Your browser will prompt you to download a .RDP file, which you can open on your local computer.
- Open the .RDP file and enter the user and password that you set up in the earlier steps. If you are on a domain, you might have to put a \ in front of the username.
- You should now be logged into your remote session.
<Return to section navigation list>
Live Windows Azure Apps, APIs, Tools and Test Harnesses
MarketWire asserted “Delphi [Automotive] Unveils New MyFi Connecting With Safety Vehicle at 2012 CES; Delphi Uses Microsoft Windows Azure to House and Deliver Vehicle Content to Drivers” in an introduction to its Delphi [Automotive] MyFi(tm) Systems Connect With Safety and Provide Access to Online Services Through Cloud-Based Portal press release of 1/4/2011:
Harnessing deep infotainment, user experience and safety expertise in a fully integrated system, Delphi Automotive … is demonstrating the possibilities of its MyFi(tm) Connecting with Safety vision at the 2012 Consumer Electronics Show (Las Vegas Convention Center, CP26), January 10-13 in Las Vegas.
"Delphi has been at the forefront of developing innovative infotainment technologies since 1936 when we integrated the first radio in a production automobile’s dashboard," said Jugal Vijayvargiya, general director of Delphi’s Infotainment & Driver Interface product business unit. "In today’s environment, innovation means creating infotainment solutions that provide the ultimate user experience while enhancing driver and passenger safety."
Recognizing the importance of drivers being focused on the driving task as well as the desire of consumers to be entertained, informed and connected 24/7, Delphi has applied its vehicle integration expertise to offer vehicle manufacturers connected systems that are designed to help maximize safety on the road.
MyFi systems allow drivers to enjoy the information and entertainment they expect while keeping their eyes on the road and hands on the wheel. Using voice recognition, text-to-speech, large touch-screens, reconfigurable displays and workload management technology, the connected systems tailor information available to drivers depending upon the driving environment. When the vehicle is in park, more information is available to users than when it is in drive. Additionally, when data from safety sensors is linked — and certain unsafe conditions are detected such as lane drift, stopped traffic ahead and driver drowsiness — audible and visual warnings redirect the driver’s attention and if necessary, automatic braking is engaged.
At CES, Delphi will unveil its newest MyFi feature, a cloud-based portal that uses Windows Azure to bring OEMs and consumers a unique global connectivity solution. Delphi selected Windows Azure for its agility, efficiency and global reach. With Azure, Microsoft manages content storage and delivery, while Delphi provides the unique convenience features of its cloud-based solution. [Emphasis added.]
"Windows Azure allows us to rapidly respond to customer needs and to deploy back-end services efficiently," noted Doug Welk, chief engineer of Delphi Advanced Infotainment and Driver Interface. "Its global reach is helping Delphi to support customers worldwide and to provide contemporary, new value — the ability of the vehicle to exchange data with the web."
"Imagine the opportunities this type of system makes possible," added Vijayvargiya. "Not only will it provide the functionality consumers expect today, it will enable much more. With this system’s flexibility, vehicle owners can add new apps and functionality throughout vehicle life. Drivers can personalize the display of the vehicle’s instrument cluster from home, selecting from multiple designs or even creating their own, arrange features and controls to make them function effectively in a way that wasn’t possible before, and check diagnostics such as tire pressure, engine health and brake life from a portable tablet or smart phone. You can even create and store personalized accounts for different drivers. When companies like Delphi and Microsoft work together, the possibilities are limitless."
Unique Cohesive Value Showcasing advanced automotive electronics that work together to enhance virtually every aspect of the driving experience, Delphi has merged safety, connectivity and user experience technology in an integrated vehicle system that provides the latest connectivity features, permits a remarkable range of communication, manages driver workload and mitigates driver distraction.
"Delphi’s MyFi Connecting with Safety vehicle creates the optimal user experience," concluded Vijayvargiya. "It exemplifies our vision: a world where it is simple for drivers to interact with their vehicles from wherever they are, where being connected and entertained does not compromise safety and where drivers are always focused on the driving task."
Interviews at CES Delphi will have technical specialists and business leaders available for interviews at CES. Please contact linda.s.ferries@delphi.com for additional information.
About Delphi Delphi is a leading global supplier of electronics and technologies for automotive, commercial vehicle and other market segments. Operating major technical centers, manufacturing sites and customer support facilities in 30 countries, Delphi delivers real-world innovations that make products smarter and safer as well as more powerful and efficient. Connect to innovation at www.delphi.com
The names of actual companies and products mentioned herein may be the trademarks of their respective owners.
SOURCE: Delphi Corporation
Delphi Automotive is a spinoff from General Motors Corp.
<Return to section navigation list>
Visual Studio LightSwitch and Entity Framework 4.1+
Sumit Sarkar described Getting started with Microsoft Visual Studio LightSwitch to Oracle, Sybase, and DB2 in a 1/3/2012 post to Progress DataDirect blog:
Interest in Microsoft Visual Studio LightSwitch is definitely trending up based on recent data access consulting projects. Organizations are getting their feet wet with SQL Server for a POC, and then expanding to other systems running Oracle, DB2, and Sybase. From these conversations, there is a buzz about Microsoft LightSwitch’s ability to quickly modernize proven tools such as MS Access, MS Excel, Lotus, etc. that were not necessarily designed for easy cloud or desktop deployments.
Benefits of using Connect for ADO.NET Entity Framework providers with Microsoft Visual Studio LightSwitch:
- Quickly develop and deploy LightSwitch applications for Oracle, DB2 iSeries and Sybase.
- Installation and deployment is easy!
- Absolutely no database client is required!
Review the advantages of running DataDirect Connect for ADO.NET for your next project.
The preliminary steps are:
1. Download and install a free 15-day trial today of the Progress DataDirect Connect for ADO.NET Oracle provider.
2. Download and install GA version of LightSwitch with latest patch. Note that there are known issues with pre-GA/Beta releases that have been addressed by Microsoft.
3. Launch Visual Studio LightSwitch
4. Attach to external Data Source
5. Choose ‘Oracle Database’ for DataDirect Connect for ADO.NET Entity Framework provider (note: Sybase and DB2 are listed under ‘Progress DataDirect’)
6. Enter connection information to data source.
6. Click on Advanced Tab and disable distributed transactions to support non SQL Server connections.
7. Click OK to create new external data source.
8. Create a new screen, and specify the newly created data source.
Once you have the data connectivity established with Progress DataDirect, you can find additional tutorials on the web:
http://www.lightswitchtutorial.com/2010/08/lightswitch-tutorial-creating-a-basic-application/
Corrected a few instances of Lightswitch to LightSwitch.
Return to section navigation list>
Windows Azure Infrastructure and DevOps
Paul MacDougall asserted “Look for Linux and Hadoop twists as Microsoft bolsters its cloud platform. Some observers say the changes could move Azure into more direct competition with Amazon Web Services” in a deck for his Microsoft Azure In 2012: Watch Out Amazon? article of 1/4/2011 for Information Week:
Microsoft plans to bolster Windows Azure in the coming months as it looks to fill gaps in the cloud OS. Redmond’s goal: to make Azure a more compelling environment for mission-critical enterprise applications and services while reducing migration hassles.
Observers suggest the changes will go so far as to move Azure from platform as a service (PaaS) into a more comprehensive infrastructure as a service (IaaS) play, a market where Amazon dominates with Amazon Web Services. PaaS typically offers users and developers a plug-and-play environment for apps, but key choices, like underlying OS and database, are limited. IaaS provides raw computing power, and users get more choices, but also more responsibilities.
Microsoft is reportedly making changes to the way Azure implements virtual machines (VMs) so that it can accommodate a wider variety of software–even Linux, which could run on top of Azure in a VM. Azure’s present VM role is extremely limited by the fact that it does not offer a persistent state, meaning that data is lost in the event of a reboot, failover, or other interruption.
The addition of a so-called persistent VM to Azure, which would effectively create a hypervisor in the sky, means businesses could in theory upload VMs running Linux, SharePoint, SQL Server, or other "stateful" applications. Microsoft is said to be preparing a Community Technology Preview (CTP) of such features that could roll out soon. Company officials wouldn’t comment.
Benjamin Day, principal at Benjamin Day Consulting in Boston and a Microsoft MVP, told me that he is "extremely confident" that Azure will gain persistent VM capabilities, although he is not certain when. "It is killer and it’s going to be really valuable," Day said. "The fact that it has not been available has been just awful, because the Amazon platform has done it for years."
Day said the technology would allow businesses to upload virtually any job or application they are running in Hyper-V in Windows Server to Azure, making the service a more practical and potentially less costly option for many organizations. "This is where Azure has to go to be competitive," he said.
Microsoft, and this is confirmed, is also rolling out a CTP of Apache Hadoop for Azure. The idea is to make Azure a service that can handle so-called Big Data–large data sets that businesses collect from everything from call centers to electronic smart sensors embedded in their products.
The company has added tools atop Hadoop that allow users to set up and configure the framework on Azure "in a few hours instead of days" according to Val Fontama, the company’s senior product manager for SQL Server, in a blog post.
Microsoft has also added JavaScript libraries that let programmers write JavaScript programs for MapReduce, the Google-inspired distributed computing framework that’s at the foundation of Hadoop, and access them on Azure through a browser. The idea is to make JavaScript "a first-class programming language in Hadoop," said Fontama.
"These improvements reduce the barrier to entry by enabling customers to easily deploy and explore Hadoop on Windows," said Fontama. The Hadoop-on-Azure CTP will also offer an add-in for Hive, which layers data warehousing capabilities onto Hadoop. That will give users a way to interact with Hadoop data through Excel and Microsoft business intelligence tools. For programmers unfamiliar with the Hadoop environment, Microsoft has conveniently added its Metro interface, borrowed from Windows Phone and Windows 8, over Hadoop tools.
Azure, launched two years ago, is key to Microsoft’s cloud strategy, but the company is cagey about how many business customers it’s attracting. To be sure, there are some high-profile wins like Boeing, Toyota, and Fujitsu, but Redmond won’t say how many users Azure has in total, nor does it break out revenue for the service. Customer fees vary depending on the amount of compute and storage resources consumed.
With the changes coming to Azure in 2012, Microsoft "wants to make sure you have no excuses for avoiding their platform," Day said. Amazon, which says it’s got 20,000 active customers on AWS CloudFront, should take notice.
Steve Plank (@plankytronixx) pointed to a new Microsoft white paper in his The Cloud Skills Debate post of 1/4/2011:
The notion of computing in the cloud naturally attracts the interest of developers but a number of IT Pros see it as a threat. In a lot of cases the core skills transfer directly to cloud technologies but some IT Pros might want to develop new skills to meet the demand of the new set of roles that are emerging. There is a de-emphasis on patching, upgrading and general service management and a greater emphasis on monitoring and integration with the core on-premises infrastructure plus a greater understanding of virtualisation and hybrid clouds.
A new white paper on cloud skills might be of help.
From the Cloud Computing: What IT Professionals Need to know whitepaper’s introduction:
Cloud computing promises new career opportunities for IT professionals. In many cases, existing core skill sets transfer directly to cloud technologies. In other instances, IT pros need to develop new skill sets that meet the demand of emerging cloud job roles.
Companies that consider moving to cloud computing will want to educate their IT professionals about the potential opportunities ahead so that they can build staff capabilities and skills ahead of the change. Chief Information Officers (CIO) who want to generate more business value from IT by necessity have to be in the front line of cloud skills education — both for themselves and to build training capacity for their IT staff.
The emerging cloud world offers those with the capability to build and grow their portfolio of skills. This paper explores the advantages of moving to the cloud and outlines the delta skill sets IT pros will want to acquire. It describes what the cloud offers and how it applies to and impacts existing infrastructure, including such issues as cost, security, data control, and integrity.
Stephen O’Grady (@sogrady) posted Revisiting the 2011 Predictions, Part 1 to his tecosystems (Redmonk) blog on 1/4/2011. Following are his 2011 predictions related to Windows Azure:
Predicting is an easier business than it once was. True, technology is hysterically accelerating rates of change and disruption, but that’s only relevant if the substance of your predictions matters. Which all too often, these days, it doesn’t. Analysts and pundits are able to prognosticate with relative impunity; who has the time to go back and check their accuracy? Pageview driven models, in fact, reward wilder predictions because the error cost is, generally, approaching zero. Unless you predicted, say, that Linux would be killed off by Windows NT, nobody will remember later.
I find value in reviewing my annual predictions, however. If they prove correct, that’s useful. If they were not, understanding the reasons why is important to adjusting our models moving forward.
Because I made the mistake of making better than a dozen predictions last year, this year’s review will be delivered in two parts. Part 1, below, will cover my predictions for browsers, the cloud, data, developers and programming language frameworks. Part 2, covering predictions within hardware, mobile, NoSQL, open source and programming languages, will hit tomorrow.
If you’d prefer to read last year’s first, they can be found here. …
Cloud
PaaS Adoption Will Begin to Show Traction, With Little Impact on IaaS Traction
The first Platform-as-a-Service providers essentially asked developers to trade choice for development speed. Like Ruby on Rails – itself the basis for multiple first generation PaaS platforms – PaaS was built for those that would embrace constraints. But PaaS platforms never saw the type of growth that Rails experienced, in part because of the further loss of control that the cloud represents. It’s one thing to have a web framework like Rails dictate the way that you build web applications; having PaaS platforms also choose the operating system, database, version control systems and more was too much.
Which is why second and third generation PaaS providers have furiously removed barriers to entry, adding additional runtimes, open sourcing the underlying platform and allowing you to pick your provider. Which, in turn, is why adoption of PaaS is accelerating. VMware CEO Paul Maritz calls PaaS “the 21st-century equivalent of Linux,” which explains not only why they feel compelled to compete in the space, but also why Red Hat might.
Virtually every vendor in this space is reporting growth similar to the Hacker News trajectories for Cloud Foundry and Openshift (below).
In spite of the growth of PaaS, however, none of the metrics we track reflect any decline in usage of general infrastructure platforms. Quite the contrary, in fact.
I count this as a hit. …
Data
…
Hadoop Will Become the MySQL of Big Data
EMC, HP, IBM, NetApp and even Oracle all have Hadoop – or in EMC’s case, MapReduce – plays in market. Microsoft actually deprecated its own Dryad initiative in favor of the Apache project. Players from AsterData to CouchBase to EnterpriseDB to MarkLogic to Tableau to Vertica have purpose built Hadoop connectors. The commerical distribution space, once essentially owned solely by Cloudera, has expanded to multiple third parties with varying points of differentiation. (Emphasis added.)
Hadoop interest elsewhere, meanwhile, has not slowed.
Need I say more about the growing ubiquity of Hadoop? I count this as a hit. …
Frameworks
Node.js Will Continue its Growth Trajectory
October was a rough month for Node.js, with posts like Node.js is Cancer and node.js Is VB6 – Does node.js Suck? following the tradition of March reddit discussions like Is NodeJS Wrong? The Trough of Disillusionment, it seemed, had arrived well ahead of schedule.
Except that interest metrics showed no commensurate decline. Node took – again – three of the Top 5 spots in inbound search queries within RedMonk Analytics. Which is unsurprising against the backdrop of Google’s Insights for Search numbers.
Over on GitHub, meanwhile, which itself has achieved dramatic growth, Node.js is the second most popular watched repository, ahead of Rails, jQuery, HTML5-Boilerplate, and Homebrew. Microsoft clearly perceives this growth, because it has worked with Joyent to create a stable build of Node for Windows which in turn led to an SDK for Azure.
All of which means nothing except that Node’s growth trajectory continues.
I count this as a hit.
<Return to section navigation list>
Windows Azure Platform Appliance (WAPA), Hyper-V and Private/Hybrid Clouds
Turgut Haspolat (@turguthaspolat) described Some Experinces in Our Private Cloud Implementation for the Turkish government in a 12/30/2011 post that succinctly describes Microsoft System Center features:
We are carrying out a big project for the Turkish Public Procurement Authority as a Fujitsu Technology Solutions Company. The project aims to achieve ensuring the execution of public procurement from the definition of the needs to the signing of the contract in a completely electronic environment. Our electronic procurement project is considered as one of the 11 components of E-Government Projects in Turkey.
Private Cloud Implementation
Firstly we know that the importance of private cloud model. By thinking strategically, we have preferred the use of private cloud technologies due to the needs of mission-critical operations that have high requirements for security, availability, and require a greater degree of control and ownership of the data.
A private cloud provides a more effective way to deploy, use, and manage VMs, applications, and other IT resources on hardware that’s dedicated to a single organization. Microsoft’s private cloud technologies, embodied in System Center 2012 and Hyper-V, make this possible. We have the virtualization capability, and end-to-end service management capabilities taking advantage of Microsoft System Center and Hyper-V.
Our aim is to provide the users with access to our services for a high percentage of time while reducing unscheduled outages.
Business continuity
In this context, we started to prepare a business continuity architecture and strategy considering a private cloud model as part of our infrastructure to take advantage of the benefits of Microsoft virtualization and the System Center suite to transform the way we deliver services to the business.
Business continuity is our one of the important issues which we should address in our mission-critical business activities. Also we have developed disaster prevention and emergency response plans.
We needed to ensure that our business-critical IT system infrastructure is ready for any kind of threat to our business operations and services – from simple failures like unplugged cables or hard drive crashes, to more catastrophic events like earthquakes, hurricanes, and other natural disasters, as well as terrorist attacks. So we put together an appropriate strategy for disaster readiness or resilience of critical business processes. So, we focused on getting to the core of how to continue critical business activities with minimal resources.
We’ve prepared a Business Continuity Plan based on the visualization and standardization of entire business processes in compliance with standards which we use BS 25999. We’ve started with a methodology that comprises, of course, defining Program Scope, Threat Assessment, Business Impact Analysis, Recovery Strategy issues.
At first, we’ve identified threat identifications, analysis and evaluation for all information system functional areas. And then we prioritized business functions by assessing the potential financial (quantitative) and non-financial (qualitative) damages to our business when a particular event disrupts our operations and services in the Business Impact Analysis phase.
Thus, we defined Recovery Time Objectives (RTOs) and Recovery Point Objectives (RPOs) in accordance with this analysis as well as the critical resource with the highest impact. After that, we provided the measures and strategies to ensure our business recovery and continuity in case of disasters, failures or other major outage. We have determined the optimum balance between improved responsiveness to an unexpected situation and the amount invested into measures for business continuity by developing action plans in Recovery Strategy phase.
Finally, we’ve developed a Recovery Plan based on risk evaluation and business impact analysis and optimized action plans ensure that the system engineers in charge will respond quickly in their roles so that critical business functions can continue.
Effective business continuity and disaster recovery management requires not only technical expertise, but also a detailed understanding of business requirements. At first we had some challenges to realize that adopting a private cloud infrastructure. Actually Private Cloud isn’t just a technology change—it also requires new attitudes and behaviors and also cultural transformation. We’ve experienced it.
Microsoft System Center Suite
We have continued to make efforts to increase system availability by adopting the System Center Suite. We have also put in place plans to ensure smooth business operation in the event of a disaster affecting IT systems. These plans are able to minimize damage and ensure immediate recovery from such damage.
So we decided to roll out the Microsoft System Center suite of infrastructure management solutions to introduce proactive monitoring of business-critical systems for the first time. The prior existence of a Microsoft Enterprise Agreement for Volume Licensing with Premier Support was a supportive argument in favor of Microsoft.
Our Microsoft consultants provided valuable technical support for our operational team during the deployment. We are close to complete designing, operating and maintaining a program for business continuity. Our methodology focuses on getting how to continue critical business activities with minimal resources. And also we are about to sing Microsoft System Center 2012 Private Cloud Technical Adaption Program Agreement. This will provide us passing the private cloud implementation smoothly.
We are architecting and building system that follows industry best practices, standards and guidelines, and processes to deliver infrastructure solutions that meet the requirements of the business which include high availability, high performance and secure systems based on ISO 27001, BS 25999, ITIL, and MOF framework.
Microsoft System Center Configuration Manager (SCCM)
By using SCCM, we have started to deploy and update servers, desktop PCs, and laptop computers. Configuration Manager provides us visibility into which hardware and software assets we have, who is using them, and where they are. We also have simplified the management and delivery of software updates, and it helps ensure that our IT systems comply with desired configuration states and power management policies.
Microsoft System Center Operations Manager (SCOM)
We had been using SCOM 2007 R2 and then we have started to use SCOM 2012 RC when it released to perform our end-to-end service-management needs. With Operations Manager, we have a single, comprehensive view into state, health, and performance data across our network. Automated alerts are generated when potential issues related to availability, performance, configuration, or security are identified, enabling IT administrators to address them before they turn into major problems that can affect service levels.
AVICode
We had been received lots of ticket about performance issue from Tier1. Our system was a under considerable burden. Our number of registered users is about 300.000, we have 1000 database transactions per second, daily about 2 million requests are responded by our system, and daily page view is about 100.000.
As you know performance issues play a major role especially in the complex systems. Bottlenecks can be caused by many components includes network, application layer, system hardware’s or software’s. So we had to do something. Actually we had a tool that comes with SCOM which is AVIcode 5.7 version. It monitors .NET applications and provide real-time health alerts when those applications code failures or performance bottlenecks. After implemented, it provided us a high-level view of application behavior and health status, including an overall application quality assessment. In addition, AVIcode provides a dynamic baseline and real-time monitoring automatically generates a trend analysis of application performance data and correlates it to CPU, I/O and memory utilization.
Thus, we identified easily application failures and performance issues as well as hotspots where problem resolution resources would be best focused. Also we have empowered to make better-informed decisions regarding server consolidation and virtualization. And we optimized the application environment and prioritize troubleshooting efforts. We started to monitor top performance issues, failures, connectivity and security problems affecting applications and we took measures to prevent bottlenecks enabling faster resolution. We were able to access into the data in order to better understand and more quickly resolve resource utilization problems along with their root causes.
By using AVIcode, we are able to determine whether our application meets its performance objectives and identify bottlenecks. By leveraging performance metrics collected with counters we are able to see response time, throughput, and resource utilization how much CPU, memory, disk I/O, and network bandwidth our application consumes while performing its tasks. So we realized that our long SQL calls, uploaded and downloaded file’s I/O times, long internal execution times resulted from complex code and remoting Web Services calls caused performance issues. And we immediately fixed it. And then thanks to AVIcode we observed a remarkable improvement of our system performance. Finally our system became a continuously improves itself.
Microsoft System Center Service Manager (SCSM)
We are using SCSM 2010 to meet for our help-desk need. With Service Manager, we have powerful, adaptable tools for problem, asset, and change management. Integrated with Configuration Manager, Operations Manager, and Active Directory Domain Services we have the ability for service-desk personnel to access information across the our infrastructure to work more efficiently and solve problems more quickly.
System Center Service Manager is now our primary ticketing system for all functions within the Technical Services department, including desktop support, server support, and network support. Even though none of our technicians had used Service Manager before, we were able to master it in just a few months.
Users can submit service-desk requests in several ways, including by phone, by email, or through the built-in web portal in Service Manager. Instead, users can submit a request through email or the Service Manager web portal.
The way that Service Manager integrates with Operations Manager and Configuration Manager is also streamlining IT operations. Technicians using Service Manager have access to all the information in Configuration Manager, allowing them to get a quick and accurate picture of the entire user environment. They can take over a user’s machine remotely and can just as easily use Configuration Manager to remotely install software or reimage the user’s PC.
No significant articles today.
<Return to section navigation list>
Cloud Security and Governance
David Linthicum (@DavidLinthicum) asserted “The Stop Online Piracy Act could oblige cloud providers to harm their own customers” in a deck for his How SOPA threatens the move to the cloud article of 1/4/2012 for InfoWorld’s Cloud Computing blog:
The Stop Online Piracy Act (SOPA), introduced in the U.S. House of Representatives in late October, would allow the Department of Justice and copyright holders to seek court orders to block payment processors and online advertising networks from doing business with foreign sites accused of infringing copyright.
If this thing passes, we could see court orders that bar search engines from linking to the allegedly infringing sites. Or most interesting, it would require domain name registrars to stop resolving queries that direct traffic to those sites — and even require Internet service providers to block subscriber access to sites accused of infringing.
As you may expect, nobody likes this act due to the potential for abuse. Indeed, in the last 30 days we’ve seen the Internet in an uproar. This includes a movement to boycott Go Daddy, which has now changed its tune on SOPA from supporting to not supporting.
Although few cloud providers have chimed in on this controversial issue, we did hear from Lanham Napier, CEO of Rackspace:
SOPA would require that Rackspace and other Internet service providers censor their customers with little in the way of due process, trumping the protections present in the current Digital Millennium Copyright Act. What’s more, the SOPA bill would seriously disrupt the Domain Name Service that is crucial to the smooth operation of the Web.
The bottom line is that this legislation sounds like a good idea for those who make a living by providing copyrighted content. However, giving government the power to pull domains and block access could lead to instances where the innocent are caught up in a legal mess they can’t afford to fight — without due process.
On this issue, cloud providers such as Rackspace, Amazon.com, and Microsoft are between the proverbial rock and a hard place. They’ll be forced to carry out these actions. But as a result — at a time when businesses are moving data and content to the cloud — SOPA would provide a reason not to trust cloud computing providers for fear one day the providers would be legally obliged to turn off the users’ business, without a day in court.
<Return to section navigation list>
Cloud Computing Events
Himanshu Singh asked At CodeMash Next Week? Don’t Miss The Training On Social Games using Windows Azure and HTML5 in a 1/4/2011 post to the Windows Azure blog:
If you plan to attend CodeMash next week in Sandusky, Ohio, don’t miss the full-day pre-compiler session, Building Social Games using HTML5 with Windows 8, Windows Azure, and .NET, which will cover the development of social games and feature Windows Azure.
In the morning, Nathan Totten along with members of the Windows Azure engineering and evangelism teams will be delivering short presentations on how to get started with building social games using Windows Azure, Windows 8 and leveraging .NET, Node.JS and HTML5. You’ll hear from experts Nathan Totten, Nick Harris, Jennifer Marsman, Wade Wegner, Brian Prince, and Glenn Block.
In the afternoon, we’ll let you get hands-on with these technologies building your own social game while having experts in the room to help you make your ideas real. We’ll have resources available, including a walkthrough of building your first social game with Windows Azure.
If you plan to join us for the hands-on portion, it will be helpful to have the prerequisite software listed below installed on your laptop. Happy coding!
Prerequisite
Windows Azure Camps Kit: The Windows Azure Camps Training Kit uses the new Content Installer to install all prerequisites, hands-on labs and presentations that are used for the Windows Azure Camp events.
Optional
If you’d like to work on any of the following scenarios, you will want to download the appropriate Toolkit ahead of time.
- Windows Azure Toolkit for Social Games: The Windows Azure Toolkit for Social Games is a set of guidance, samples, and tools that helps developers quickly get started building a casual or social game on Windows Azure.
- Windows Azure Toolkit for Windows Phone: The Windows Azure Toolkit for Windows Phone is designed to make it easier for you to build mobile applications that leverage cloud services running in Windows Azure.
- Windows Azure Toolkit for Windows 8: This toolkit includes Visual Studio project templates for a sample Metro style app and a Windows Azure cloud project.
Manual Install
If you’d like to manually install the tools and SDKs, you can use the following links:
- Visual Studio 2010 Express (or higher)
- Windows Azure SDK for .NET – November 2011 (or higher)
- IIS: Tracing
- SQL Server 2008 R2 Management Studio Express with SP1
Click here to learn more about this session.
<Return to section navigation list>
Other Cloud Computing Platforms and Services
Matty Noble described How Collections Work in the AWS SDK for Ruby in a 1/4/2011 post to the AWS blog:
Today we have a guest blog post from Matty Noble, Software Development Engineer, SDKs and Tools Team.
- rodica
We’ve seen a few questions lately about how to work with collections of resources in the SDK for Ruby, so I’d like to take a moment to explain some of the common patterns and how to use them. There are many different kinds of collections in the SDK. To keep thing simple, I’ll focus on Amazon EC2, but most of what you’ll see here applies to other service interfaces as well.
Before we do anything else, let’s start up an IRB session and configure a service interface to talk to EC2:
$ irb -r rubygems -r aws-sdk > ec2 = AWS::EC2.new(:access_key_id => "KEY", :secret_access_key => "SECRET")There are quite a few collections available to us in EC2, but one of the first things we need to do in any EC2 application is to find a machine image (AMI) that we can use to start instances. We can manage the images available to us using the images collection:
> ec2.images => <AWS::EC2::ImageCollection>When you call this method, you’ll notice that it returns very quickly; the SDK for Ruby lazy-loads all of its collections, so just getting the collection doesn’t do any work. This is good, because often you don’t want to fetch the entire collection. For example, if you know the ID of the AMI you want, you can reference it directly like this:
> image = ec2.images["ami-310bcb58"] => <AWS::EC2::Image id:ami-310bcb58>Again, this returns very quickly. We’ve told the SDK that we want ami-310bcb58, but we haven’t said anything about what we want to do with it. Let’s get the description:
> image.description => "Amazon Linux AMI i386 EBS"This takes a little longer, and if you have logging enabled you’ll see a message like this:
[AWS EC2 200 0.411906] describe_images(:image_ids=>["ami-310bcb58"])Now that we’ve said we want the description of this AMI, the SDK will ask EC2 for just the information we need. The SDK doesn’t cache this information, so if we do the same thing again, the SDK will make another request. This might not seem very useful at first — but by not caching, the SDK allows you to do things like polling for state changes very easily. For example, if we want to wait until an instance is no longer pending, we can do this:
> sleep 1 until ec2.instances["i-123"].status != :pendingThe [] method is useful for getting information about one resource, but what if we want information about multiple resources? Again, let’s look at EC2 images as an example. Let’s start by counting the images available to us:
> ec2.images.to_a.size [AWS EC2 200 29.406704] describe_images() => 7677The to_a method gives us an array containing all of the images. Now, let’s try to get some information about these images. All collections include Enumerable, so we can use standard methods like map or inject. Let’s try to get all the image descriptions using map:
> ec2.images.map(&:description)This takes a very long time. Why? As we saw earlier, the SDK doesn’t cache anything by default, so it has to make one request to get the list of all images, and then one request for each returned image (in sequence) to get the description. That’s a lot of round trips — and it’s mostly wasted effort, because EC2 provides all the information we need in the response to the first call (the one that lists all the images). The SDK doesn’t know what to do with that data, so the information is lost and has to be re-fetched image by image. We can get the descriptions much more efficiently like this:
> AWS.memoize { ec2.images.map(&:description) }AWS.memoize tells the SDK to hold on to all the information it gets from the service in the scope of the block. So when it gets the list of images along with their descriptions (and other information) it puts all that data into a thread-local cache. When we call Image#description on each item in the array, the SDK knows that the data might already be cached (because of the memoize block) so it checks the cache before fetching any information from the service.
We’ve just scratched the surface of what you can do with collections in the AWS SDK for Ruby. In addition to the basic patterns above, many of our APIs allow for more sophisticated filtering and pagination options. For more information about these APIs, you can take a look at the extensive API reference documentation for the SDK. Also don’t hesitate to ask questions or leave feedback in our Ruby development forum.
A note about AWS.memoize
AWS.memoize works with both EC2, IAM and ELB; we’d like to extend it to other services, and we’d also like to hear what you think about it. Is the behavior easy to understand? Does it work well in practice? Where would this feature be most beneficial to your application?
Jeff Barr (@jeffbarr) announced Help Wanted – Manager and Senior Developers for new AWS Media Product (for Amazon Instant Videos?) on 1/4/2012:
We are staffing up a brand-new AWS team to take advantage of some really interesting opportunities in the digital media space.
This team is being launched from an existing product. This particular product has seen exponential growth in the size of its user base on an annualized (run rate) basis, along with 30x revenue growth over the same period.
In order to address this opportunity, we need to hire a Senior Development Manager and multiple Senior Developers ASAP. You’ll be able to start from scratch, building a large-scale distributed application as part of the Seattle-based team.
The team is currently searching for developers at our "SDE III" level. Successful applicants for an SDE III position typically have six or more years of development experience in the industry, along with a BS or MS in Computer Science. They are able to solve large problems in the face of ambiguity, and are able to work on the architecture and the code. They will also have launched projects of significant complexity in the recent past, and have the ability to balance technical complexity with business value.
The developers on this team will drive the architecture and the technology choices. They’ll need to have a broad knowledge of emerging technologies and will know the ins and outs of Java, C or C++, and Linux or Windows. They will also have significant experience with networking, multi-threaded applications, interprocess communication, and the architecture of fault-tolerant systems.
We also need a senior manager of software development to build and run this team of top-performers. The manager will establish a project framework and will also be responsible for putting the right development practices in to place. The manager will also be responsible for providing technical leadership and guidance to the team. Well-qualified applicants will have been managing teams of developers for four or more years, and will have shipped one or more highly available large-scale internet applications.
If you are qualified for one of these positions and you would like to apply, please send your resume to newawsproject@amazon.com.
<Return to section navigation list>
Technorati Tags: Windows Azure, Windows Azure Platform, Azure Services Platform, Azure Storage Services, Azure Table Services, Azure Blob Services, Azure Drive Services, Azure Queue Services, Azure Service Broker, Azure Access Services, SQL Azure Database, Open Data Protocol, OData, Cloud Computing, Visual Studio LightSwitch, LightSwitch, Amazon Web Services, AWS, Ruby, Progress Data Direct, Oracle, Sybase, DB2, CodeMash, Delphi Automotive, Apache Hadoop, Hadoop on Azure
http://oakleafblog.blogspot.com/2012/01/windows-azure-and-cloud-computing-posts_04.html







































































