Serverless Computing on Cloud Platforms – The Game Changer

Amazon’s Lambda and Google Cloud Functions are ushering in the next generation computational model, which in my opinion will have a profound impact on how applications will be developed, operated and maintained. As the capabilities of Lambda and Cloud Functions improve there will be less incentive for developing applications based on virtual machines and containers. We are closing in on the iOS application model on Cloud platforms.

IoT Platform Comparison – Amazon Web Services vs. Google Cloud Platform vs. Microsoft Azure

This post is a follow on to the “What I Would Want In An IoT Platform” post and examines IoT (Internet of Things) platform offerings from the 3 leading Cloud Computing platforms.

As highlighted in the original post an end-to-end Internet of Things (IoT) platform needs to provide more than just data collection and data analysis capabilities. The following outlines the list of capabilities desired in an IoT platform.

  • SDK that can be used as part of firmware in IoT enabled devices
  • Device and related firmware/software management
  • Potential for plug and play global connectivity – GPRS/3G/4G/LTE
  • Support for standard protocols
  • Bi-directional communication for data transfer and control
  • Security
  • Processing of streaming data
  • Time-series data management
  • Archiving and large scale data storage
  • Data analysis
  • Workflow

The following picture provides a simplified view and potential architecture of a connected product/service, a typical scenario for IoT.

iot-architecture

Now let’s look at how each of the Cloud Computing platforms do with respect to the above list of desired capabilities.

SDK that can be used as part of firmware in IoT enabled devices

  • Amazon Web Services offers a compelling device SDK that is clearly well adopted by a number of chipset manufacturers based on the listings in Amazon marketplace.
  • Google Brillo is a great starting point for IoT development and operations. It would be a lot better if it came with in-built support for supporting backend IoT data management and analytics services (Big Table, Big Query) like Amazon’s SDK does. But it may only take small amount of code to accomplish that.
  • Windows 10 for IoT is part of the Universal Windows environment and should be an obvious candidate for Windows developers.

Device and related firmware/software management

IoT-centric design offers a great opportunity to continuously improve products and services with over-the-air (or via any network) software updates. This is a much desired capability if the platform and related SDKs can offer the capability.

  • Amazon Web Services offers a continuous delivery and integration service with Code Pipeline.
  • Google Brillo seems to offer core capabilities for managing device firmware and updates.
  • Similarly Windows Update Service support is now added to Windows IoT core

In general, all IoT platforms require a sophisticate continuous delivery and integration and update service to manage millions of devices and related firmware/software in the field. The Big 3 have significant experience managing millions of devices and related software with their Kindle/Echo, Nexus/Android and Windows/XBox businesses. You should be confident that they do the job for you too rather easily.

Potential for plug and play global connectivity (GPRS/3G/LTE…)

This is usually something a telecommunications service provider offers to customers. However, the Big 3 of Cloud Computing could make the on-ramp for IoT easier by creating some sort of an MVNO. In fact, Google is well on its way with Fi.

Support for Standard Protocols

It can seem like there are as many IoT protocols as there as IoT consortiums. There is AllSeen, Weave from Google, MQTT and so on. The device SDK teams are most likely waiting to see which ones of these protocols will become the dominant one. MQTT is the standard that Amazon supports and MQTT client for Windows IoT can be found in the open source community.

Security

Security should be paramount in all IoT design and platform discussions. Every layer of the IoT platform should provide exceptional support for security.

  • Brillo seems to offer strong security at the hardware and software level
  • Windows IoT has been making strong strides when it comes to security
  • Amazon IoT SDK and supporting IoT connection services offer a strong security model based on certificates, authentication and authorizations

Stream Data Processing

One of the most common things IoT devices do is to generate and in many cases send a torrent of data to Cloud services. It is ideal if the Cloud Platform offers a stream data processing as a service that can be made part of the overall IoT information architecture

Time-series Data Management

You would be surprised how often you need to store and analyze time-series data in an IoT application. From metering to monitoring performance, time-series is every where. Capabilities offered by solutions like KairosDB,  which is built on Cassandra, are what one could use as core capabilities of IoT platform. Unfortunately, none of the Big 3 cloud vendors offer it yet. Of course, you could easily develop time-series data management capabilities on top of BigTable, Dynamo or SQLServer. But it would be a lot more convenient if time-series data management is a core capabilities of IoT platforms.

Archiving and Large Scale Data Storage

In large scale IoT scenarios before you know it you can find yourself knee deep in data that you may or may not need. First and foremost, the IoT platform should be able to support managing large amount (often in terabytes or more) of measurement and event data cost-effectively. Secondly, the IoT platform should provide and easy way to archive data in various tiers since historical data does not always require low latency storage and your organization may need to keep historical data for regulatory reasons.

Data Analysis

Now that you have managed to set up your IoT service on one of the IoT platforms from the Big 3 you will find a need to conduct simple to sophisticated analysis. The Big 3 provide a rich suite of analysis capabilities that range from simple data warehousing to sophisticated machine leaning.

Overall, if you are looking to set up a moderate to large scale IoT infrastructure you will go wrong with any of the IoT platforms from the Big 3. Amazon’s IoT Stack and Microsoft Azure’s IoT Suite look especially attractive given the breadth and depth of coverage of IoT stack.

What I Would Want In An IoT Platform

Despite some overzealous efforts related Internet of Things (IoT) the trend has legs and real business value. Internet of Things is one of the foundational elements of Digital Transformation. Advances in IoT are transforming products into connected experiences and services.

An End-to-End Internet of Things (IoT) platform, however, needs to provide more than just data collection and data analysis capabilities. Based on my experience with a real world (cost constrained, patchy connectivity, low power) IoT scenario I found the need for following capabilities in an end-to-end IoT (software only) platform. Hardware aspects are omitted in this post because of the diversity of use cases and design considerations that cannot easily be generalized.

  • SDK that can be used as part of firmware in IoT enabled devices
  • Ability to update device firmware (OTA/network)
  • Configuration management (for firmware) for operational devices
  • Device management oriented operations center
  • Potential for plug and play global connectivity – GPRS/3G/4G/LTE
  • Support for standard protocols
  • Bi-directional communication for data transfer and control
  • Security
  • Processing of streaming data
  • Time-series data management
  • Archiving and large scale data storage
  • Data analysis
  • Workflow

The following picture provides a simplified view and potential architecture of a connected product/service, a typical scenario for IoT.

iot-architecture

In a later post we will look at desired capabilities in detail and IoT platforms from the Big 3 of Cloud Computing.

Architecting An Enterprise Document Management System With Google Drive And Google Object Storage

There is no doubt that Google Drive is a great document management and collaboration system for individuals and teams. Google Drive does start to show its limitations when you begin to use it as a document management system for the whole organization.

Organizations use an enterprise document management (like Documentum, Alfresco…) to support key business processes in marketing, sales, product development, manufacturing, quality management, customer service, etc. In addition, proper document management is necessary to meet regulatory requirements in many industries. Some of the expected capabilities of an enterprise document management include the following

  • Storage capability for terabytes of data and millions of documents
  • Sophisticated access control
  • Audit trail
  • Workflow
  • Archiving
  • Collaboration (internal and external)
  • Revision control
  • Search, Full text indexing, etc.

While Drive has some of the above capabilities, some of the limitations can make it difficult to use it as the enterprise document management system. This is where Google Cloud Storage comes into the picture. You can learn more about Google Cloud Storage here.

The figure below shows the high-level architecture of how to leverage Drive and Google Cloud Storage to create your enterprise document storage management system.

google-drive-cloud-storage-01

In the example above, users (Homer Simpson and Lisa Simpson) use Google Drive for local storage and collaboration and Google Cloud Storage as the central enterprise wide data storage.

However, certain amount of integration and application code needs to be written between Google Drive and Google Cloud Storage to complete the enterprise document management system. We will cover the integration and application code in a later post.

In summary, the combination of Google Drive and Google Cloud Storage creates a compelling enterprise document management solution for organizations of all sizes that users will love to use and CIOs approve.

Amazon’s NICE move

Nice move, Amazon. High Performance Computing (Simulation, Computer Aided Engineering (CAE), Graphics rendering, etc.) has been one of the earliest use cases and for Cloud Computing. This was true even before the advent of AWS and the likes. The computers and infrastructure that were traditionally used for these jobs were Super Computers from Department of Energy, Super Computing Centers and/or clusters of computers. Move over, DoE. We have a new super computing cluster in the town.

Why Aren’t Cloud Computing Platform App Stores Like Apple iOS App Store

We can confidently say that the Apple iPhone would not be the business juggernaut that it is without the App Store. Apple App Store generated close to 20 Billion in revenue in 2015 and by itself would be among the top 150 of Fortune 500. Apple says that it has returned close to 40 Billion dollars to developers since 2008. It pays to be a top developer in the App Store ecosystem. This is no small part due of the extremely fine-tuned distribution and monetization model of App Store and the supporting iOS architecture.

In contrast the Cloud Computing App Marketplaces of Amazon Web Services, Google Cloud Platform and Microsoft Azure are made up of primarily Virtual Machine Images or non-platform solutions that are integrated with the platform. Contrast this to typical application architecture of an iOS App which is developed with iOS APIs and runs within the sandbox provided by iOS. Apple’s review and approval process ensures third party applications are safe and can be trusted by users. In addition, iOS App Store provides one of the most frictionless process of distribution with integrated payment and one-click download. These 3 integrated elements are what make the iOS platform one of the most powerful platforms, and certainly the most profitable, in the world.

None of the 3 cloud computing platforms provide all of the above 3 integrated elements. Amazon Web Services provides some of the elements (integrated billing and one-click installs) with AMIs (Amazon Machine Images), which can be instantiated. But instantiating VM images is only a marginal improvement over the current situation. This approach does not allow developers to take advantage of all the services/products that cloud computing platforms provide.

The ideal scenario would be to have a uniform API that exposes different underlying services (Object Storage, SQL, NoSQL, etc.) that application developers can use to write applications and a scalable sandbox that can run these applications. A good review and approval process would also make it easier for customers to trust third party applications. Finally integrated billing and one click app download eliminates friction in the buying process. A try and buy approach would even take the risk factor out of the purchase process.

Casualties of Cloud Computing

Ever since Amazon first rented its computers by the hour in 2006 the rise of public cloud computing has been relentless and exponential. The efficiency and effectiveness of public cloud infrastructure has laid bare the extreme inefficiency and bad quality of legacy IT ecosystem. The public cloud computing companies are almost providing 10X value to end customers. They are effectively transforming a Trillion dollar ecosystem into a 100 Billion market and will likely capture majority of the profits. As a result the casualties are piling up at every level and layer of the IT ecosystem. The list follows

Broad Computing

  • HP
  • IBM
  • Sun
  • Dell
  • Oracle
  • VMWare

Networking

  • Cisco
  • Juniper
  • Lucent

Storage

  • EMC
  • NetApp

Consulting and Systems Integration

  • EDS

Cloud Platform for Independent Software Vendors (ISVs) – Amazon Web Services vs. Microsoft Azure vs. Google Cloud Platform vs. Salesforce – Details

This is a continuation of this post on evaluating cloud platforms for independent software vendors. In that post we identified the following as the most important criteria for selecting a Cloud Platform for ISVs.

Development Focused

  • Programming language support
  • Application servers (and containers)
  • Database services (SQL)
  • NoSQL database services
  • Object/File storage services
  • Analytics services
  • Multi-tenancy

Operations Focused

  • High availability support
  • Data centers in multiple geographies
  • Scalability

Distribution and Monetization Focused

  • Support for application marketplace
  • Integration with platform user management and single sign-on
  • Easy application installation
  • Ability to leverage platform billing

The following provides details on how each Cloud Platform performs against listed criteria.

Programming Language Support

Developers like diversity. More the better when it comes to programming language support. And standards based support for popular languages is a MUST. Developers reluctantly adopt a non-standard platform specific language and only when there is a compelling reason.

Amazon Web Services Google Cloud Platform Microsoft Azure Salesforce
Programming Language Support Excellent Excellent Very Good OK
  • Amazon clearly is the champion here with support for Java, PHP, Ruby, Javascript, iOS, Android, Python and .NET.
  • Google is not too far behind with good support for Java, PHP, Python and Go
  • Microsoft Azure is generally considered a .NET cloud, but the new App Service is certainly getting more open for other languages and runtime environment. And the development tools are outstanding.
  • Salesforce development is primarily done with Apex, which works very well with Salesforce platform. Non-standard, but well integrated with their platform.

Application Servers and Containers

The ideal scenario is that the cloud platform provides a scalable application server environment that developers can use to develop, test and run their applications easily. And this application server environment via use of standard languages and SDKs can use other platform services to develop and run a full-fledged software application. Kind of like the iOS + iPhone/iPad symbiotic relationship, only for the Cloud platform.

Amazon Web Services Google Cloud Platform Microsoft Azure Salesforce
Application Server(s) Excellent Excellent Very Good OK
  • Amazon with Elastic Beanstalk is clearly the champion here with support for Java, PHP, Ruby, .NET, Python, Go and .NET.
  • Google is not too far behind with Google AppEngine. In many ways the AppEngine type of computation service is ideal because of its support for standards oriented development model (JPA..) and tight integration with many supporting services, including BigTable among others.
  • Microsoft Azure’s App Service has also made a lot of progress in a very short time. And Microsoft’s development tools (Visual Studio) have always been the best in the business.
  • Salesforce is the oddball here because of multi-tenant platform architecture and use of APEX. The application server (that runs apps) is barely visible and built into the Salesforce platform itself. The app server and the platform promotes Salesforce-ish development model with tight integration to the underlying Salesforce data model and services. It could be reasonable platform if the ISV application were an extension of Salesforce processes, especially around CRM. But the platform may be too limited and too expensive for a generic ISV application.

SQL Database Services

Every ISV application more or less needs an SQL Database.

Amazon Web Services Google Cloud Platform Microsoft Azure Salesforce
SQL Database Services Excellent Very good Excellent N/A
  • Amazon – RDS, Oracle, SQL Server, Aurora.. You name it, they have it.
  • Google’s Cloud SQL is a welcome addition to the platform and well integrated with the AppEngine environment
  • Microsoft Azure has one of the best SQL databases in SQL Server.
  • Salesforce does not provide a real database that you can pull, push and get the most out as part of your application. It has a concept of object, master-detail relationships, etc. But that is not a real database.

NoSQL Database Services

NoSQL databases are here to stay. For a variety of use cases and data types (events, etc.) NoSQL is a better option than SQL.

Amazon Web Services Google Cloud Platform Microsoft Azure Salesforce
NoSQL Database Service Excellent Excellent excellent N/A
  • Amazon’s DynamoDB is the granddaddy and started the whole NoSQL movement
  • Google’s BigTable is the behemoth of NoSQL. It is also well integrated with Google AppEngine and its object model.
  • Microsoft is new to this with DocumentDB as part of Azure
  • You are out of luck with Salesforce

Object Storage Services

For many applications that deal with multimedia data file/object storage capabilities are a key and primary requirement. Cost-effective storage and access is also just as important.

Amazon Web Services Google Cloud Platform Microsoft Azure Salesforce
Object Storage Service Excellent Very Good Very Good N/A
  • Amazon’s S3 is the most mature object storage offering in the market. You can’t go wrong with it.
  • Google’s Cloud Storage service is also compelling.
  • Microsoft’s awkwardly named Blob Storage does the job too.
  • If file storage and serving is a primary requirement of your application you might want to skip Salesforce.

Analytics Services

Analytics is usually less in practice (pivot tables) than what it portrays (statistics) to be. But for the case of this comparison we will primarily look at capabilities for multi-dimensional data analysis and self-service business intelligence tools.

Amazon Web Services Google Cloud Platform Microsoft Azure Salesforce
Analytics Services Excellent Very Good Excellent OK
  • This is getting repetitive. Amazon is again one of the leaders with Redshift and newly unveiled QuickSight
  • Google’s BigQuery is pretty good. But lack of good self-service business intelligence tools makes it less compelling.
  • What’s not to like about SQL Data Warehouse and Power BI
  • Salesforce’s Wave Analytics is the most recent addition to the platform. But is seems most useful for reporting and analytics related data in Salesforce. It is less attractive for general purpose analytics

 

Multi-tenancy

If you are an ISV I suspect you want your application to be used by more than one customer. Multi-tenancy consideration (or not) then becomes a strategic aspect of how you support multiple customers. Multi-tenancy is also an important aspect of SaaS product development, architecture, operations and security.

Depending on the cloud platform that you select it may or may not be important for you to consider from a development and architecture standpoint. For example, if you are developing applications for Salesforce platform there is no need for you to worry about Multi-tenancy because the platform is itself multi-tenant. Each customer of Salesforce operate in their own tenant space and any third party ISV application when adopted by customers is tightly bound to the tenant space and its objects. Google AppEngine supports the concept of multi-tenancy, but then your application has to explicitly take into account. AWS and Microsoft Azure do not support the notion of multi-tenancy and it is up to you to architect and operate the multi-tenant version, which can be quite complicated from a development perspective, of your application. Of course, you can use VMs, containers and host of other approaches to make multi-tenancy possible and with ease these days. In the end and for this shootout we like the Salesforce’s approach (you will see why once you finish reading the whole post) the best, although the platform is only appropriate for simple applications.

Amazon Web Services Google Cloud Platform Microsoft Azure Salesforce
Multi-tenancy N/A Very Good N/A Excellent

 

High Availability

High availability of the platform and its core services (like SQL, Object Storage, etc.) are fundamental to how you can achieve high availability for your own ISV application. All of the platforms have proven to be excellent.

Amazon Web Services Google Cloud Platform Microsoft Azure Salesforce
High Availability Excellent Excellent Excellent Excellent

 

Data Centers in Multiple Regions

The Kerfuffle around Safe Harbor and regulations in various countries make it necessary that certain types of data be resident in the region or country. The makes it necessary that the cloud platform have data centers in as many regions/countries as possible.

Amazon Web Services Google Cloud Platform Microsoft Azure Salesforce
Data Centers in Multiple Regions Excellent Good Very Good OK

 

Scalability

For some applications scalability is a major concern. Imagine you were developing a Netflix or a Dropbox, you would expect the Cloud Platform to offer highly scalable foundational services (Object storage, NoSQL service, etc.).

Amazon Web Services Google Cloud Platform Microsoft Azure Salesforce
Scalability Excellent Excellent Excellent N/A

 

The Big 3 of Cloud Computing

We should wait before anointing the Big 3 of Cloud Computing until they collectively command a majority share of the cloud market. But all indications point to Amazon Web Services, Microsoft Azure and Google becoming the Big 3 of Cloud Computing. The common thread here is the very large consumer oriented operations they all run.