Cloud Layers – The network is our computer

We are comfortable and familiar with personal computing – how the device on our desk or lap offers applications (e.g. word-processing with Word); communications (e.g. email or instant messaging), storage (using the computer hard drive) and processing (using of course the computer’s CPU).

We are less comfortable and familiar with the network as our computer – how the Internet offers applications (e.g. Google Docs); communication (e.g. webmail, twitter), storage (e.g. skydrive or Amazon Simple Storage Service ) and processing (e.g. Amazon Elastic Compute Cloud ).  Check the Web 2 search engine for a massive list of web applications. Check csteinberg.com for an example  list of web applications that could be used in Education.

Like a personal computer the network as a computer can be regarded as set of layers – O’Reilly recently suggested three layers.

Cloud Layer 1:

As an equivalent to the physical layer he identifies “utility computing” – those offering the “lower” level foundations of the “machine” – those offering storage, computation and virtual machines – these are like Nick Carr’s “Big Switch” power stations – Microsoft’s cloud data centre even looks like a power station to me.  

Cloud Layer 2:

In the second layer O’Reilly describes “Platform Services” where you can write and run applications e.g. Google App engine, and Salesforce’s force.com . Indeed, recently we had the news of the collaboration between Amazon and Force that you could run your application at Force and use Amazon for storage and database. Yahoo Pipes  was an early user accessible service in this layer. Pipes allowed users to connect various data sources together and carry out limited programming functions on them. Yahoo further recognise the potential of this layer with their announcement of the Yahoo Operating System (Y!OS) concept to “rewire” yahoo, open it up for developers to access Yahoo programmatically.  Since then Yahoo have been announcing an exciting set of Y!OS APIs . There is the Yahoo Application Platform (YAP), The Yahoo Query Language (YQL) and the Yahoo Social Platform (YSP).  Microsoft seem to be focusing on the provision of virtual services for their server products such as Exchange, Sharepoint and SQL server in what I would regard as the layer 1 utility computing layer. However, Microsoft have a limited equivalent to pipes called Popfly that is very quick and easy to use – this flickr x live maps MASH took me less than 3 minutes to make.   

Cloud Layer 3:

In the top layer O’Reilly describes “Cloud-based end-user applications” of the sort many are familiar with – Google docs, Youtube, Flickr, webmail etc.

There is another layer within the cloud which we are even less familiar with one of the fastest growing areas with the biggest potential – because it is invisible. This is the area where programs talk to each other – the Application Programming Interface (API). We are familiar with the web pages of web sites but increasingly these web sites offer a variety of ways for other sites and programs to access them.

One of the easiest ways for people to become familiar with APIs and to appreciate their potential is with the use of RSS feeds – check this video for an explanation of RSS in “plain English”.  Instead of visiting a site to check for changes you can use a feed reader (e.g. netvibes, pageflakes, google reader etc) to interface with the site’s RSS feed interface. The site’s RSS feed presents site changes in chronological order and your feed reader will display these. Feed readers also allow you access and display site feeds from several sites so that you only have to access one site to view the changes from several sites (feed aggregation) – you no longer have to visit the sites themselves. You can see an example by checking my netvibes page.

In November 2008 the API tracking site the programmable web reached a much heralded milestone when it added the 1,000^th API to its API directory. However, it isn’t only the quantity that is noteworthy but also the quality, variety, rate of growth, and most of all, impact that APIs are having. Most (if not all) of the major sites have APIs Google’s maps API is probably the most well known API. Flickr, Youtube and Twitter have very popular and well used APIs. Combining APIs from different sites is called MASHUP. This site presents a MASHUP of BBC news x Google maps, Gtraffic presents a MASHUP of BBC travel news and Google maps. Earth Album presents a MASHUP of Flickr and Google maps. Twittervision is a MASHUP of Twitter and Googlemaps.  

The examples above are just the tip of the iceberg and like an iceberg much of the internet is invisible. Back in 2001 eBay presented one of the first APIs – this now takes over 6 billion API calls per month and accounts for 60% of eBay listings. Twitter has a rich and highly exploited API which passes 10 times the traffic of the visible Twitter site.

It is difficult to persuade development funds into invisible applications and interfaces but many are now seeing the cost advantages in developing APIs instead of the more time consuming, complex and expensive visible sites. We are seeing the development of what could be described as the “invisible web” – services on which the visible web “ecosystem” feeds.  To mix metaphors, Cloud APIs could represent the larger base upon which the visible tip of the iceberg rests.

Recommended reading

Dion Hinchcliff on Web APIs

Dion Hinchcliff on MASHUPs as the new development platform

Tim O’Reilly on Cloud computing – layers and network effects

Cloud Forecast

In the spirit of “I never make predictions and I never will”   This blog looks at some of the possible consequences of Cloud computing.

Part 1 – Summary of some of the factors affected by Cloud Computing

Flattening the world

Cloud computing throws all the issues of traditional IT up in the air and creates a level playing field. Small companies can make use of the same resources as the largest Global enterprise – on a “pay as you go” or even free model the only difference is in scale of use rather than installation. The cost of entry is significantly reduced. Small companies get access to enterprise level technology and reduce capital expenditure. This could also be a big benefit to people developing countries who could in theory run a large enterprise class IT from a computer with a broadband connection – assuming they can get access to a broadband connection – Google is helping with this however.

Agility

Cloud computing and IT as a service reduces development and implementation time and costs and facilitates fast delivery.  It is no longer necessary to wait for delivery, install, configure, maintain, support and update traditional on-site services instead you can point a browser at a supplier and configure your service – it is a lot, lot faster. In theory at least, cloud services can be changed, expanded and shrunk as required and new services created by “stitching” together existing and new services.

Innovation

Innovation often takes you down a dead end and the cost in time and money often put people off from innovation – it is safer to follow known working models than to risk early advantage. Cloud computing and IT as a service reduces the resources used to innovate so that the consequences of failure are smaller.

Collaboration

Where once the Internet was used as a communications channel between “islands” of business, systems and people it can now be used to host systems and business. Collaboration and interoperation through firewalled systems is a problem – interoperation is a lot easier and natural in the cloud environment.

Location

Traditional IT has been centred on physical location and protected from the internet by firewalls. Such systems are well suited to tradition where people travel to work at organisational premises. Providing flexible external access under these conditions has been difficult – usually by allowing increasing numbers of staff external access to internal resources by use of a VPN which goes through the firewall which protects the internal systems from the Internet.  Physical location loses meaning when you use cloud computing – the services are on the Internet – it doesn’t matter where you access them from.

Consumer IT

Through mass market scale and consumer interest in computing the cost of computers has dropped considerably. Where computer companies once looked at Business as customers they now look at consumers and as a result most business IT is now driven by consumer trends. Consumers want to operate their IT equipment like gadgets – turn them on and use them – they don’t want to difficult configuration and setup and to worry about security and updates. The proportion of device cost taken by software becomes increasingly difficult   hardware cost continues to fall – when a device might cost just £200 it seems difficult to then pay another £200 for software.

The Internet

Internet growth itself could be argued as an example of the network effect  as more people use it the more useful it becomes and the more people use it and the more it expands. Cloud computing depends upon highly available reliable Internet connections – people and organisations will be prioritising their Internet connections – this alone will add to growth in Internet availability

Part 2 – The predictions

As the Internet mediates more of our experience we should expect a step change as we live our lives more in Internet time.

Things will change faster (when faster is the new fast) as the cloud and Internet enable things to be done so much quicker than in traditional ways.

Competition will be keener than ever before as smaller organisations can compete effectively with larger organisations in certain sectors – especially in innovative new service delivery.

We should expect to see remarkable product and service innovations as barriers to innovation are lowered the speed and ability to innovate improved.

We should expect to see an explosion in collaboration of all kinds – between systems (Mashups), people and organisations.

Cloud based systems operate outside traditional firewalled “island” systems and we should expect to see the concept of company and organisational operation challenged.  

We should expect to see growth in flexible remote working – a “de-location” or work , life and service as it becomes less necessary to be within the premises of a company to work for that company.

We should expect cheaper and easier to use Internet access devices from traditional laptop and mini laptop style units to more consumer oriented gadget type units as well amazingly functional smartphone units – there will be more “plug, play and go” and a growth in non Microsoft software and new software types.

We should expect to pervasive Internet access. As device costs fall and Internet access improves we should expect to see all manner of devices and gadgets with Internet access and we should increasingly expect to get on-line anywhere anytime – hyperconnectivity.

The trends above interact in a positive feedback loop style so that the changes become ever faster.  If network effects are present then as more people collaborate and innovate faster then change will happen faster and faster.

Cloud Formations

This blog is my interpretation of the major service approaches developing in the cloud.

Google

Having started and developed in the cloud Google has no traditional IT base to maintain, support and protect – Google generally offer what I regard as a “pure cloud” service. Google’s offer is described as Software As a Service – the focus is on applications and interfaces rather than data files.

Google’s services are fully on-line (off-line features are being added but are limited), require only a browser and occasionally a browser plug in, they have both free (ad supported) and paid for service.  As long as you can get on-line then Google’s service is available free without local dependencies on computer, operating system or application.

Google offer a large portfolio of services including “Office” applications, web sites, blogs, email, calendar, photo storage and sharing, video storage and sharing, a virtual world and a social network.  

Below are some of the ones I find very useful

Google Docs  Work with “office” type applications – Documents, Spreadsheets, Presentations and forms. Not only are these fully functional applications but they can be shared and collaboratively edited in real time anywhere on the net.

Google sites Create your own web site and a publication point for Google docs, calendar, blogs etc.  quite limited in many ways but with good potential.

Microsoft

Having started and developed a massive traditional IT base Microsoft’s cloud service development will be fascinating to watch as it seeks to offer a cloud service while protecting its traditional base. The future of Microsoft is at stake here – if it doesn’t produce a cloud service it will lose income yet of it does produce a cloud service this will “compete” with its traditional Office “cash cow”.

Microsoft’s approach to the Cloud is different to Google’s – an approach which it describes as Software Plus Services.  Whereas Google starts and ends in the cloud Microsoft starts with locally installed applications and adds cloud services to these. In this way you still need to buy Microsoft applications and install them on your computer but you can use the cloud to compliment them.

Microsoft’s cloud offer is called Live and while these services can be operated fully on-line the major missing application is an on-line “Office” – Microsoft provide an excellent cloud storage space called Livespace to store data files but you must use a local application to work with these files. The contrast is that Google don’t provide a storage space for files – Google provide applications. Microsoft have plans for a web based “office” for sometime in 2009 but have indicated that this will provide light editing compared to the fully featured local office

Microsoft’s Live Mesh is a good example of how Microsoft are thinking about the cloud – with Live Mesh you can link you local application to the “cloud” so that you can access documents created on one machine from another machine – as long as that other machine has the application and the connection to Live Mesh. In this way Microsoft can get you to buy even more of its software as you need to have the software on each machine you want to “Mesh”!

Yahoo

Yahoo had seemed to lose its way and risked being crushed between Google and Microsoft. Yahoo had seemed to lack vision and direction but out of necessity recent announcements set Yahoo on perhaps the most interesting path of all – I just hope it isn’t too late. Yahoo must realise that they might find it difficult to offer the scale and scope of cloud services that Google and Microsoft could offer. Instead Yahoo has been announcing something that could eventually propel them beyond Google and Microsoft.

 With Pipes yahoo provided a glimpse a few years ago of what is possible. The Pipes may have informed their recent  work on interfaces and the development  of Yahoo APIs (Application Programming Interfaces) – the sort of things that let programs communicate with each other so that you can build one program by making use of other programs. The RSS feed reader is a simple example of this – an RSS feed reader takes outputs from the RSS links you specify from other websites and displays them. Web API’s are the stuff of MASH – the ability to create applications from the interfaces of other applications – to treat the web itself as an Operating system and as programmable.

Yahoo cheekily describe their approach as Browser Plus – like Gears, Silverlight or AIR, this provides  Yahoo with a browser based platform for their API vision. I am less interested in the Browser Plus project but more interested in their “back-end” cloud API announcements. Back in the Spring of 2008 at the Web 2.0 expo yahoo announced the Yahoo Operating System (Y!OS) concept to “rewire” yahoo, open it up for developers to access Yahoo programmatically.  Since then Yahoo have been announcing an exciting set of Y!OS APIs . There is the Yahoo Application Platform (YAP), The Yahoo Query Language (YQL) and the Yahoo Social Platform (YSP). Yahoo also announce support for Web standards Oauth, OpenSocial. Yahoo have an interesting strategy but they are not the only organisation looking at the “programmable web” Google for example have a highly developed developer community and code offer

 

Specialised Cloud services

In addition to the large “portal” style services of Microsoft, Google and Yahoo are the myriad of more specialised services such as those below:

Amazon Web Services (AWS)

Amazon Web Services have become something of a powerhouse in compute and storage services. Their Elastic Compute Cloud (EC2) is well regarded and offers a scaleable platform to deploy  various operating systems and systems software – for example you can provision a Microsoft SQL server database running on Windows server 2003 up in Amazon rather than on your own site.  You can also use the Amazon Simple Storage Service (AS3) to store data.

Salesforce

Salesforce have become something of a powerhouse on the area of cloud based Customer relationship management applications and have been extending their platform for more generic application. Of particular interest to me is the recent announcement of Force for Amazon Web Services a relationship with Amazon that would allow you to build an application on the Force service and make use of database or storage services from Amazon.

Conclusions

There is the “pure service” from Google using application services.

There is the Hybrid service from Microsoft using a combination of local and cloud applications and data

There is the API service from Yahoo using programmable interfaces that can be used to build applications from yahoo services and external services turning the cloud into something like an operating system.

There is a place for all the services and you can choose the one that best fits your needs. For me the main interest is with the API service approach and the potential of this to build and “evolve” the next development of the web – the web as operating system – a sort of invisible web!

Cloud Conditions

It isn’t possible to specify any one factor in the development of cloud computing  but like many significant events there are a set of interacting and reinforcing factors that react together to set the conditions for cloud computing.  In no particular order here are some of the factors that set the conditions where the cloud can develop and where the cloud can be a natural resource.

Internet access

Good Internet access is a necessity for Cloud computing and over the last few years Internet access has become progressively cheaper, faster, more reliable and more pervasive – it is increasingly common to be able to get on the net from almost anywhere in the developed world.

In order to use and develop cloud computing you must have good internet access.

Collaboration

Education and work today asks and requires more collaboration.

Collaboration is something that has been “bolted” on to “old style computing” and can be difficult. Trying to work on a document with a group of colleagues using email or file sharing is frustrating and version management is a nightmare. Email attachments fly backwards and forwards or users can’t edit a shared document at the same time.

The cloud is natural for collaboration – collaboration has been built in from the beginning and the resources are naturally “out there” and accessible to collaborators. 

One other big advantage for usign the Cloud for collaboration is compatibility – collaborators only need to have a browser and Internet connection. With “old style” IT each collaborator would have to have the same application installed on their computer to access the files being used for collaboration. If using Word for example each person would need to have Word installed on their computer (buy it, install it, maintain and secure it) and then have the right version to read the files (Word 2003 cannot natively open Word 2007 files). Using the Cloud people can collaborate on computers using systems from Microsoft, Apple or Linux for example.

24/7 Mobile and remote work

Education and work today asks and requires more flexible mobile and remote working – from homework to work placement and partnerships outside the organisation.

External access is something that has been “bolted on” to “old style computing” – organisations use network “firewalls” to protect their private networks and providing external access to these private resources is awkward. Either resources are placed outside the firewall (in a DMZ) or “tunnels” are provided to allow external access to internal resources. As more and more people require external access the whole concept of firewalls and tunnel access becomes difficult to sustain.

The cloud is a natural for flexible mobile and remote work – the resources are naturally “out there” and accessible from anywhere with Internet access. You can create a document in the cloud and work on it at home, in work, at a meeting with a partner organisation etc etc.  

Consumerisation and Personalisation

Education asks and requires more personalisation. The IT industry is increasingly focused on consumer issues rather than corporate issues. Students and young workers are comfortable with IT and can use their own resources to get things done.

“Old style computing” was formed from business use of IT and is focused on control and application – the ability of users to “do there own thing” is designed out of such systems. No wonder IT “users” in companies get frustrated with corporate IT. This is all made worse by the “bloat” and complexity of dealing with modern applications which makes it so difficult for “normal” people to look after their “old style” IT – they become dependent upon the IT department or those who know how to deal with this stuff. Cloud systems avoid all this – no need to spend hours installing applications and dealign with computer issues like driver problems – just point a browser.

The cloud is a natural for Consumerisation and Personalisation  - the resources are naturally “out there” and available for people to use on their own terms. People can choose and use their own communications tools and applications from social network to webmail, cloud documents and microblogging etc etc.  IT is possible for people to make use of the “natural resources” of the cloud and get things done themselves without having to wait for overworked corporate IT departments to come along and do it for them.

Security

More of our lives are mediated by the Internet and criminal activity on the net is increasing and becoming more sophisticated – the security of our on-line presence is increasingly important.

“Old style computing” was formed from business use of IT on private networks and “standalone” isolated computers – security grew out of physical access to IT (from inside a firewall to access to the computer itself). Security measures regarding the Internet and the unknown have been “bolted on and have proved difficult and only partially effective – consider the monthly security patches for Microsoft software through to the very concept of a firewall.  The problem for traditional computing is that people or organisations need to keep abreast of security issues and practice and to secure the increasing amount of equipment they use.

The cloud has been built with security in mind – rather than starting life isolated behind a firewall or cut of from a network the cloud is naturally “out there” and exposed from the start. The big advantage is that security can be delegated to experts in the cloud – I’m sure that dedicated experts at Google, Amazon or Microsoft can keep their cloud systems more secure than I can keep my computer for example.

The “old style” of IT uses local running applications and files such as Word and Word files (although these applications and files may be delivered from a server they run on the local computer). Both of these present security problems.  The local files and applications are a target for viruses and hackers – the majority of viruses ae now aimed at applications rather than operating systems. The other problem is that people carry these files around or email copies – these local files can be accessed by people with physical access to a computer or storage device – the majority of data breaches have been with lost or stolen laptops and removeable storage.  

Jack Schofield recently talked about cloud security – comparing the cloud to Fort Knox and traditional IT as Gas stations – the “cost-benefit”  involved makes targeting small installations a better option than targeting large well secured installations. A recent article at silicon.com describes the advantages of using the cloud for security. 

Another aspect of security is business continuity and availability. “old style computing” developed before the Internet and required you to buy, install and maintain your own computers and for  people to become IT experts. Increasingly people and organisation want to use their time and money using IT rather than dealing IT (hardware and software complexities for example). A small company no longer needs to install and maintain its own staff and servers for file sharing, database, web and email these resources can be accessed from web browsers while the likes of Microsoft, Amazon or Google for example manage the hardware continuity- take a look at Microsoft’s “Fort Knox”. If I had a penny for the people I have come across who have lost files from problems with either removeable media or computer hard drives I would be a wealthy man – forget local data corruption or loss – place your data in the cloud.  

In summary, where you have good Internet access and wish to develop any of collaboration, 24/7 mobile and remote work, consumerisation and personalisation and have security and continuity concerns then these are ideal cloud conditions.

What is the Cloud

Application, storage and communications services operating in the Internet and accessed through a web browser.

The Internet is a network of networks and the phrase “Cloud Computing” comes from internet diagrams that use a cloud symbol to hide the complexity of the way networks are connected. My network is connected to your network somehow through the “cloud” – I don’t need to know the details of how this happens. I only need to know how to connect to my internet service provider.

The idea in the use of the phrase “Cloud” is to simplify and hide complexity and to focus on service.

If you use a webmail system (e.g. Googlemail, Hotmail) then you already have some experience of cloud computing.

Many older email systems operated as client – server. The client was an email program that you needed to install on your computer. Servers provided a system for email clients to send and receive email messages between each other. Messages were downloaded and stored on the client computers. This meant that messages stored on one computer wouldn’t be available to you on another computer.

Webmail combines the application, storage and communications aspects of email into one service available from any computer with a web browser and an Internet connection. Webmail provides a good familiar example of what cloud computing is all about and provide my definition of cloud computing - application, storage and communications services operating in the Internet and accessed through a web browser.

Webmail has been around for ages now but the development of new web programming technologies in recent years has allowed the advantages of cloud computing to be applied to most areas of computing – hence the rapid development of new applications – here are a few popular examples of cloud computing:

Social networking – Myspace, Facebook, Beebo etc

Web2.0 – Youtube, Flickr, Blogger, Twitter, Wikipedia  etc

Applications – Googledocs, Zoho etc

For more information about Cloud computing follow the links below

A good video on the concept of cloud computing using Google Docs

A reasonable general item on Cloud computing

My Cloud links