Behind the Big Data Curtain: A Peek at How AWS Really Works

Want to know how you can tell a technology is transforming our culture? When it has its own week-long convention that spans multiple locations in Las Vegas, Nevada. If you guessed we’re hinting at AWS re:Invent, then you guessed right!

AWS re:Invent just finished up its sixth year back in November 2017. Clearly it has a rich history and a supportive community that continues to grow. But what is it, and more to the point, why should you care? Aren’t we talking about Amazon Web Services and cloud computing?

AWS re:Invent is a special event for the AWS Partner Network (APN) and which affords many opportunities to learn, connect, collaborate and explore upcoming features. AWS — which you hopefully know already — is simply “Amazon Web Services,” or the tech giant’s cloud services platform. It can be used and leveraged for many things, from computing power and cloud storage to content delivery and more. Amazon is one of the biggest players in the cloud computing industry and serves hundreds — if not thousands — of partners and customers.

You’ll learn more about the platform’s inner workings and what benefits it has to offer in a bit. Before we get too involved, though, let’s dive in and take a closer look at the history of AWS and its evolution over the past few years.

A Brief History of AWS

Believe it or not, the service originally started as an internal development process meant to improve the efficiency of Amazon’s own infrastructure. It was later adapted to add support for external parties — more specifically, for developers and the surrounding community.

In 2002, the company released a beta called Amazon.com Web Service. It targeted developers who wanted to collaborate with Amazon’s platform. The initial phase introduced SOAP and XML interfaces that could tap into and leverage Amazon’s vast product catalog. This was the fork in the road, if you will, that led the company down the path toward delivering more robust infrastructure services and developer tools.

AWS is one of the most important pieces of business intelligence tech available to big data contractors.

In 2004, Jeff Barr first introduced AWS to the public in a blog post with a call for success stories to be shared. And on March 19, 2006, AWS officially launched. It was a shell of what it is today, first with Simple Storage Service (S3) and then Elastic Compute Cloud (EC2) and Simple Queue Service (SQS) shortly after. In 2009 — after a European launch of S3 and EC2 — the Elastic Block Store (EBS) was made public and Amazon CloudFront (CDN) was established.

It didn’t take long for major companies to catch on to the potential of the service — especially thanks to its developer-friendly structure. Data-driven brands like Dropbox, Netflix and Reddit were some of the first to adopt AWS formally before 2010.

Today, the partner base has grown immensely to include additional brands like Unilever, GE Oil and Gas, Kellogg’s, Adobe, Comcast, McDonalds, Square Enix, Xiaomi, Spotify, Lyft and IMDb. Even big data federal contractors rely on the service, such as the U.S. Department of State and other agencies.

The rest, as they say, is history.

At this point, it’s worth noting that while AWS is the leading cloud services provider by far, there’s been a lot of buzz lately about some of the other competitors in the market — Microsoft, to be exact, thanks to their Azure platform. Amazon’s service likely isn’t going to be dethroned anytime soon, but it is important to understand there are other services and providers out there.

To sum it all up, Amazon Web Services or AWS is Amazon’s cloud computing platform that formally allows any business, big or small, to reach the immense scale of Amazon’s web platform. The allure is that the infrastructure already exists, cutting down on the overall expenses of building out such a system internally.

Furthermore, Amazon’s service is secure, scalable and highly reliable — all features that would take a long time to establish for a self-deployed cloud service, not to mention costly. It has a storied history with an incredibly supportive community and development team propping it up.

How AWS Works

In a nutshell, cloud computing is a streaming, always-on, and on-demand delivery of IT resources and computing via the internet. The services are often attached to payment plans in a pay-as-you-go structure similar to a subscription, with multiple tiers of support.

Big data contractors need to be familiar with the amazing business intelligence management power of AWS

Relying on a cloud computing service like AWS is cheaper than buying, owning, maintaining and growing your own data center and server base. Handling this yourself with an internal team means scaling up or down to meet the needs of your products and services. As a result, there’s often a “purgatory” phase between shrinkage or growth where resources are either left unused or are in high demand and unavailable.

Through cloud computing, organizations and businesses can instead outsource the computing power, storage, database facilitation and similar IT services on an as-needed basis. You also have the option to scale up (or down) instantaneously. The provider manages, maintains and secures the infrastructure for you, generally with more capable hardware and personnel.

The cloud computing movement follows this concept of transferring processes, systems and infrastructure onto the web and off-site. AWS essentially was, and still is, the cornerstone of this movement. Amazon was one of the first providers to truly offer outsourced and cloud computing power and resources to everyone and not just close business partners and major organizations. That has helped make it the immense success it is today.

AWS is particularly useful for startups and growing companies that don’t have the resources, manpower and capital to host these systems internally in their own data centers. You’ll also find that business intelligence management is a common use for the AWS platform by many organizations. The tools, applications and methodologies required to collect data from internal systems and external sources — like customers — have much more power and reliability through Amazon’s massive support structure. Not to mention, it must be stored securely somewhere and the requirements of said storage are constantly growing. As more data comes in, more resources are needed to both store and process it appropriately.

The flexible scaling support of cloud computing systems allows this to happen much more freely and efficiently.

As for what you can do or implement through AWS, choosing just one thing is difficult to do. The platform itself can be useful for virtually any use-case imaginable. There are several services or tools offered, starting with SS3 and stretching to CDN or content delivery. A total of over 50 services are available to anyone with “just a few mouse clicks,” as Amazon states.

If you are a big data federal contractor, you can find AWS globally through their 50 availability zones.

AWS is also globally available via 50 “availability zones” spread across 18 geographic regions around the world. In addition, the company has announced plans to establish 12 more availability zones in four more regions, including Bahrain, Hong Kong, SAR and Sweden. A second AWS GovCloud Region will be established in the U.S., too. This highlights the immense support and reliability offered by Amazon’s network.

Getting started with AWS is actually pretty simple, which is the next logical step after learning about the service. You just need to sign up for a free account, and although you will need to include payment information up front, it won’t be charged unless you chose to upgrade to a paid plan. The reason we’re even mentioning this is because it’s a good idea to set up a free account, poke around on the platform. and train yourself — that is, of course, if you’re considering adopting the service for your business.

Is It Safe?

All this talk about cloud computing, streaming and content stored on the internet brings one glaring concern to the forefront: security. Cybersecurity and data integrity is a major concern, especially in the current landscape. With so many companies and organizations being attacked or experiencing damaging data breaches, what’s to say your business won’t be next?

In reality, no infrastructure is completely safe — certainly not one hosted in the cloud and publicly accessible. However, that doesn’t mean AWS is unsafe to use. Quite the contrary, actually. Security for cloud computing is recognized as stronger and more reliable than on-premises networks and infrastructures for several reasons.

For starters, we’re talking about organizations (like Amazon) that have an incredibly vast supply of resources and experience with IT services. This includes an incredibly large team of professionals with years of compounded experience, talent and knowledge. Even if you have a skilled internal team, they’re no match for the massive support Amazon and other cloud providers have in their arsenal.

That same supply of resources also extends to hardware, applications, development and everything else associated with establishing, maintaining and securing such an infrastructure. Cloud providers specialize in, well, the cloud, so they are constantly updating and maintaining their systems and hardware. This affords several advanced security features or mechanics that traditional and on-site infrastructures simply cannot provide. Cloud computing networks have much stronger surveillance tools and perimeter security measures and they undergo regular audits so vulnerabilities can be discovered and dealt with.

On the employee side of the equation, a cloud computing system is stored off-site, which means there’s limited and controlled access. It is physically separate from a company’s mission-critical components and data. If an employee plugs in a USB drive on the local side, infecting their system, there’s a lot less danger involved. It’s still concerning for many reasons, but the infrastructure itself won’t be immediately available to attacking parties, if at all.

Gartner predicts that through 2020, public-cloud-infrastructure-as-a-service workloads will be hit by 60 percent fewer security incidents than traditional data centers. In the same report, they claim that 60 percent of enterprises implementing cloud visibility and control tools will deal with one-third fewer security failures by 2018. Needless to say, neither of these scenarios would be possible if cloud computing were insecure and dangerous.

Why AWS Is the Next Step for Businesses

The only logical conclusion for an organization looking to bolster, secure or improve their infrastructure at this point is to shift to the cloud. The benefits are vast.

Business Intelligence can't be intelligent without AWS.

You’ll see cost savings and support, proactive infrastructure management, nearly guaranteed uptimes and ultimate scalability. Good luck establishing or achieving that with a legacy system. It’s possible, but it’s going to be costly and resource-intensive and will take a considerable amount of time and patience — and trial and error.

We’re not going to preach that cloud computing is the end-all, be-all solution for every single scenario, because it’s not. At least, not to the extent where you need to move all of your assets, systems and operations to the cloud right this second. Some have found it better to adopt a hybrid approach. It’s not always a good idea to have everything stored in the cloud anyway. But the point we’re trying to make is that the cloud — read: AWS — will offer a degree of efficiency and reliability to nearly every team and organization that a legacy system simply cannot.

Even if you’re not willing to dive in now, at least take the time to educate and train yourself on the necessary applications and implementations of modern cloud computing systems. There are many resources at the ready that can help you learn.

DataSync Technologies is a great example. Take advantage of our resources to become familiar with this technology. As we recommended earlier, you might even consider signing up for a free account on Amazon’s Web Services portal and poking around a bit.

Where Can I Learn More?

Since you may be looking for additional resources at this point, we thought it was a great idea to briefly touch on some more relevant sources.

Amazon actually has their own digital training and certification program, all of which is available online. If you have a busy schedule and limited knowledge currently, this is definitely where you’ll want to start. Popular online curriculum portal Udemy also has a couple different programs for you to check out. There’s the AWS Certified Solutions Architect and then there’s the Certified AWS Developer Program.

You can also access the official AWS documentation portal as well as the open-source documentation repository on GitHub.

If you do happen to complete and look over everything listed here, then it’s off to the races! You’ll have gained enough knowledge and insights to jump right into using the AWS service. For more guidance, remember to subscribe to the DataSync Technologies online newsletter.