Over the last years, I have become quite interested in cloud solutions. After completing the vendor-neutral Cloudschool.com’s Cloud Architect certification track, I decided it was time to dive into the real thing.
Although I come from an extensive Oracle background, until now I have not been really impressed with the Oracle Cloud solutions from my perspective as a developer. But when a colleague presented a talk at an internal company event about Amazon Web Services, I became quite attracted to the solution development possibilities on the AWS platform.
To get an initial overview of the huge AWS ecosystem, I decided to take the associate-level AWS Certification Exams for Architect and Developer (still want to do the DevOps). As part of my preparation for these exams, I studied the ACloud.guru training materials and learnt they were running their entire site on AWS, with a good part of it being supported by AWS Lambda.
Serverless
AWS Lambda is a so-called ‘serverless’ compute platform, executing code on demand. Seeing a platform like this offers great benefits when you can partition your app into single-purpose stateless functions. This frees the business and developers from all these (boring) infrastructural concerns like provisioning and scaling, and creates more time for cool new features and coding!
Yes, underneath the hood there is infrastructure and there probably is a lot of complexity going on, but I like this to be taken care of by other experts. As much as I like driving my car, I do not maintain it myself nor do I maintain the infrastructure itself – these are taken care of by a lot abler hands than mine, leaving me time to focus on what I’d rather do: coding.
In FaaS (function as a service), the entire platform will be provided by the Cloud vendor (Amazon Lambda, Google Cloud Functions and Microsoft Azure Functions follow this model), although there are also ‘serverless’ solutions that provide an execution runtime for your functions where you would need to make arrangements for the infrastructure yourself (like Oracle’s Open Source Fn project).
And then … there was Serverless Paris
During my travels over the internet and reading Peter Sbarski’s excellent “Serverless Architectures on AWS”, I found out that there was an actual one-day conference on Serverless in Paris, optionally preceded by an extra day of pre-conference workshops. I decided I really wanted to attend to get a better idea of what could be achieved by following this new paradigm and better understand where the challenges may lie from experiences with real-life projects.
Day 1: Workshop day
Workshop day proved to be a nice hands-on experience for me. There were four different all-day workshops on offer, but as I came onto this path by learning about AWS, I decided it was time to get my hands dirty on AWS and attend the A Cloud Guru workshop, co-hosted by the local organiser D2SI, on “AWS Lambda, API Gateway and Firebase”.
Apparently I was not the only one interested in AWS, as the workshop was flatly sold out. During this workshop, we got – from my point of view as a starter in AWS – a nice and gentle introduction and build a kind of video sharing website:
At the end of the day, we had built a website hosted locally on a node.js server, offering video upload capabilities. The videos were uploaded to an AWS S3 bucket and would trigger the execution of the first lambda function by means of raising an event to which the lambda function subscribes.
Inside the event is all the required information like which bucket was hit and the specific key of the uploaded file. From the first lambda function, a job was submitted to invoke another AWS supplied service, Elastic Transcoder, that can generate different encodings for different devices (e.g. sizes, frame rates and resolutions) from the input video, outputting the transcoded files to a different S3 bucket.
Again, creation of the (transcoded) video files triggered a different lambda function, this time a function to construct the video file’s URL and insert this into a Firebase database hosted by Google.
As part of the (mostly precooked) web site, the newly added videos where directly pushed by Firebase to show up in the user’s browser.
The entire website was secured using yet another additional (non-AWS) service, Auth0, to allow users to register and authenticate. Using this scenario, a number of AWS provided services and external ones, some glue code in the form of AWS Lambda was enough to build a fully functional website in a couple of hours!
When in Paris …
What I have always regretted since my last visit to Paris is that I have not taken the opportunity to visit the Louvre museum, so when I got some spare time off in the evening I decided to spent a couple of hours in this wonderful museum … and I wasn’t disappointed. Apart from the world famous and crowded (!) “Mona Lisa”, the museum hosts an overwhelming amount of classical art, like the Nike of Samothrace (https://en.wikipedia.org/wiki/Winged_Victory_of_Samothrace).
Day 2: conference
On the second day, the actual conference was held. Fortunately, it was a single track conference so there was no need to make any difficult choices allowing all participants to attend all 15 sessions! The sessions overall were quite interesting, discussing common challenges like: how to observe your serverless system, without access to the host server and its operating system to install your custom agents? Usually, you’re left with whatever your cloud provider has empowered you to work with, like AWS’ CloudWatch/CloudTrail, AWS X-Ray (which apparently does not yet support API Gateway). Yan Cui pointed to the fact that by default, AWS Logs do not expire and that for a complex system you should not be interested in the health of the system as a whole (how could you?), but in the actual health of the event or the slice that is processing.
In another session, Mx Kassandra Perch shed some more light on observability of a serverless system by discussing the benefits of implementing tools like iopipe (https://www.iopipe.com), which not only allow for custom tracing, but also brings you memory and CPU stats (to allow you to better tailor the environmental settings for your AWS Lambda function), metrics, monitoring and alerts and a bunch of other cool things that are not obtainable from AWS monitoring aids.
Several customers shared their experiences with Lambda, like Expedia’s Technology VP Subbu Allamaraju. Expedia is doing some 2000 deployments on a daily basis and their solution is invoking lambda functions some 6 billion times each month. Subbu is a passionate advocate of disaggregation state from your applications and shit the stateful complexity to managed cloud services. Other advice included to not be afraid of vendor lock-in, but to fear the inability to change!
A presentation on IBMs Cloud Functions & Apache OpenWhisk left me feeling somewhat anxious … they discussed a customer’s scenario where a radio or television commercial might play an ultrasonic sound that is then picked up by your mobile phone or tablet by the customer’s app. Some serverless function would get triggered and serve you a complimentary popup or coupon, but this image – if I remember correctly – left me feeling spied upon.
The last presentation I find worth mentioning is the presentation by AWS’ Randall Hunt, who has shown a lot of cool use cases of lambda functionality like the WhereML twitterbot trying to find your location by using the pixels (not the EXIF) of a picture tweeted to it:
Some interesting topics he mentioned are the availability of canary deployments for AWS API Gateway, increasing the Lambda memory limit to 10 GB, support for .Net 2.0 and the Go programming language and about the introduction of Aurora Serverless, a relational database solution pay-per-second & DB-capacity used.
Finally, About AWS Lambda, initially only running Node.js:
Looking back to this workshop and seminar, I have been able to get hands-on with some hot new cloud technology and learned useful lessons and insights from early adopters and tool vendors.
The only disappointments were that the lunch on the second day did not exactly “scale” according to the crowd present (‘mais heureuxement, il y a beaucoup des boulangeries artisanales en France’) and that I did not get to meet Peter Sbarski (VP of Engineering @ A Cloud Guru) to have him autograph my copy of his “Serverless Architectures on AWS”:
