Last week I hosted a “Bits & Bites” session on Serverless for my colleagues and some guests. These sessions are centered around some new technology, concepts or functionality (the “Bits” part) that one of us has encountered in his daily work and wants to spread the word. These events are great fun as you get to meet with your colleagues who frequently are working some other project and have dinner together (the “Bites” part) – these kinds of extras make the work fun!

As always, I had prepared way too much material for the hands-on exercises. To the positive side, nobody can claim that they finished early and had to keep themselves busy.

The scenario

In one of the scenarios, participants had to integrate their GitHub repositories with Twitter, to trigger a tweet being sent out whenever a commit was being pushed to the repository. However, a couple of days before the actual workshop when double-checking I noticed that access to Twitter APIs had become more difficult: instead of just registering for a developer account, people now had to go through the process of providing details about the application and complete an registration process, where the information provided would be reviewed before granting access. This was probably going to take days … What a bummer.

Yammer to the rescue

Fortunately, we also have (now Microsoft-owner) Yammer as one of our information exchange platforms, and we can access Yammer using its REST APIs. So I decided that we could alter the scenario slightly, sending the messages to a new Yammer group created for this purpose instead of Tweeting.

Yammering Around

As I did not want to annoy the other employees too much with my commit messages, I registered a new Yammer group “AWS-YammerBot” with our Yammer network; from the same browser – so you still have the authorisation cookie – you can retrieve the actual group_id using the Groups REST API (https://www.yammer.com/api/v1/groups.json) – store it somewhere.

Next, you will need to register a new application in Yammer (“MilcosYammerBot”) and on the confirmation page you’ll have the option to generate a developer OAuth access token – this one does not seem to expire: again, store its value for later use. We’ll be treating this token as a “shared secret” that we can potentially share across multiple applications or serverless functions.

Cloud9

Cloud9 was a full fledged development environment running in the cloud that I have described in one of my previous blogs – see https://www.syntouch.nl/first-steps-cloud-based-js-development.

At the time of writing, I was already quite fond of the product, but now AWS have acquired and integrated it, extending its use to AWS Lambda (the Serverless implementation we were using), my enthousiasm only increased.
One of the great features is that the IDE comes with the Amazon CLI installed and configured for your account, so interaction using the CLI is a breeze. From my freshly created Cloud9 IDE I opened a terminal window to store my Yammer authentication token as an encrypted parameter inside AWS Systems Manager’s parameter store:

Setting up Serverless

From the Cloud9 IDE, you can easily setup Serverless Functions (Serverless Applications are just a bundling mechanism) and also configure these to be exposed as web APIs using the API Gateway:

Adding some GitHub

Cloud9 uses the ~/environment directory as its home on the EC2 instance it’s running in; here, I initialized a new local Git repository, configured by Git name and email address, added a .gitignore (handsomely generated using the gitignore.io project) and hooked up my local Git repository to my GuysDontWantToCommit repository hosted on GitHub – where I had already added the public key I generated for this environment:

cd ~/environment
git init .
git config –global user.name “Milco Numan”
git config –global user.email milco.numan@gmail.com
# generates ~/.ssh/id_rsa.pub public key to be added to your GitHub account

ssh-keygen -t rsa -b 4096 -C “milco.numan@gmail.com”

git remote add origin git@github.com:mnuman/GuysDontWantToCommit.git

SAM

When you define your Serverless Application and Function in Cloud9, it will generate a template.yaml file. This template file is an extension of AWS’ CloudFormation template, SAM, the Serverless Application Model that allows you to declaratively describe the target environment. CloudFormation will take this template upon deployment and create the required resources in the correct order!

Before actually deploying the template, there were additions to be made to the template.yaml file:

• Define a proper function name and description
• Add an environment variable for the id of the actual Yammer group
• Define access rights for the Serverless function in a role, to allow it to read the SSM keystore (yes, permissions are very strict in AWS, you cannot go around ‘just’ accessing services – and that is a good thing)

Coding the Lambda Function

The actual code of the lambda function is quite simple; we’re not building a monolith, right? It retrieves the ‘secret parameter’ from the SSM Parameter store and uses this to construct the required authentication token (a simple OAuth Bearer token), outside the actual function so the code will not be executed when reusing a ‘warm’ container as the token is not going to change. Next up, I construct a simple message to be sent to the world using the push event’s properties using the author’s name, the commit’s URL and the actual commit message. The response is formatted for AWS proxy integration (supporting all methods on a single endpoint):

import boto3

import json

import os

import requests

ssm = boto3.client(‘ssm’)

parameter = ssm.get_parameter(Name =’SecretYammerToken’, WithDecryption=True)

oauthHeader = { “Authorization” : “Bearer ” + parameter[‘Parameter’][‘Value’]}

def lambda_handler(event, context):

body = json.loads(event[‘body’])

myMessage = f”Hi, {body[‘head_commit’][‘author’][‘name’]} has just committed code at {body[‘head_commit’][‘url’]} with message {body[‘head_commit’][‘message’]}”

payload = { “body” : myMessage, “group_id” : os.environ[‘YAMMER_GROUP_ID’]}

r = requests.post(“https://www.yammer.com/api/v1/messages.json”, headers=oauthHeader, data=payload)

return {

“isBase64Encoded” :”false”,

“statusCode” : r.status_code,

“headers” : {},

“body” : “Acknowledge receiving webhook request”

}

Deploy! Deploy! Deploy!

Deploying the code (actually: rolling out the stack in cloud formation) is quite easily done using Cloud9 – just a simple “Deploy” on the context menu. Behind the scenes, however, CloudFormation will be provisioning a stack, zipping and uploading your implementation code, defining roles, creating an API and performing whatever other actions are necessary to create the resources you request in the template:

After deployment has completed, it’s time to find out where your API will be exposed to the big, bad internet – the invoke URL:

Hooking Up GitHub

After retrieving the address of our API, this needs to be registered as a Webhook in your GitHub repository. Effectively, a webhook is a HTTP push mechanism where the event payload is posted (in this case, we’ll be using a JSON format):

Proof of the Pudding

One of my favourite quotes is “The proof of the pudding is in the testing” … So: commit all changes to the local git repository and push these into GitHub – this should trigger a message being sent out to my new Yammer group:

As the URL has been provided in the message, let’s drill down into the actual commit:

Conclusion

Wow, that was a simple scenario to automate two disparate systems, GitHub and Twitter, using some small “glue” code in Python to transfer and format the relevant information!  Although … both Yammer and GitHub are now part of the Microsoft portfolio, so who knows how easy it will become in the coming time to configure such an integration out of the box.

For now, this demonstrates the power of AWS Lambda to create new possibilities with only a small amount of code, easily coded using AWS’ Cloud9 and deployed as a CloudFormation/SAM template!