Join for free and connect with our local tech scene

Stay on top of the latest companies and upcoming events with our weekly newsletter, and be counted among the people building the future of your local tech community.

Charleston, SC /

AWS Kinesis, Websockets, and how Snag leverages Kinesis for live log filtering

Snagajob 174 Meeting Street , Charleston, sc 29401 (map)

Greetings Everyone!

We’re back from the dead, and with any luck, after this next meetup we will be meeting monthly again. Not only would I like to expand the topics this meetup group will cover beyond just AWS services, but I’d like to also take DevSecCloudOpsy topic requests that we can structure future presentations around.

At our next meetup we’ll be going over AWS Kinesis and how Snag has leveraged AWS Kinesis streams to ship, filter, transform, query, and archive our application and infrastructure logs at scale. Through this pipeline we are able to extract live insights from our logs within seconds of their delivery, enabling anomaly detection and eventually automated incident remediation. Cool right?! We see a daily throughput of around 250 to 300 million messages per day but are built to scale well into the 10’s or hundreds of billions through our fully serverless Kinesis streaming platform. Come check it out, grab some free beer, pizza, sodas, popcorn, candy, probably desserts too. Heck we’ll even take food requests if it means you’ll show up! :)

We'll also be going over AWS API Gateway Websockets with Lambda + Kinesis and how these work into our pipeline.

I’m shamelessly including an architectural diagram with the hopes of piquing your interest and increasing attendance of our official AWS CHS meetup resurrection. If you were wondering where Zach Roof is (the last organizer of this AWS meetup group), he’s still here, works at Snag, and with any luck should be at this meetup too!

Comments

Sign in to comment.