← back

Development

Cloud Automation

Image by rawpixel.com

Keeping my lifetime interest in software development ...

Since typing my very first lines of rather basic 'Basic' and CP/M; coding in all its forms has remained at the core of what has kept me interested in technology. The idea of being able to start with an idea or a manually intensive process and code up a working solution never stops being a strong pull.

The most recent language / framework that I've focused on is React and NextJS. For many years I have been able to code using traditional 'backend' languages such as Java and (preferably) Scala, but until recently I have never focused in depth on the frontend. Ever since 2015 when I needed to hire a professional Angular developer to code up a frontend for a personal project, I have wanted to make sure that I can code on the UI/UX side of things.

A personal project using EC2 Spots in an elastic cluster...

The 2015 personal project was an algorithmic trading idea that would allow anyone to test out trading strategies. The backend was in Scala and strongly leveraged Akka to implement the Actor model to facilitate 'worker requests' being distributed across a dynamically scalable AWS EC2 cluster of nodes. The cluster would dynamically scale out based on worker CPU utilisation and made heavy use of AWS Spot Instances to keep costs under control. Although the frontend integration was a challenge, by far the hardest part was a customised authentication and authorisation process that required compiling a custom C module into Nginx. Thankfully the likes of AWS Cognito and other cloud native solutions make this task considerably easier these days.

Although this project is effectively mothballed, it utilised a few concepts that I think I could easily reuse in the future. Primary amongst them is how it is possible to implement a horizontally scalable cluster that can cope with node failures but which does NOT depend on K8 (Akka helps a lot here !).

Other languages and toolsets that I have recent practical experience of include Neo4j and R. Neo4j in particular is something I regularly come back to because I have often found that it's the relationship between data sets that yields the most insights - and a graph database is perfect for that.

← back