close

Blogroll

Showing posts with label AWS. Show all posts
Showing posts with label AWS. Show all posts

📚 30+ Serverless Blogs Every Developer Should Follow

These are some of the blogs I follow for high-signal content, real-world lessons, and emerging patterns.
  1. The Burning Monk (Yan Cui) – Deep technical guides, real-world insights, and event-driven serverless at its best.
    🔗 https://theburningmonk.com/

  2. Off-by-none (Jeremy Daly) – Weekly curated newsletter filled with top serverless news, tools, and community updates.
    🔗 https://offbynone.io/

  3. A Cloud Guru Blog – Trusted training content with strong coverage of AWS, certifications, and serverless how-tos.
    🔗 https://acloudguru.com/blog

  4. Serverless.com Blog – Official blog for the Serverless Framework team with tutorials, product updates, and ecosystem insights.
    🔗 https://www.serverless.com/blog

  5. AWS Community Builders Blog – Real-world articles from active AWS contributors and thought leaders.
    🔗 https://dev.to/aws-builders

  6. Serverless Transformation (Aleios) – Strategy-rich serverless insights, especially around event-driven architecture.
    🔗 https://medium.com/serverless-transformation

  7. AWS Compute Blog – Deep dives into AWS Lambda, Step Functions, and compute services directly from AWS.
    🔗 https://aws.amazon.com/blogs/compute/

  8. AWS Architecture Blog – High-level cloud-native architecture practices and guidance for scalable apps.
    🔗 https://aws.amazon.com/blogs/architecture/

  9. Lumigo Blog – Focused on observability, monitoring, and debugging in serverless environments.
    🔗 https://lumigo.io/blog/

  10. Serverless360 Blog – Azure-first serverless content with strong focus on Azure Functions and service management.
    🔗 https://www.serverless360.com/blog

  11. Nick Tune’s Blog – Sociotechnical thinking and service design strategies for microservices and serverless.
    🔗 https://medium.com/nick-tune-tech-strategy-blog

  12. AWS Enterprise Strategy Blog – Executive cloud transformation strategies, ideal for leadership and enterprise architects.
    🔗 https://aws.amazon.com/blogs/enterprise-strategy/

  13. Theodo Blog – Engineering-led content featuring serverless, migration, and rapid product delivery stories.
    🔗 https://blog.theodo.com/

  14. Serverless First (Paul Swail) – Practical daily tips, guides, and architecture breakdowns from an experienced consultant.
    🔗 https://serverlessfirst.com/articles/

  15. Serverless Land – AWS-curated library of patterns, tutorials, and EDA content for serverless professionals.
    🔗 https://serverlessland.com/

  16. Lego Engineers Blog (Sheen Brisals & team) – Real enterprise-scale implementation stories from a global brand.
    🔗 https://medium.com/lego-engineering

  17. Serverless Chats Podcast – Interviews with top serverless minds — with full transcripts for every episode.
    🔗 https://www.serverlesschats.com/

  18. Serverless Guru Blog – Transformation-focused advice and engineering best practices from a consultancy team.
    🔗 https://www.serverlessguru.com/blog

  19. The Serverless Edge Blog – Cloud strategy, org design, and the “value flywheel effect” in serverless transformation.
    🔗 https://theserverlessedge.com/

  20. InfoQ Serverless – Aggregated industry-wide coverage, articles, and videos on serverless and architecture.
    🔗 https://www.infoq.com/serverless/

  21. Jeremy Daly’s Personal Blog – Detailed architectural posts and in-depth serverless exploration beyond the newsletter.
    🔗 https://www.jeremydaly.com/posts

  22. Lee Gilmore’s Blog – Valuable insights on enterprise serverless adoption and scaling strategies.
    🔗 https://blog.serverlessadvocate.com/

  23. Ready, Set, Cloud (Allen Helton) – Friendly, digestible blog posts with great diagrams and tutorials.
    🔗 https://www.readysetcloud.io/blog/

  24. Sheen Brisals’s Blog – Engineering leadership perspectives with an emphasis on EDA and maturity.
    🔗 https://sbrisals.medium.com/

  25. Aiden Steele’s Blog – Low-level AWS insights with rare gems on Lambda internals and advanced configurations.
    🔗 https://awsteele.com/

  26. Luc van Donkersgoed’s Blog – Visual, simple explanations of complex cloud patterns and serverless tips.
    🔗 https://lucvandonkersgoed.com/

  27. Benjamen Pyle’s Blog (Binary Heap) – Great technical content on building serverless in Rust.
    🔗 https://binaryheap.com/

  28. Ben Kehoe’s Blog – Deep strategic thinking on cloud operations, organizational design, and serverless ops.
    🔗 https://ben11kehoe.medium.com/

  29. Alex DeBrie’s Blog – Author of The DynamoDB Book, Alex shares thorough serverless data modeling guides.
    🔗 https://www.alexdebrie.com/posts/

  30. Last Week in AWS (Corey Quinn) – Hilarious, opinionated, and insightful commentary on all things AWS — including serverless.
    🔗 https://www.lastweekinaws.com/

  31. AWS Fundamentals Blog – Accessible cloud concepts from community experts.
    🔗 https://awsfundamentals.com/blog

  32. Vadym Kazulkin on Dev.to – Detailed explorations of Lambda SnapStart and Java serverless.🔗 https://dev.to/vkazulkin

  33. Cloudonaut Blog (Wittig Brothers) – Production-grade AWS best practices and architectural tips.
    🔗 https://cloudonaut.io/
Do you know other technical blogs on serverless or cloud? Comment below and share your favorites. 

A Camel Demo for Amazon's Simple Worklfow Service

In a previous post I explained why AWS SWF service is good and announced the new Camel SWF component. Now the component documentation is ready and here is a simplistic fully working demo. It consist of three independent standalone Camel routes:
A workflow producer allows us to interact with a workflow. It can start a new workflow execution, query its state, send signals to a running workflow, or terminate and cancel it. In our demo, the WorkflowProducer starts a route that schedules 10 workflow executions where the each execution receives as an argument a number.
Once a workflow execution is scheduled, we need a process that will decide what are the next steps for it. In Camel it is done using a Workflow Consumer. A workflow consumer represents the workflow logic. When it is started, it will start polling workflow decision tasks and process them. In addition to processing decision tasks, a workflow consumer route, will also receive signals (send from a workflow producer) or state queries. The primary purpose of a workflow consumer is to schedule activity tasks for execution using activity producers. Actually activity tasks can be scheduled only from a thread started by a workflow consumer.
The logic in our demo decider is simple: if the incoming argument is greater than 5, we schedule a task for execution. Otherwise the workflow will complete as there are no other tasks to be executed. Notice that it also has branches for handing signal and state query events.

The final peace of our distributed (since it consists of three independent applications) workflow application is the ActivityConsumer that actually performs some calculations. It has the simplest possible implementation: increments the given argument and returns it.
All you need to do to run this demo is to create the appropriate workflow domain and add your key/secret to the route.

Building Distributed Workflow Applications on Amazon with Camel

Pipeline with SNS-SQS
A workflow consist of independent tasks performed in particular sequence determined by dynamic conditions. Very often a workflow represents a business process, for example the order processing steps in a ecommerce store.
Amazon Web Services offer various tools for building distributed and scalable workflow applications. One approach for building such an application is to use topics and queues for connecting the distinct steps in the workflow process. Then we can use publish/subscribe,  competing consumers and other mechanisms to scale our application and soon even the simplest application takes a shape similar to this:
BERJAYA
Each step of the pipeline is connected to the next one with a queue and each step performs some actions and takes decision what is the next step. In addition using SNS/SQS involves some other low level tasks:
- Serialize/deserialize the data
- Ensure consistency (FIFO order) for SQSmessages
- Make sure message size is not exceeded
- Invent some kind of auditing support
- Subscriber queues to topics, assign permissions
- Manage DLQs
At the end it works, but overcoming these technical challenges takes as much time as writing the actual code that delivers the business value.
Simple Workflow Service
SWF on the other hand offers a higher level API for writing distributed, asynchronous workflow applications. It automatically serializes/deserializes data, manages application state, offers auditability, guarantees strong consistency, supports multiple versions. Most importantly, it ensures that the workflow orchestration and business  logic execution are separated. Any typical SWF application has the following building blocks:
BERJAYA
In SWF terms, a workflow is the actual template that describes the distinct steps a process should follow. And a workflow execution is one run of this template.
Starter - the process that can start, stop and interact with a workflow execution.
Decider - the process that orchestrates and decides what is the next step of a workflow exection.
Worker - a process that executes a tasks from a specific type.
SWF Console - provides full visibility and control of the execution.
An example workflow execution can go through the following steps: a starter starts a workflow execution, SWF receives it, asks the decider what is the next step, then based on the decision passes the task to an appropriate activity worker. Once the result from the activity worker is received SWF asks the decider again for the next step, and depending on the response may execute another worker or not. This flow continues till the decider replies that the workflow is completed. You can see how the decider orchestrate each of the steps of the workflow and the activity workers perform the individual tasks. All that is managed by SWF and auditable at any stage.
Why use Camel?
The amazon provided Java clients work by using annotations to generate proxy classes to access SWF services. The whole process of generating and using proxy classes combined with the dependency from the starter to the decider, and from the decider to the activity workers is not very joyful. And what can be better than using a Camel route for orchestration and another route for the actual activity worker? The result is a Camel SWF component that is in Camel master now. Camel-swf component has two types of endpoints: workflow and activity.
A workflow producer allows us to start, terminate, cancel, signal, get state or retrieve the whole execution history of a workflow execution. In our diagram it represents the starter. Here is an example of how to start a workflow execution:
A workflow consumer is the decider. It receives decision tasks from SWF service and either schedules activity tasks for execution or indicates that the workflow execution has completed. It is a stateless deterministic route that only job is to orchestrate tasks:
The activity endpoints allow us to interact with the activity tasks. An activity producer is used to schedule activity tasks, and it can be used only from a decider route (actually decider thread). It is because only a decider can schedule activity tasks. The last box in our diagram that we have to provide implementation is the activity worker, which can be created using an activity consumer. This endpoint will receive activity tasks from SWF, execute them and return the results back to SWF. This is the bit that actually performs the business logic:
So any SWF application consist of a starter(workflow producer) that starts the execution, a decider (worfklow consumer) that receives decision tasks and schedules activity tasks (using activity producer) and the activity workers (activity consumer) that performs the tasks. And the communication between these endpoints is asynchronous, consistent and managed by SWF service.
It is not the easiest component to use, but it pays off with a simple and scalable architecture.
PS: Thanks to my ex-manager S. Wheeler for letting me contribute this component back to the Camel community.

How to do FIFO messaging with Amazon SQS

If you have used Amazon Web Services, you probably know Simple Queue Service(SQS) - it is a reliable, highly scalable hosted queue for storing messages. One of the main drawbacks of SQS is that it does not guarantee first-in, first-out (FIFO) access to messages and that's clearly stated in the Amazon documentation:

"Amazon SQS does not guarantee FIFO access to messages in Amazon SQS queues, mainly because of the distributed nature of the Amazon SQS. If you require specific message ordering, you should design your application to handle it."

Here is a quick example how Camel Resequencer pattern can help you overcome this drawback in 2-3 lines of code. To setup our example scenario, let's first create a route that will populate our queue with 100 messages each containing the message number: Then create a consumer route, that will read from the queue and log each message:
To prove that Amazon doesn't guarantee FIFO ordering we will write a test: Don't be misled by the short size of the test, it starts both routes and verifies that all 100 messages are received in the right order.

The above test fails in most of the runs and proves that SQS doesn't support FIFO order(when the messages are sent too quickly). The idea in this example is that our producer will index the messages or provide some kind of sequencing information, so that the message consumer can interpret it and order the messages. To do that we simply send a number as message body, but in a real world application that can be a field in a JSON or XML message. Then to make the test pass and ensure that the messages are received in the same order as they were sent, all we have to do is to add the Resequencer pattern in our consumer route: The streaming based Resequencer will let the messages go without any delay as long as they are in the right order. If the messages are not in the right sequence, it can hold up to 100 messages for 1 second while waiting for the missing message. Depending on your message load, you should adjust these numbers to hold enough messages while waiting for the missing one, but not hold for too long and reduce the throughput. Another option would be to try the non-streaming batch based Resequencer, which always collects a number of messages before sorting and releasing them.
BERJAYAIf you are new to Apache Camel and Enterprise Integration Patterns (like the Resequencer), have a look at my recently published "Instant Apache Camel Message Routing" book where similar patterns and how to use them in Camel are explained in a short and focused manner.
Or if you want to deep dive into the integration world, I recommend you start from "Enterprise Integration Patterns" and "Camel in Action" books.

An old pet project based on Sencha Ext JS and Apache OFBiz

Play with the DEMO using username: scrum and password: scrum

Couple of years ago while working on software project we used Pivotal Tracker as our project management tool. It was a great free SAAS when it suddenly changed its terms and conditions and became a paid one. As a developer with great enthusiasm I said to myself "I know a great UI library (with not so great license) - Sencha Ext JS and great backend project with not so great UI - Apache OFBiz, why not combine them and create something better and still free". After couple of months I created LazyPlanner and realized that it is full with free project management tools out there, so it never went live. I wish I had read Eric Ries's The Lean Startup book earlier and had not started my idea by coding it first.
BERJAYA
Any way, now I found this old project on my computer and put it on github. It is a standard component for OFBiz which works just by putting it in hot-deploy folder. For this demo installation I put some data and created a project with couple of task lists (called sprints) and few tasks. It supports multiple projects, with multiple tasks lists and tasks... It will be also running on the cloud for couple of days, so play with it and if you find it appealing get the code and use it on your own risk.

Accessing AWS without key and secret

If you are using Amazon Web Services(AWS), you are probably aware how to access and use resources like SNS, SQS, S3 using key and secret. With the aws-java-sdk that is straight forward:
AmazonSNSClient snsClient = new AmazonSNSClient(
new BasicAWSCredentials("your key", "your secret"))
One of the difficulties with this approach is storing the key/secret securely especially when there are different set of these for different environments. Using java property files, combined with maven or spring profiles might help a little bit to externalize the key/secret out of your source code, but still doesn't solve the issue of securely accessing these resources.
Amazon has another service to help you in this occasion. No, no, this is not one more service to pay for in order to use the previous services. It is a free service, actually it is a feature of the amazon account. AWS Identity and Access Management (IAM) lets you securely control access to AWS services and resources for your users, you can manage users and groups and define permissions for AWS resources.
One interesting functionality of IAM is the ability to assign roles to EC2 instances. The idea is you create roles with sets of permissions and you launch an EC2 instance by assigning the role to the instance. And when you deploy an application on that instance, the application doesn't need to have access key and secret in order to access other amazon resource. The application will use the role credentials to sign the requests. This has a number of benefits like a centralized place to control all the instances credentials, reduced risk with auto refreshing credentials and so on. 
Once you have role based security enabled for an instance, to access other resources from that instances you have to create and AwsClient using the chained credential provider:
AmazonSNSClient snsClient = new AmazonSNSClient(
new DefaultAWSCredentialsProviderChain())
The provider will search your system properties, environment properties and finally call instance metadata API to retrieve the role credentials in chain of responsibility fashion. It will also refresh the credentials in the background periodically depending on its expiration period.
And finally, if you want to use role based security from Camel applications running on Amazon, all you have to do is create an instance of the client with configured chained credentials object and don't specify any key or secret:
from("direct:start")
.to("aws-sns://MyTopic?amazonSNSClient=#snsClient");