Here are a set of instructions to build and package from source either the storm-0.8.2.jar or the complete storm-0.8.2.zip (with all dependencies). I assume packaging later versions will be similar, just be careful about dependencies versions.
Read the rest of this entry »
In this post, I illustrate how to maintain in DB the current state of a real time event-driven process in a scalable and lock free manner thanks to the Storm framework.
Storm is an event based data processing engine. Its model relies on basic primitives like event transformation, filtering, aggregation… that we assemble into topologies. The execution of a topology is typically distributed over several nodes and a storm cluster can also execute several instances of a given topology in parallel. At design time, it’s thus important to have in mind which Storm primitives execute with partition scope, i.e. at the level of one cluster node, and which ones are cluster-wide Read the rest of this entry »
Play 2 has been announced a few weeks ago.
I am currently working on a project based on version 1 of this framework, as I am envisaging a migration, I decided to write two blog posts about this experience. The text below contains my personal thoughts on Play 1, I’ll try at a later stage to hunt for some time to use this as a comparison point for my feed-back on Play 2 and the migration process (if I confirm it should happen…)
Here are some major characteristics of Play (just check their online docs for more details, that is not the point of this post):
- Web framework for Java and/or Scala
- Not based on servlet, although a project can be converted to a war package
- No server-side session
- Java controller methods are public static void
I am developping a small web app for a few months during my free time. I used to deploy it to the AWS Beanstalk platform, today I’ve been amazed how easy it has been to migrate it to Cloud Foundry :-).
My app is mostly based on JSF2/jquery/Spring/Jackson + the AWS-specific APIs for storing data into AWS SDB and S3.
The migration to Cloudfoundry is the most straightforward migration experience I’ve seen up to now, it boiled down to: Read the rest of this entry »
I am presenting here a simple two steps architectural approach based on stored events as a workaround for the lack of full atomic transaction support in so-called “NOSQL” databases.
Being fairly new to NOSQL-based architectures, I have the annoying intuition that I am about to write nothing but a set of obvious statements. On the other hand, I have not yet read a detailed description of this anywhere, so hopefully it will be useful to some other developers as well.
[Edit: 1rst Oct 2012]: look also at the slides of Nathan Marz’ recent presentation on event based “Big Data architecture”: http://www.slideshare.net/nathanmarz/runaway-complexity-in-big-data-and-a-plan-to-stop-it, this one has some similarities with what I present below and is very clear to understand.
NOSQL databases do not offer atomic transactions over several update operations on different data entities(*). A simplistic explanation to this is that Read the rest of this entry »
In most non-trivial SOA landscapes, keeping track of the constantly evolving integrations among systems can be hard unless there is in place a clearly identified way to publish and find the appropriate pieces of information. An overview of the IT landscape, defining what is currently or will be connected to what, is a prerequisite for being able to maintain the environment. Absence of this typically leads to a feeling of “Spaghetti Oriented Environment” and reluctance to start anything big.
This statement sounds obvious but it is not always taken into account in practice. Some organizations either do not have in place such a centralized control of integration or have stopped using it because it “just got in the way of anything”. At best, this means that the integration information is kept in the head of some key Read the rest of this entry »