I'm guessing that you're updating records. So you're resolving what's "truth" during the inbound operation. (For readers: data may be received out of order, so implementers must resolve the single source of truth in near real-time.)
Our systems used to do that.
My redesign eliminated updates, did only inserts, and resolved the "truth" during the query. Greatly simplified everything.
No more "historical" table, where deltas are logged. Determining what was known when (determining liability) queries were simply date bounded. No need to "rerun the data" when a mapping or business rule changed. Etc.
No updates, no real deletes either (only virtual ones). Data about transactions is stored separately from the clinical data, in an optimised store for log data.
I came up with deliver https://github.com/gerhard/deliver to address this very problem. It's bash utility that automates git-based deploys and comes with pre-built strategies for the most common deployment scenarios: generated sites (think Jekyll), shared (WordPress, PHP etc.), ruby, node-js, S3 etc. I did a talk on it at my London Ruby User Group in March: https://speakerdeck.com/gerhardlazu/deliver
That's very cool. I really dig the minimalism of it. I'm currently using a system that involves about the same amount of config and leans on Capistrano for the heavy lifting, but I think I might investigate using deliver for my next project.
We are behind howareyou.com, an online personal health record completely free to all NHS patients.
We are committed to building the best clinical API (https://api.howareyou.com/), highly available and secure. Our API is a collection of services distributed across multiple infrastructures (including AWS).
The right candidate will be interested in:
* RESTful APIs - including the hypermedia component
* non-web Ruby services - we’re big on EBI (Entity-Boundary-Interactor)
* non-Rails apps - we are slowly phasing out Rails
* client-side apps - our newest apps are all Angular.js talking to APIs
* Service Oriented Architecture
* hybrid infrastructure
* Redis and Riak - masterless clusters is where we’re heading
You will be working alongside the best in their field from Google Campus, less than 5 minutes walking distance from Old Street station.
For the best contractors, we are willing to pay £450 per day. All candidates are encouraged to get in touch with us on jobs.hn@howareyou.com