Thoughts on Architecture and Software Design

To be honest and fair, these are not my thoughts but great principles I have learnt in my programming career. Its a unassorted list of references that I keep going back to, when discussing architectural concepts with development teams. Penning it here as my own reference.

Service Oriented Design

Services are the building blocks of architecture. They need to be simple, self contained and designed for re-use; right from the start. Application features expose their data and functionality through service interfaces. Services communicate with each other only through the interface over the network (Http / Messaging). No other communication allowed - binary dependency, direct data store access, shared memory model and all other backdoor access should be disallowed. All services should be designed to be externalizable.

Ref: https://plus.google.com/app/basic/stream/z12ld3fwhlnexv5b004cjv0oapmuflzrezc0k

Design Paradigms

API Driven - Abstraction is the key component of providing an integrated service offering. Applications needs to be built on top of a rich API which assists down the line with integration with downstream and upstream systems and implementations can be changed without impact.

Scalable - Applications should be decomposed into small, loosely coupled, stateless building blocks that can be scaled independently. Architecture should always bear cost in mind and allow business levers to scale both horizontally and vertically as necessary.

Adaptive - Dont make applications more complicated than they should ever be. Follow the KISS principle. Look for Apple product design guidelines as an inspiration to make things as intuitive as possible.

Resilient - Protecting your organization's / customer's data is the first priority for applications. Applications must be able to absorb individual failures and provide a seamless degradation of service based on availability. Don't treat failure as an exception but as an event of regular happening.

Secure - Integrating security into applications should be a ground-up activity and not an afterthought. Enterprise level of authentication, access-controls and data encryption should be applied from basic design.

Ref:
Embracing uncertainty
Clean architecture
Distributed Systems
Designing a good API
Design for failure - The Simian Army
Frequency reduces difficulty

Continuous Delivery

Principles vs Practices - Best practices are subjective and depend largely on context, while principles are eternal and universal. The delivery lifecycle should be based on fundamental 'agile manifesto' rather than beating around on specific development practices. - http://simpleprogrammer.com/2013/02/17/principles-are-timeless-best-practices-are-fads/

Continuous business value vs Time-boxed iterations - The delivery should be based on providing continuous business value in the form of small functional releases rather than arbitrary time boxes. - http://kief.com/iterations-considered-harmful.html

Measurements and analytics - You really have to treat the software development process more like a relationship than like a factory. Instead of focussing on measurements and metrics, the emphasis should be given to trends and variance of the development lifecycle. - http://simpleprogrammer.com/2013/02/11/we-cant-measure-anything-in-software-development/

Philosophy on Estimations - The key is that you first accept that making accurate long-term estimates is fundamentally impossible. - http://blog.hut8labs.com/coding-fast-and-slow.html

Simple Conferencing with Twilio, BlockSpring and AWS

Recently an entrepreneur friend of mine complained that plain teleconferencing services are expensive and in certain business situations, it is not possible to use social applications like Skype or Google Hangouts. To quantify it, he said an hour call between 4 people in the US and UK cost around ~ GBP 15 or around $23 at professional providers like GotoMeeting or Powwownow. Having known about Twilio and the power of Cloud, I felt this was not right and set about to write a simple conferencing facility POC for him which turned to be a huge cost saver. Interestingly a TollFree number costs about $2 per month and a local number about 1$, that comes to around 4$ per month recurring or admin cost as he decided to have a TollFree and Local number in the US and just a local number in the UK. A similar conference for 4 people now only costed $2.5 per hour and it was super efficient, also when we added a web based calling to it, the cost flattened around a dollar or less.

One of the salient features of the POC was that I decided not to build any backend application and use BlockSpring one of the recent discoveries for me to build a function that can allow to integrate Twilio. And use only Amazon AWS S3 to host a static web page which is a client-side Javascript application as the end point. This post simply details the steps and the code that was used to build the POC and make it open to anyone who would like to try on their own.

1. Create a static website using Amazon S3

You can refer to the official guide or follow this simple tutorial to create a static website on Amazon S3. Also you can setup a custom domain name like simple-conference.mydomain.com if that is something you'd prefer.

2. Register two applications with Twilio

Sign-up for a new account at Twilio if you already did not have one and do spend sometime looking at the self-explanatory guides and documentation which are very useful. For the conferencing app, we will create two application end-points, one to act as an administrator account which would begin and end the conference and the second one adds users as participants to the call.

NOTE: Please remember that we are using Amazon S3 to host our TwiML end-points, so the request URL must have the "HTTP GET" method, otherwise you'd have exceptions.

3. Add the following code end-points to S3 website

4. Create an account at BlockSpring

BlockSpring allows developers to create, run and share functions across multiple server side languages including PHP, Python, NodeJs, Ruby, R etc. Though the following function is open-source and can be called from your application, you will still need to register an account and get an API Key of your own to connect to the function. 

5. Use the Twilio Token Generator Function from BlockSpring

You can use my public open-source function hosted at "https://api.blockspring.com/vshank77/82c3cb17747dcc677ab5816969962f91".

Since at the time of this writing, there is no integration between Github GIST and BlockSpring or way to embed a BlockSpring function, I have elaborated the source code of the Twilio.

6. Create the Javascript App and add it your S3 website

Please note that the account ID / Application ID in the following code are hard-coded to test accounts and will not work. You will need to use your own Twilio Account ID and Begin/Join App_ID when you upload this code.

7. Optionally associate calling numbers to the Twilio Function

If you also need to connect to the application from regular phone number, buy as many numbers as you want and point them to the "Simple Conference Join" TwiML function. The only caveat is that since we don't have any backend service of our own, you can create administrator functions on the phone and will have to use web site to begin the conference.

That's it; You now have a fully functioning conference at a very low cost.

Hard reverting a Maven release with Git

When trying to release a maven project, Git automatically tags your repository and then updates the project to a new version. However if there are any failures in between, the release cannot be fully undone and needs manual intervention to clean up the state.

The following code snippet shows how you can fully revert your project by removing both the tags and resetting the previous commit.

New Age Architecture principles learnt from AWS re: Invent keynote

Was listening to the Keynote from Werner Vogels at AWS re: Invent and wanted to record all the concepts that he discussed. These are so valuable and applicable for large applications in today's environment and worth blogging about.

Fundamental Principle - EVERYTHING IS A PROGRAMMABLE RESOURCE 

Controllable

• Decompose into small, loosely coupled, stateless building blocks 
• Automate processes and services
• Let business levers control the system
• Architect with cost in mind

Resilient

• Protecting your customer is the first priority
• In production, deploy to at least two availability zones
• Integrate security into your application from the group up
• Build, test, integrate and deploy continuously
• Don't think in single failures
• Don't treat failure as an exception

Adaptive

• Entla non sunt multiplicanda praeter necessitiatem - dont make things more complicated than it should be 
• Assume Nothing
• Use late binding

Data Driven

• Instrument everything, all the time
• Inspect the whole distribution
• Put everthing in logs

Some of the quotable quotes from the session include

"you are allowed to make mistakes"
"if you cannot collect data, you cannot act"
"whole distribution not average or median - inspect 99.9 percentile"
"Thou shalt turn off the lights"

Introducing Polyglotted

My name is Shankar Vasudevan, a developer and architect, working for a leading investment bank within the London city. I have been architecting large scale enterprise applications for the past 15 years. You can visit my professional profile on LinkedIn or follow me @vshank77 on Twitter.

Over the past 15 years, I have grown a great fascination for data storage and analysis. The advent of Big Data and NoSQL in the recent years have got me drooling and my professional work in the fields of information retrieval, analytical processing, adword classification and scoring, enterprise search have added fuel to the fire. More recently I have become a true believer of Polyglot Persistence a keyword coined by Martin Fowler and apply the philosophy in day-to-day job.

Polyglotted.org is to give back to the community, projects that I conceive in the area of information analysis and polyglot persistence.

If you are still interested, Martin Fowler has an interesting introduction to NoSQL Databases and Polyglot Persistence

Cross-posted from Polyglotted Blog

JMockit & JavaAgent - Enterprise integration woes

We've been using JMockit as the preferred mocking utility at a large Bank that I'm consulting at. While generally the product and its non-intrusive mocking features are commendable, some of the under-the-hood mechanisms have caused us a nightmare with our continuous integration.After upgrading to latest version of JMockit, version 0.996.0, we had ALL our unit tests failing with the following spurious error

java.lang.NoClassDefFoundError: org.junit.runner.Runner at org.junit.runners.model.RunnerBuilder.safeRunnerForClass(RunnerBuilder.java:59) at org.junit.internal.requests.ClassRequest.getRunner(ClassRequest.java:24) at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.(JUnit4TestReference.java:29) at org.eclipse.jdt.internal.junit4.runner.JUnit4TestClassReference.(JUnit4TestClassReference.java:25) at org.eclipse.jdt.internal.junit4.runner.JUnit4TestLoader.createTest(JUnit4TestLoader.java:40) at org.eclipse.jdt.internal.junit4.runner.JUnit4TestLoader.loadTests(JUnit4TestLoader.java:30) at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:452) at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:683) at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:390) at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:197)

While my colleague who had been advocating the use of the utility asked me for a way to change the order of libraries in the classpath, we smelt rot and started debugging under the hood. After quite a bit of digging around, we found that the actual error was caused by the failure to load the JMockit agent dynamically. The following code is the sequence of calls which JMockit makes to load the agent.

 public class Startup {   // other code omitted    public static void verifyInitialization()   {      if (instrumentation == null) {         new AgentInitialization().initializeAccordingToJDKVersion();      }   }}public final class AgentInitialization{   public void initializeAccordingToJDKVersion()   {      String jarFilePath = discoverPathToJarFile();      if (Startup.jdk6OrLater) {         new JDK6AgentLoader(jarFilePath).loadAgent();      }      else if ("1.5".equals(Startup.javaSpecVersion)) {         throw new IllegalStateException(            "JMockit has not been initialized. Check that your Java 5 VM has been started " +            "with the -javaagent:" + jarFilePath + " command line option.");      }      else {         throw new IllegalStateException("JMockit requires a Java 5 VM or later.");      }   }   private String discoverPathToJarFile()   {      CodeSource codeSource = AgentInitialization.class.getProtectionDomain().getCodeSource();      if (codeSource == null) {         return findPathToJarFileFromClasspath();      }      URI jarFileURI; // URI is needed to deal with spaces and non-ASCII characters      try {         jarFileURI = codeSource.getLocation().toURI();      }      catch (URISyntaxException e) {         throw new RuntimeException(e);      }      return new File(jarFileURI).getPath();   }   private String findPathToJarFileFromClasspath()   {      String[] classPath = System.getProperty("java.class.path").split(File.pathSeparator);      for (String cpEntry : classPath) {         if (cpEntry.matches(".*jmockit[-.\\d]*.jar")) {            return cpEntry;         }      }      return null;   }}

While this approach is non-intrusive and works out of the box in most cases, in large enterprise organisations, you are often limited extensively and work in a confined environment. My current environment loads files from a network file system and the following lines from the discoverPathToJarFile() cause the class initialization error

 jarFileURI = codeSource.getLocation().toURI(); 

returns

 CodeSourcefile://remoterepository/libraries/jmockit/0.996.0/install/common/lib/jmockit.jar 

and

 return new File(jarFileURI).getPath(); 

returns

 IllegalArgumentException: URI has an authority component 

We had been using JMockit for a while now since 0.991.0 version and the agent loading had always been a problem. In our original usage, we had to do two things. We had to create our own version of the JDK6AgentLoader as this class has package level scope and create our own JUnit4 runner which loaded the agent from a hard-coded location, as below

public final class MyJMockitRunner extends BlockJUnit4ClassRunner{   static   {      MyJDK6AgentLoader.loadAgent(hardCodedPathtoAgentJar);   }   public JMockit(Class testClass) throws InitializationError   {      super(testClass);   }}

Why the problem now?
What changed majorly between the two versions was, Shadowing the org.junit.runner.Runner class which in the modified version loaded the java agent before initialization of the class.

public abstract class Runner implements Describable{   static   {      Startup.initializeIfNeeded();   }   public abstract Description getDescription();   public abstract void run(RunNotifier notifier);   public int testCount()   {      return getDescription().testCount();   }}

So even though we are trying to load the agent in our custom Runner, the loading of the Runner class itself preceding the sequence and aggressively loads the agent in its own strategy.While this is a possible approach, I would highly criticize the usage of class shadowing for the following reasons

1. Open source API generally provide certain basic guarantees, and is generally tested extensively by the commnuity in different enviroments to ensure they work in the way they are designed. By shadowing a public class, you override this guarantee and for most common developers, the fact that they are using a modified class is oblivious. This can lead to late night debugging sessions and nightmares when things go wrong.
2. In this particular case, while only the tests that used JMockit should have failed, thousands of test cases across multiple projects failed at once. As a programmer, I always believe to lazily load resources and only when they are needed. But there is a necessity for mocking or not, the agent is going to be loaded whenever a Unit test is run. This is fundamentally wrong.
3. Any workarounds that had been applied to the library fail miserably with this aggressive approach to force your library to behave in a specific way.
4. While I agree that the owner cannot foresee all the environments his library would be used, trying to perform gimmicks under the hood on an open library could have been avoided.

Solution
While I've criticized the usage of shadowing a class, unfortunately the only solution that worked for us was to shadow the JMockit class that loaded the agent as below.

public final class AgentInitialization{   public void initializeAccordingToJDKVersion()   { MyJDK6AgentLoader.loadAgent(hardCodedPathtoAgentJar);   }}

Better approach
There are a couple of suggestions for the JMockit API to overcome the shortcomings described above

1. Please do remove the shadowed JUnit class and make the agent loading explicit. This is already the case when you use the JMockit.class as the runner.
2. While the auto detection of the agent is brilliant, there could be some hooking mechanism which allowed the developers to extend it with their own strategies.

Last but not least, there definitely was an obvious solution to the problem, which was to explicitly specify the agent path in the VM arguments. The issue we had in that was there were too many OS, at least a few IDE, a shared continuous integration server and many many ant build scripts that need to be updated. It would have been a far more involved effort to get all this modified and maintained than applying a code fix which worked "under the hood"

Google Appengine and Maven - another hack for datanucleus plugin

After researching for mavenizing an Appengine pet project, most of the links that Google search returned were really old when first version of AppEngine was released and did not match the expectations. Also unfortunately there is really no maven plugin that supports the latest version as Google publishes them or a plugin that allows you to point to a local installation, so that you could update at the speed of Google releasing updates for AppEngine (as of this writing, there were atleast 3 updates in the shorter period of 2 weeks)

Thanks to Salient Point, this was the best article to configure a maven project and it allowed to configure any AppEngine version with the simplicity of changing a couple of variables in a script file and building the maven project. However the only caveat that needed a few hours of debugging was the maven-datanucleus-plugin, as again the versions of the Jar files provided by datanucleus repository and the ones that are published by Google were slightly different and it was more simpler to use the same philosophy of having all the Jars provided by Google rather than downloading them from a different repository.

On this note, the following are the only deviation of configurations from the original post 1) Define the local directory to which the AppEngine Jars have been downloaded, the preferred way is to use profiles such that we could specify a default location in each platform (windows / mac)

        <profile>
            <id>gwt-dev-windows</id>
            <properties>
                <platform>windows</platform>
                <appengine.sdk.root>C:\Java\appengine-java-sdk-${appengineVersion}</appengine.sdk.root>
            </properties>
            <activation>
                <activeByDefault>false</activeByDefault>
                <os>
                    <family>windows</family>
                </os>
            </activation>
        </profile>

2) Specify a antrun task that performs the datanucleus enhancement for entity objects rather than the datanucleus plugin

            <plugin>
                <artifactId>maven-antrun-plugin</artifactId>
                <executions>
                    <execution>
                        <id>datanucleus-enhance</id>
                        <phase>process-classes</phase>
                        <configuration>
                            <tasks>
                                <property name="appengine.tools.classpath"
                                    location="${appengine.sdk.root}/lib/appengine-tools-api.jar"/>
                                <property name="dependency_classpath" refid="maven.dependency.classpath"/>

                                <taskdef name="enhance" classname="com.google.appengine.tools.enhancer.EnhancerTask"
                                    classpath="${appengine.tools.classpath}"/>
                                <enhance failonerror="true" api="JPA">
                                    <classpath>
                                        <pathelement path="${appengine.tools.classpath}"/>
                                        <pathelement path="${dependency_classpath}"/>
                                        <pathelement path="target/classes"/>
                                    </classpath>
                                    <fileset dir="target/classes" includes="**/*.class"/>
                                </enhance>
                            </tasks>
                        </configuration>
                        <goals>
                            <goal>run</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>

Alas it works like magic for us and we are able to upgrade the AppEngine versions as and when they are released by Google.

Also to note is the link on configuring a multi-project setup that is documented on the maven-gwt-plugin documentation that helps to organize your code into multiple projects (separating the core API / UI / service implementation and AppEngine deployment). Refer to our sample project setup that is hosted at http://code.google.com/p/scrumsp

Builder Pattern Abused

Everyone has read Joshua Bloch's builder pattern, which he has also described at his "Effective Java Reloaded" sessions at Javaone. However in this post, I am going to describe a misuse of this pattern, when used in conjunction with inheritance. 
When building a relational model, the biggest challenge is to maintain the integrity of inheritance and reduce the visibility of the properties completely. However if you need builders for this model, then you either have to create a class hierarchy of builders similar to the model itself or make all accesses protected (ugly #1) so that builders could appropriately initialize them. Making the properties less visible has at least two disadvantages though (1) you cannot make them final and ensure construction safe and (2) you'll end up copying the builders to more than one class (ugly #2) I recently came across a horrendous implementation of the hierarchy of builders where the author could not solve the above issues elegantly. So he had resorted to creating copy constructors (ugly #3) for the builders and also duplicating the methods for each of the builders. The following code snippet exactly reproduces the code, in an example class hierarchy and its builders.

abstract class Shape {
    private String name;
    private Dimension dimension;

    public enum Dimension {
        TWO_D, THREE_D;
    }

    public static abstract class ShapeBuilder {
        private String name;
        private Dimension dimension;

        protected ShapeBuilder copyOf(ShapeBuilder builder) {
            ShapeBuilder copy = createBuilder();
            copy.name = builder.name;
            copy.dimension = builder.dimension;
            return copy;
        }
        public ShapeBuilder name(String name) {
            ShapeBuilder bldr = copyOf(this);
            bldr.name = name;
            return bldr;
        }
        public ShapeBuilder dimension(Dimension dimension) {
            ShapeBuilder bldr = copyOf(this);
            bldr.dimension = dimension;
            return bldr;
        }
        public Shape build() {
            Shape s = createInstance();
            s.name = this.name;
            s.dimension = this.dimension;
            return s;
        }
        abstract Shape createInstance();
        abstract ShapeBuilder createBuilder();
    }
    // omitting getters for clarity
}
abstract class TwoDShape extends Shape {
    private double area;

    public static abstract class TwoDShapeBuilder extends ShapeBuilder {
        private double area;

        protected TwoDShapeBuilder copyOf(TwoDShapeBuilder builder) {
            TwoDShapeBuilder copy = (TwoDShapeBuilder) super.copyOf(builder);
            copy.area = this.area;
            return copy;
        }
        public TwoDShapeBuilder name(String name) {
            return (TwoDShapeBuilder) super.name(name);
        }
        public TwoDShapeBuilder dimension(Dimension dimension) {
            return (TwoDShapeBuilder) super.dimension(dimension);
        }
        public TwoDShapeBuilder area(double area) {
            TwoDShapeBuilder bldr = copyOf(this);
            bldr.area = area;
            return bldr;
        }
        public TwoDShape build() {
            TwoDShape s = (TwoDShape) super.build();
            s.area = this.area;
            return s;
        }
    }
}
class Circle extends TwoDShape {
    private double radius;

    public static class CircleBuilder extends TwoDShapeBuilder {
        private double radius;

        protected CircleBuilder copyOf(CircleBuilder builder) {
            CircleBuilder copy = (CircleBuilder) super.copyOf(builder);
            copy.radius = this.radius;
            return copy;
        }
        public CircleBuilder name(String name) {
            return (CircleBuilder) super.name(name);
        }
        public CircleBuilder dimension(Dimension dimension) {
            return (CircleBuilder) super.dimension(dimension);
        }
        public CircleBuilder area(double area) {
            return (CircleBuilder) super.area(area);
        }
        public CircleBuilder radius(double radius) {
            CircleBuilder bldr = copyOf(this);
            bldr.radius = radius;
            return bldr;
        }
        public Circle build() {
            Circle s = (Circle) super.build();
            s.radius = this.radius;
            return s;
        }
        @Override
        protected Shape createInstance() {
            return new Circle();
        }
        @Override
        protected ShapeBuilder createBuilder() {
            return new CircleBuilder();
        }
    }
}
abstract class ThreeDShape extends Shape {
    private double volume;
    // builder similar to 2DShape
}
class Cube extends ThreeDShape {
    private int side;
    // builder similar to Circle
}

Now extrapolate this class into an enterprise relational model consisting of 10-15 objects with nested hierarchies and you can imagine the amount of stupid code that's been written. If you haven't solved this problem yourself before, please stop here and think for a solution. The biggest issue here is, when you create a CircleBuilder and call the name() method, you are again expecting a CircleBuilder back and not a ShapeBuilder. Thankfully there is a generic implementation (partly under documented) which helps to solve problems of this nature. The modified code is given below

abstract class Shape {
    private String name;
    private Dimension dimension;

    public enum Dimension {
        TWO_D, THREE_D;
    }

    public static abstract class ShapeBuilder<S extends Shape, SB extends ShapeBuilder<S, SB>> {
        private String name;
        private Dimension dimension;

        @SuppressWarnings("unchecked")
        public SB name(String name) {
            this.name = name;
            return (SB) this;
        }
        @SuppressWarnings("unchecked")
        public SB dimension(Dimension dimension) {
            this.dimension = dimension;
            return (SB) this;
        }
        public T build() {
            T result = createInstance();
            result.name = this.name;
            result.dimension = this.dimension;
            return result;
        }
        abstract T createInstance();
    }
}
abstract class TwoDShape extends Shape {
    private double area;

    public static abstract class TwoDShapeBuilder<S extends TwoDShape, SB extends TwoDShapeBuilder<S, SB>>
            extends ShapeBuilder<T, SB> {
        private double area;

        @SuppressWarnings("unchecked")
        public SB area(double area) {
            this.area = area;
            return (SB) this;
        }
        public T build() {
            T result = (T) super.build();
            result.area = this.area;
            return result;
        }
    }
}
class Circle extends TwoDShape {
    private double radius;

    public static class CircleBuilder extends
            TwoDShapeBuilder<Circle, CircleBuilder> {
        private double radius;

        public CircleBuilder radius(double radius) {
            this.radius = radius;
            return this;
        }
        public Circle build() {
            Circle s = (Circle) super.build();
            s.radius = this.radius;
            return s;
        }
        @Override
        protected Circle createInstance() {
            return new Circle();
        }
    }
}

The only thing ugly about this code is that the builders themselves have to be cast to the generic implementation because of the type erasure. However this would be solved by the new "Type Inference" language feature that will be introduced in Java 7.