Developing an online collaborate KJ tool with Google’s Web Toolkit and App Engine

The Problem

Last year I was participating in a Six Sigma project that was focused on requirement elicitation and risk reduction. One of the tools we were advised to run was a KJ (also known as an affinity diagram); a method of organising ideas into logical groups based on their relationships.

The problem was that one of our team members was based in India, we wanted to include him in the process but typically, due to the large amount of data, KJs are performed in meeting rooms with many post-it notes representing the ideas (voices) and a large canvas (or wall) to move and categorise them on – not very feasible with a 4.5k mile difference.

The Idea

Around about the same time, I discovered the Google App Engine and Google Web Toolkit – two awesome development products from the bright sparks over at Google. I really wanted to get my teeth into these tools, experimenting with Collaborative web applications and some of design patterns I just didn’t get exposed to in my professional career (MVP mostly).

I decided to try out the App Engine and Web toolkit in writing an app that would allow all KJ team members to see the Canvas on a laptop screen and walk through the KJ together (using a conference call or VOIP for discussion), every action a team member took would be replicated on every member’s screen – it should feel like they’re all in the room together but all over a web browser.

I couldn’t do this in Work’s time, after all, I have a day job as a Program Manager, so I picked it up as a bit of fun at home.

High Level Use Cases

These are the very high level cases I proved out.

Authenticate with a Google Account

The tool should permit a user to authenticate with an already existing Google Account.

Import Data from an Online Survey

The tool should be capable of integration with Google Documents contained on the authenticated account. It should allow the importing of data collected from Google Document Forms.

Engage in text-based communication

Users participating in a session should be able to communicate in a “chat room”.

Reduce quantity of visible statements

Any user in the session should be able to mark a statement for including in later phases of the KJ; marking a statement should result in the displaying of a “red dot” (simulating a red marker pen dotted on a post-it note) on all user’s screens immediately. Statements with no dot should be hidden from future phases.

Rewrite content of statements

Text on a statement should be able to be modified by any user in the KJ, changes should be reflected throughout the participating users immediately.

Translate Statements

Text on a statement must be rewritten by the team (or just copied), changes should be reflected throughout the participating users immediately.

Group Statements

Statements must be dragged and dropped together, the statement being dragged should not be visible once dropped. The action of dropping relates the two statements together within the same context. There should be two phases of grouping, the first generating “red” groups, the second generating “blue” groups. The target of the dropped statement should change colour to reflect it’s level.

Export the Data

The results of the KJ should be exported to Excel and OpenOffice format, visualising the relationships between the various groups.

Patterns and Technology implemented

Model-View-Presenter (MVP)

All development was modelled on the Model-View-Presenter design pattern, this enforces the separation of logic between the data (model), the user interface (view) and presenter (domain logic). It’s a great pattern and I fully encourage any developer who hasn’t been exposed to it, or one of it’s related patterns (MVC), to check it out.

Activities and Places

This is an extension of the MVP pattern and handles the invocation of views and presenters based on tokens on the URL, it’s mostly about history management – it was a little clunky and not what I’m used to at all but once I got into it’s statelessness and lack of GET request handling, it was pretty handy.

Google App Engine

All application logic was implemented using Asynchronous GWT RPC calls to the Google App Engine, which was hosing my services and Data store.

I used the Channel API to achieve the collaborative aspect of the application – the main focus of the application. This is very cool, essentially holding an open socket between the browser and server, allowing the server to send commands to the client; it’s a much, much better implementation than Comet based implementations but totally threw me in that the Development mode for GWT actually uses a Comet implementation. If you’re wondering why you deployed application isn’t behaving the same as development; this might be why!

Scalable Data Model

I wanted to get to grips with non-relational database, the App Engine was ideal for this, it’s primary data model is Entities (an elaborate HashMap implementation). It’s simple, fast, distributed and pretty awesome. With little effort (and a lot of forethought) you can build highly scalable, cloud-savvy applications.

The Use of Translation

I’d never run a KJ session before, nor participated in one, so I based the application logic off some cheat-sheets I’d received in training. It occurred to me mid-development that my interpretation of the translation phase was wrong. I’d interpreted it as the subject matter experts and Business Analysts in the KJ session reading and translating the voices generated from the user base into verbage that can be used within the grouping phases.

I was wrong wasn’t quite right. This strategy would work for KJ sessions not intended to solve problems, like the project I was working on; gathering and categorising new requirements. It wouldn’t be totally suitable for KJs aimed at the identification of problems and potential solutions.

The translation within a KJ is intended to map “Images” onto “Voices” and is used to bind together issues and solutions. I think a better use would have been logic similar to below: –

  • Decide on the intention of the KJ, a problem statement
  • Send a survey out to the user base with questions covering what the issues are (Images) and how improvements could be made (Voices)
  • Extract the Images (problems), take them through reduction, red level and blue level grouping.
  • Extract the Voices (solutions), take them through reduction, red level and blue level grouping.
  • Run a translation by getting the Red level statements from the Images and finding red level statements from the Voices that map
  • The result is 2 KJs, one representing what the problems are, one representing the potential solutions and a series of mappings between them – the translations

Next Steps

I’m not planning on continuing the tool past Alpha and the GAE Application is currently disabled. I proved out all the cases in GWT with GAE’s channel API (see the demo below) and had great fun doing it, plus we managed to include our Indian colleague in our KJ session – that’s good enough for me!

Early Alpha Demonstration

The following is an early Alpha demo of the tool, it’s not very pretty but is fully functional and got the job done. I’m simulating two users participating in a session with two windows; we ran the Alpha for our Project with 4 participants and it worked great.

http://www.youtube.com/watch?v=aSe7qfGwK3U

2 comments on “Developing an online collaborate KJ tool with Google’s Web Toolkit and App Engine

  1. Hi Rob,

    Given my general disdain for 6S and many of the associated processes – I have had to find time to squeeze into my life. Today – it happened – KJ over a distributed area.
    Is there a tool – no…you might be onto something with this Rob!

    Anyhow besides that, How are things with you?

    Games, Girl, Dog, House all well i hope

    Mark

    • Now then fella!

      Everything’s good mate, busy… very busy.

      Games are lacking, Girl is now Fiance, Dog’s well and house is improving; new carpets next week!

      How ’bout yourself? You contracting now or something?

Leave a Reply

Your email address will not be published. Required fields are marked *