This blog is all about my experiences and insights as an Offshore Software Development Entrepreneur. It describes the inherent challenges that come with offshore development as well as the solutions we use at Ignite to create a high-performance cost-effective onsite-offshore software development model.

December 8, 2007

When Offshore meet Agile - Take 3

Agile development recognizes that in reality the development process is spiral and involves frequent "regressions" to earlier phases, and creates an iterative process that imitates reality. It breaks the project down into short development cycles, generally monthly or bi-monthly.

Each of these cycles covers detailed design, development and testing of the software segment - a working application with some of the desired functionality starting from the first milestone. Most important, immediate client feedback is required at the completion of each cycle. This feedback is feasible since milestone delivery includes a product, as opposed to fragmented pieces of compiled code, that provides a certain percentage of the desired functionality and can be checked against the requirements.

An iterative process is created that rapidly converges into the final requirements, the detailed design and the finished product. The client's involvement in the development process boosts his confidence in its quality already at the early stages. Also, the large number of development cycles and the corresponding large number of quality assurance cycles improve the software quality, despite the shorter development period.

Off course, everything has its price. The above iterative process demands high discipline by the development team members, skill sets that are easier discussed than effectively achieved (Test Automation, Continuous Integration) and significant overhead costs. In addition, a psychological gap exists that must be bridged. It is mentally difficult for R&D managers to adopt a process that recognizes that the requirements are likely to change.

How does Agile development methodology effects offshore projects in reality?

In a specific software project performed by us, it was required to set up a Provisioning system using a Business Process Management (BPM) platform. The project contained a high degree of uncertainty since the client did not fully recognize the capabilities of the BPM platform and did not properly define either the business processes or the system's functional requirements. At the start of the project, the offshore team was unfamiliar with the platform and the external systems with which the provisioning system had to interface. Instead of building a standard development process, according to which after six months we would proceed to integrate the systems and discover various problems, we built an Agile process based on monthly deliveries, as follows:

- The infrastructure of the provisioning system was defined and developed.
- A pilot provisioning process was defined and developed to give the client a feeling of the end-
user experience.
- A Web Service was developed for every external system with which the provisioning system
had to interface.
- The final provisioning process was defined and developed following the client's requirements.
- Additional capabilities were added such as Failover, Clustering, transaction management, error
handling, etc.

The client felt deeply involved in this development process. The monthly deliveries helped him to specify the final requirements as the development work progressed. The regular feedbacks enabled the offshore team to quickly grasp the client's business logic and part of the requirements were in fact proposed by the development team! The automatic testing process ensured that progressive changes made would not affect the product's quality and the subsequent integration phase was reasonably smooth.

For the sake of comparison, a previous Provisioning system developed by a different development team engaged by the client used twice as many resources and finished the project at a significantly lower level of quality .
October 13, 2007

When offshore meet Agile take-2

The best way to demonstrate the simple concept of agile development is through an example:

A few months ago I traveled to India and Thailand for my honeymoon. When my wife and I arrived to Bangkok, I decided (like many other business men visiting Thailand) to seize the opportunity and get myself a real "Armani-like" suit, tailored just for me by a local elite tailor.

We visited a number of stores that specialize in sewing suits (it seems that in Thailand most tailors and garment workers are usually Indian, Chinese or Burmese - the local Thai garment workers are too expensive...), until we finally arrived to a store we both agreed on.
When we were about to close the deal, the tailor informed us that we would have to come in the store almost every day, during the 10 day process of making the suit, for measuring and fittings.
My newlywed who has planned to pet tigers at the Tiger Temple in Kanchanaburi cried out:
"Why can't you do all the measurements on the first day and we'll come back on the 10th day to pick up the suit?"

"Ahhh..." answered the tailor with a smile, "and if we where to make a mistake and sew a smaller suit, or if the Mister will eat well during this week, will you still buy the suit???"
We decide to listen to our tailor, and during the next 10 days, I visited on a daily basis at his store: to start with, I was given the jacket without the sleeves, for fitting and measuring. The tailor spun around me with pins in one hand and a white chalk in the other. Later came the pants, and the day after, the jacket sleeves and then the first real fitting. Again, the tailor made adjustments, second fitting, more adjustments, third fitting... and we were happily on our way home, with a sharp looking suit. The project ended on time, at the expected budget, at the preset capacity and at the expected quality.

These are in fact the principals on which the agile methodology is based on:
- Collaborating with the client during the development process
- Dividing the project into short iterations and a checkpoint at the end of each one
- Adaptive to change (my waist line...) during development stage

Why does agile development gain such a big momentum in the development world today?

The old methodologies in software development are based on the Waterfall approach. In the Waterfall approach, during a project's first stage, more time is dedicated to analyzing requests and requirements. From the business requirements you obtain the technical requirements and the product's architecture.

The next stage is the Design phase, development/coding, testing and production. It's the same way architects create a project for building a bridge for instance.

Waterfall

Agile

The reason Agile development is becoming more and more popular is because a Software Development project is more similar to sewing a tailored made suit then to a project of building a bridge:

- The requirements of a Software Development project are more likely to change frequently.

- Software development is handicraft work (ambivalently!) and not automatic or mechanized
at its most, that is why human errors (bugs) are so common.

- It is very expensive to change software that does not fit the requirements, just like it's
expensive to change a suit that is too small for its owner...

- Software that did not arrive in the market on time becomes a white elephant, just like a suit
that was done after the client had returned home.

On my next posts I will provide a case study of a provisioning system developed by Ignite to demostrate how Agile methodology and and offshore development go hand in hand.

When offshore meet Agile take-1

Imagine a construction project in India implemented for an American client. The project has been estimated, priced and initiated although no building blue prints were produced. The client has never visited the construction site, nor has he met the project manager or any of the construction staff. The client has not yet firmly decided whether to build a residential tower or a shopping mall. In a regular weekly phone conversation, the Indian project manager generally reports that the project is progressing exactly on schedule. The workers' field of vision is restricted to the particular wall being built at the moment. After construction actually began, the floor structure was built using asbestos, but as work progressed it was switched to an innovative structural system based on the use of feather-weight cardboard panels. Would you want to buy an apartment (or shop) in this project?

Offshore outsourcing for software development is one of the most challenging fields handled by offshore outsourcing providers. Much has been written about the challenges involved in a software development process conducted by a third party contractor operating in another country. To be fair, the crux of the problem is largely in the software development process itself, which is essentially an art rather than a science.

Everything may look fine and complete on paper with most of the development process based on the Waterfall model. The project plan covers requirements analysis, software design, development, testing & quality assurance, integration, application and maintenance in a linear streamlined process. However, in reality much is lacking. The analysis of requirements is commonly inadequate. Furthermore, the market is very dynamic and requirements that were valid three months before are now irrelevant due to changing market conditions, constant technological change and new versions released by competitors. Moreover, the critical problem is that the client's feedback to the delivered products generally results in change requests and often also in regression to the requirements analysis phase. In the development phase, developers often return to the design phase and even to the requirements analysis phase. That is common experience.

These problems are accentuated when the work is done by a foreign outsourcing contractor operating in another country. In this context, a "broken telephone" syndrome is experienced due to geographic distance and mentality differences. Let's face it, people who see each other in the flesh communicate a lot better. As an example, in Hindi, "no problem" means that "there is a major problem but you are the customer and I cannot disagree". On the surface, the combined effects of the inherent problems in the development process subject to the offshore-related "broken telephone" syndrome seem to be insoluble.

Yet there are signs that there is a way to efficiently outsource software development projects to an offshore provider. When Offshore Outsourcing meets Agile development methodologies things look different.

In the following posts I will describe how we, at Ignite, utilize Agile development methodology to leverage offshore development.