Estimating Data Warehouse Project

I have always wondered what is a realistic expectation for the time to design and implement a data warehouse? I am managing a Data warehouse project in an agile environment where the key idea is to deliver business value frequently. The initial estimate to deliver complete DW was in 6 months which I was always felt was too short. There are few challenges with this project;

  • Team members were new to agile development with strong opinion about non agile delivery of DW
  • Legacy systems which has grown organically over a period of decade( partially documented)
  • Source system which was build using non English language
  • Key business decisions were being made using spreadsheets produced by few people

In a meeting with management, team committed to deliver this project in 6 months which I felt was bit ambitious. With the help of Google I came across few estimation techniques which I will be referring here. In any traditional organisation the CIO/CFO would be interested in knowing how long will it take and how much will it cost. It’s a chicken and egg kinda situation, Since the project is not approved therefore you cannot hire experts to estimates and without an estimate your CFO will not approve the project. In order to make life easy I decided to divide this problem into two parts:

  • Pre approval estimates
  • Post approval estimates

After some research I came up with the following simple tool( thanks to experts who shared their knowledge).

[ws_table id=”6″]

This will give you a high level estimate which can be used for approval from Project sponsors. Once project is approved, start building your product backlog (detailed requirements) then go through agile estimation process to come up with better estimates. If you need more information or want to organise a workshop to estimate your product back log the get in touch with sales team( and they would be able to help you in organising it.

Happy estimating



Is there a point in story point estimation?

Is there a point in story point estimation? How do we estimate accurately?

This is a relative size of the story compared to other story estimated by the same team – it has nothing to do with who is implementing it and how long it’s going to take. Story points are relative estimate, so that you know that 5 point story will take approx. 5 times longer than 1 point story.
Benefits are

  • Team’s velocity can be measured
  • Allows team to plan future sprint without over/under committing and forecasting total number of sprints. As a result no unrealistic expectations are placed on the team
  • Helps team to focus on completing dev and testing in the same sprint
  • Helps teams reach a sustainable pace and due to this the business starts to believe in the teamHours (or ideal hours) are more about time estimation. It can lead to several problems:
  • Your “hours” are not the same as mine. It can be harder to make team estimate during release planning.
  • It’s easier to make relative estimates and then calculate team velocity in points.
  • Stories are usually decomposed to tasks for the sprint. These tasks can be implemented by different people. So total effort is not simply calculated as sum of task estimates.
Usually it’s recommended to make story estimation in points for release backlogs and in hours for sprint backlog.
  1. Story points are a pure measure of size and complexity
  2. Story points are relative (say, with respect to the simplest story) and so have a much longer shelf-life
  3. Story points are usually independent of who gives the estimate (as in, an experienced developer and an apprentice can usually agree on something like complexity fairly quickly)
  4. Story points avoid the need for discussions like “what are *ideal* hours, really?” or “My ideal hours are different from your ideal hours, stupid.” These add no value.
  5. Story points don’t influence behaviour (e.g. Parkinson’s Law)
  6. Story points are easier to work with – especially when product owners start to wonder why “3 ideal days take a week…”
  7. Story points are more fun – especially when they’re in units like gummy-bears, polar bears, or other endangered species.
, ,

Agile team and home working

Agile and home working seems to be at odds with each other. One is about close communication, co-location and short feedback loops; the other is about being effective with people working away from their usual office location. If terms like “Slow network”, “communication gap”, “different time zones”, “different work culture” sound familiar then you too are going through what I am going through these days.

The purpose of this discussion is to find out if a team can be agile while teams are distributed or few team members prefer to work from home. How do we create an effective and efficient team and still deliver quickly? What are the guidelines around home working?

We do have a need for people to be in the office, and it isn’t always possible. There are certain advantages and disadvantages of working away from the team.


Home working is mostly advantageous for the worker:

  • More Flexibility: This gives employees flexibility to sort out their personal work while at work. When working from home, your schedule at home is much easier to bend. It’s much easier to tie in your personal schedule with your work schedule when you work from home.
  • Less Transportation/commuting: This can help you get a little bit more sleep in the morning and have more time for yourself in the evening for activities & leisure. Arguably the biggest money saving benefit from working at home is the transportation cost savings.
  • Less stress & anxiety: An office can bring about a lot of competition, gossip, and backstabbing among co-workers. This type of environment takes its toll after a while both mentally and physically. Working from home will save you these headaches.
  • Increased productivity: With all the time that you are saving from not having to commute to work, and not socialising half the day with co-workers, you end up getting a lot more done when you are working from home. As long as you remain motivated and driven, working from home is much more effective to deliver results (provided there is no distraction from family members or pet).
  • Closer relationship: Working from home allows you to have such freedom and increase the chances of spending quality time with your loved ones which you just cannot put a price tag on.


  • Broken communication: Critical key to home working is the team member remaining in full contact with the team. This presents a number of difficulties with communication. Although you are only a phone call/email away, still direct discussion with team member can be more effective.
  • Reduced Productivity: My team had problems when a testing team member was based in South Africa. The internet connection was very slow (bandwidth issue) and it took ages to upload or download document (test script etc). The VPN connection timed out while testing software.  Other distractions can also reduce productivity.
  • Face2Face: When there is a need for people to be in the office for the key meetings, such as Sprint Planning, workshops, User Stories elaboration and estimation sessions where conversation can become many voices and fast paced. In this situation teleconferencing is least effective.
  • Emotionless: Working from home can restrict feeling of emotions other team members are going through, especially when system is going live or found a bug which needs immediate attention or just a taste of cup-cake baked by a team member to celebrate birthday.
  • Increased resolution time: It takes longer to resolve problems when a particular team member is required to resolve a problem or issue.
  • Loss of synergy: Nothing dynamic and thus you loose the synergy of two or more people working together seamlessly in a dynamic way, especially when team members are doing pair programming.

Tools to facilitate co-location while working from home:

To overcome these problems, there are certain tools available in the market to facilitate effective and efficient home working:

Tools like Skype, MS Communicator and Live Meeting. We have had people working from home in the UK and abroad. These three pieces of software mentioned above can help to alleviate this.

Mountaingoat’s online poker tool to be good for geographically dispersed teams though the session has to be very well planned for in advance.

Use SharePoint/web based document share (i.e. google doc) for document storage and of course any web based agile planning tool helps greatly across distributed scrum teams.

SMART Board: This is something I have started using few years ago. It is expensive to buy but comes with numerous benefits. The external invitee has a pen tray with colours, shapes and an eraser to interact with board dynamically plus we can hear them over the speakers – vice versa with a microphone of course. The software provides two way interactions. Plus we can use web cam too. Consider the situation where we have developers, testers and Product Owners geographically dispersed and you want to do something simple like interact with a script, do some design clarification or even show some interaction on the front-end before signing a User Story off.

Consideration needs to be given to the role of the person who home works. For example a Development Lead who works from home/off shore 2 days a week and this can present difficulties in having to take the role of Proxy Development Lead on quite a few occasions. This is really evident at go-live when they are off shore and the team are in the office. I agree that technology can not replace human touch but we all need to compromise somewhere or other. Therefore I recommend frequent face 2 face meetings, especially during team retrospectives or during point planning session and other activities which don’t need much of interaction with others can be performed remotely and make sure you are available online to interact with the team when needed.

Any other rules?