In a number of client environments I have often been amazed by the number of people that can be assigned to a project. Project Managers, Business Users, Business Analysts, Architects, Technical Analysts, Developers, Database Analysts, Database Administrators, Data Modelers, Subject Matter Experts, Testing specialists, Data Assurance people, Production and Operations People and of course a couple of people like me, the consultants.
While it is impossible for one person to do everything I often wonder how many people on such a project team could be removed from the project without impacting and perhaps improving the outcome.
One person I worked with recently proved that a single person acting alone can accomplish amazing things if they have access to all the right tools and know how to use them. In this scenario the end user need some reports - in excel format.
My friend, used UNIX scripts against flat files and SQL queries against the database to create a number of SAS data sets. He ran SAS functions against the data to aggregate it and exported it into Excel for the end user. It took him a couple of days.
In the mean time, I went to work building a slightly more robust solution. I needed to import the flat files into a database. But first, I had to have the appropriate tables built by the DBA in the development environment, then I had to get some one with access to the flat files to write some load utility scripts. Once the data was loaded it was real easy to build the SQL queries I needed because I had access to the data and knew how I wanted the queries to work. That was until I realized that the data in the development environment was different from the production data I was testing against and I had to bring a couple more tables into the database. Which meant involving the DBA and the Unix developer again.
In the end my little experiment took about two weeks to complete (about 5 times longer than the work my friend had to do to achieve the same result). Most of that time was spent waiting for other people to complete tasks I needed them to do before I could proceed.
Turning all of that into a formal project would have added 3 to 6 more people to the project team. To get the process automated fully and roled into production.
Most of the time the end user (usually the business) is perfectly happy with a solution as long as it delivers the results they are looking for on time. If we can meet the need and meet the deadline the business perceives us as proactive and delivering.
It’s systems people that complicate matters with over engineered automata that are no more accurate and some times less accurate than the quick and dirty solutions that the end users is often requesting.
I am not suggesting we abandon best practices in favour of data and systems anarchy. But I am suggesting that we should consider approaches to development of solutions that deliver results earlier and let the value of the information returned drive the investment into further development to improve the data and analytical process. I feel this can be accomplished at less cost than long, meticulous projects that often fail to deliver on the hyped-up expectations set by their promoters.
I believe a more appropriate approach is to deliver a simple report to the user. Set their expectations that the report generated was a one-off proof of concept and they should allocate a percentage of the value they receive from the informations provided be used to streamline and improve the BI process.
Comments