Skip to content

Big Data and Projects — What changes?

Data -- Picture of stock exchange tickers

Data -- Picture of stock exchange tickersI recently chaired an interesting webinar where Paul Gerrard of Gerrard Consulting spoke about ‘big data’.   Paul spoke about some interesting aspects relating to big data, but one particular item that resonated with me is the potential impact that an increased focus on big data analysis could have on projects.


Paul spoke about how the increased focus on data-driven decisions will lead to a renewed focus on organisations progressing smaller scale incremental changes to their systems and processes.  A focus on gathering insight, creating a hypothesis and testing that hypothesis.


This approach isn’t new – I’m sure that techniques like A/B testing and some of the statistical and continuous improvement techniques including DMAIC will be familiar to many readers.  Yet with many organisations now looking to embrace big data, it will be intriguing and interesting to see what this means for projects.


Here are five hypotheses of how it may impact projects in an organisation, and specifically what it might mean for the business analysis community and business analysts specifically.  However, I’d be extremely interested in hearing your views:


1.  More experimentation:  As Paul outlined in his webinar, organisations that employ big data solutions and use these to drive hypotheses and insight will almost certainly want to make smaller, more frequent incremental changes to their systems and processes.   This might mean that small change portfolios become more important; the ability to quickly implement and test a change becomes crucial.  Agility* becomes key and even more of a focus.  Business analysts will need to be closely tied into this process to assess the impact of those changes on the process, organisation, people, information & technology.   In many organisations,  this will be amplifying an existing pattern; the difference being that change becomes much more data driven, and a pattern of “plan/do/check the data” may become more normal. *And yes, I’ve said ‘agility’ not ‘agile’.  Let’s not open that can of worms just yet 🙂 


2. A need for discipline to avoid the ‘knee jerk’:  When data flows so quickly (velocity) and changes so often, it would be extremely easy to confuse correlation with causation.  The rise of the occupation of data scientist would, one would hope, help to avoid this.  However, involvement of the business analysis discipline more generally will become key.  Data needs to be interpreted so it becomes actionable and it needs to be read in its strategic context.  It’ll be important not to miss the ‘wood for the trees’, and current underlying business analysis practices will be as relevant as ever.


3. Not every project will be (or should be) in response to big data:  There will still be significant, strategic projects that aren’t just in response to insight from big data.   There will still be projects where minimising risk (as opposed to agility) are more important.  There will still be plan-driven, waterfall projects.  There will still be infrastructure upgrades, new project launches, launching into new markets and new process, system and organisational changes that aren’t just based on insight from big data.


4. Relationships and “soft skills” will be just as important as ever:  Data will never, ever, ever remove the need for soft skills and the ability to build rapport and liaise with senior board-level stakeholders.   Project practitioners including BAs will continue to need these skills, and will still need to navigate the labyrinth of politics that exists in many organisations!


5. Systems thinking and a holistic view remain crucial:  Assessing data, hypothesis and insight in the context of the overall business system – comprising of people, process, organisation, technology, values, culture, climate and so on remains a crucial factor.  Finding the ‘root cause’ of problems is as important as ever, and understanding problems, opportunities and impacts holistically is key.


So what is your view.  How will the uptake of ‘big data’ change the project world?  What will it mean for the role of BA, PM, Tester, Sponsor?  I’d love to hear from you, please go ahead and add a comment below, and if you like my blogplease subscribe.


If you didn’t see Paul’s webinar, you can view a recording on the IIBA UK SlideShare site



About the author:

Adrian Reed is Principal Consultant at Blackmetric Business Solutions, an organisation that offers Business Analysis consulting and training solutions. Adrian is a keen advocate of the analysis profession, and is constantly looking for ways of promoting the value that good analsis can bring.

To find out more about the training and consulting services offered at Blackmetric, please visit

Blackmetric Logo

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.