How to Build an Effective Big Data Team

In the past decade some of the leading companies saw good-to-great transformations under direction of the most carefully selected CEOs, CTOs and CMOs whose dedication to work led to the stunning turnaround.

However, today’s businesses have to adhere to new rules in order to stay ahead of the competition. Most of these rules are due to the mass adoption of technological innovation and whether or not today’s corporate leadership adapts fast to the new business and social paradigm invoked by Big Data and predictive analytics will dictate the future route of any organization.

As pointed out by Erin Bartolo, head of the data science program at Syracuse University’s School of Information Studies, corporate C-Suite needs to understand data inside out, and understand the main principles of how it’s generated and managed in order to meet ever growing challenges, use new revenue sources and opportunities and comply with new business requirements.

Building effective Big Data teams within organizations is a new imperative no company will be able to avoid any time soon. Yet, unlike traditional IT / software development teams that fall primarily under the responsibilities of IT heads / PMs / Tech Leads, Big Data teams become a prerogative of CTOs, HR Directors and CMOs, as HR and marketing departments should be seamlessly embedded into technology teams to be able to successfully translate data into action plans and business intelligence (BI) that ensures high ROI, smarter decisions, higher customer satisfaction and loyalty, and overall potential for company growth.

So, what does it take to build an effective Big Data team?

To build your Big Data team, you’ll need experts in the following areas: advanced math, statistics, predictive analytics, high-performance computing, machine learning, economics and marketing.

Here’re things to remember when building your in-house or offshore Big Data team:

1. Define what exactly you’re trying to accomplish with your Big Data projects

You may spend a lot of time and money looking for and hiring “ninja” software developers and “rocket” scientists that will build your Big Data solution with beautiful interface and clear UI as well as robust back-end processing capabilities, but all of your investments will go down the drain if your company fails to execute on and gain value from the data points delivered by this solution!

Do you need your Big Data talent to boost your marketing campaigns, add merit to your market research or streamline and automate internal work processes? Depending on your needs, the size of and roles within your Big Data team will vary.

Once you’ve defined your Big Data roadmap and key goals, make sure to involve in the project all relevant departments that are to benefit from corporate Big Data, invite their heads to stand-up meetings and retrospectives, report progress on a regular basis and get their buy-in prior to implementing the solution. That’s when DevOps practices may come in handy for managing Big Data projects.

2. Hire a solutions architect

A technical architect is actually required for most of software development projects and Big Data is not an exception. The architect will conjure up all of the Big Data blueprints for your company to bring into life. Solution architects will help determine how Big Data should be best applied to company’s core business model, will help identify the most burning business questions as well as the availability of both internal and external data to answer these questions.

3. Hire senior software developers with particular backgrounds

Big Data can’t become real without software developers, that’s for sure. However, unlike typical software / mobile development projects, you should be looking to staff your team with those skilled particularly in experimental design, machine learning, predictive models, and statistics. And these skills normally come along with the seniority level and/or advanced education (e.g. Ph.D). You need your Big Data developers to design the algorithms that turn data into actionable items and produce key metrics for predictive models. That can be a rocket science for junior developers or geeks lacking the above expertise, which in turn will result in overheads and wasted budgets for your company.

It’s highly recommended that software engineers you’re seeking for your Big Data team have FinTech / MedTech backgrounds, as banking, finance and healthcare were some of the first industries to have started translating mountains of raw data into the executable actions and, thus, might have provided developers with the right skill sets in both technology development and quantitative analysis required for effective Big Data functioning.

Additionally, you may look for developers with experience building high-performance tech solutions for governments, oil and energy companies, aerospace and universities.

If finding tech talent with the above backgrounds is still a challenge for you, make sure to establish custom tailored Big Data development on-job training to turn your existing developers into Big Data specialists. Yet, in this case brace yourself for an extended Big Data development lifecycle as it can take 3 to 12 months to train your developers the right way and bring them up to speed. Outsourcing your Big Data project offshore / nearshore can be a good alternative, as you can get senior developers with the required backgrounds at a lower cost and, thus, reduce your time to hire and time to coach and accelerate your solution delivery!

Are you looking to hire Big Data specialists fast and with no HR and administrative hassle?

4. Don’t underestimate the soft skills

Geeks and data scientists usually speak a language different from non-tech folks. According to Russ Lange, founding partner of CMG Partners, “Their words are more binary and less nuanced.” Also, they use a lot of jargon when they speak to each other and can be misunderstood by other people.

As Big Data projects envision close collaboration with non-technology teams such as marketing, administrative or HR, at least Big Data Tech Lead should possess appropriate soft skills to facilitate communication with other departments and ensure developers and, say, marketers are on the same page when discussing the Big Data project. So, when looking for Tech Leads, pay attention to those with a track record of agility, innovation and even public speaking, as these guys will make it comfortable for both techies and non-techies to work together on the same project.

5. Retain your Big Data talent!

Any talent should be retained, but this is especially true for Big Data teams. As data analytics goes mainstream, the competition for data scientists and architects will become much more fierce and creative headhunting practices will be applied to outbid you.

However, competitive salary is just one piece of the puzzle. According to McKinsey, “to attract data scientists, companies need to put Big Data at the heart of the organization. If the analysts are nothing more than backroom number-crunches, they are unlikely to feel valued and will bolt for some company where they are.”

Bottom line is: keep your Big Data talent well-utilized and challenged with non-trivial solutions and make them privy to planning and design!

6. Make sure you have a competent CTO to make Big Data a focal point of your company

A perfect CTO is the embodiment of a good architect, Team Leader, facilitator and simply a person who is easygoing with other people and understands business. CTO is an intermediate member between business requirements and their technical implementation.

Ron Moritz, CTO of Symantec, says, “One of the key roles of the CTO is to provide the technical vision to complement the business vision, setting the tone and direction for the company’s technologies. Leadership, in this context, comes from being able to set the technical course and from being able to define what the company’s products and technologies might look like in two, three, or more years.”

Long story short, a good CTO is the one that will be able to drive your organization’s transformation into BI centered and open up new opportunities for talent development and business growth.

Have you already started building your Big Data team? What has your experience been so far?

Professional articles writer, translator and IT researcher inspired passionately through vision and cognition to change the world for the better.

Leave a comment