Security Architecture Consideration for Hadoop Implementation.

One of the biggest concerns in our present age revolves around the security and protection of sensitive information. In our current era of Big Data, our organizations are collecting, analyzing, and making decisions based on analysis of massive amounts of data sets from various sources, and security in this process is becoming increasingly more important. The more data you have, the more important it is that you protect it. It means that not only must we provide effective security controls on data leaving our networks, but we also must control access to data within our networks

Nowadays every organization is facing big data challenges and most organizations turn to Hadoop for the big data solution, so is our organization. Recently we are developing a Hadoop architecture strategy and roadmap, one of the architectures we need to develop is the security architecture for Hadoop implementation. Based on different business requirements and organization’s enterprise architecture principles Hadoop implementation can be on-premises or in the cloud. There will be different security concerns for different implementation.

Within our organization enterprise architecture group works closely with security architecture to first identify and understand business use cases and based on each use cases requirement to create a security requirement catalog. We will categorize the requirements into different categories and identify the existing security architecture to analysis the gaps. Considering the massive amount of data that nodes hold, there is an increasing need to focus on security architecture for the Hadoop cluster. We realize that if we are going to implement Hadoop cloud solution, business critical and sensitive data will leave the premises so adequate security controls is necessary. We prefer to adopt Security as a Service provider and the architecture should consider to integrate the Security as a Service into our organization security ecosystem for consistent operations and auditing. Some of the security consideration will be

1. How to enforce authentication for users and applications?
2. How to integrate internal data sources to the Hadoop cloud?
3. How to enforce data access control based on existing access control policies?
4. How can Hadoop integrate with existing enterprise security services?  

In our fast-paced and connected world it is critical to understand the importance of security as we process and analyze massive amounts of data. This starts with understanding our data and associated security policies, and it also revolves around understanding the security policies in our organizations and how they need to be enforced.