Security Architecture Consideration for Hadoop Implementation.

One of the biggest concerns in our present age revolves around the security and protection of sensitive information. In our current era of Big Data, our organizations are collecting, analyzing, and making decisions based on analysis of massive amounts of data sets from various sources, and security in this process is becoming increasingly more important. The more data you have, the more important it is that you protect it. It means that not only must we provide effective security controls on data leaving our networks, but we also must control access to data within our networks

Nowadays every organization is facing big data challenges and most organizations turn to Hadoop for the big data solution, so is our organization. Recently we are developing a Hadoop architecture strategy and roadmap, one of the architectures we need to develop is the security architecture for Hadoop implementation. Based on different business requirements and organization’s enterprise architecture principles Hadoop implementation can be on-premises or in the cloud. There will be different security concerns for different implementation.

Within our organization enterprise architecture group works closely with security architecture to first identify and understand business use cases and based on each use cases requirement to create a security requirement catalog. We will categorize the requirements into different categories and identify the existing security architecture to analysis the gaps. Considering the massive amount of data that nodes hold, there is an increasing need to focus on security architecture for the Hadoop cluster. We realize that if we are going to implement Hadoop cloud solution, business critical and sensitive data will leave the premises so adequate security controls is necessary. We prefer to adopt Security as a Service provider and the architecture should consider to integrate the Security as a Service into our organization security ecosystem for consistent operations and auditing. Some of the security consideration will be

1. How to enforce authentication for users and applications?
2. How to integrate internal data sources to the Hadoop cloud?
3. How to enforce data access control based on existing access control policies?
4. How can Hadoop integrate with existing enterprise security services?  

In our fast-paced and connected world it is critical to understand the importance of security as we process and analyze massive amounts of data. This starts with understanding our data and associated security policies, and it also revolves around understanding the security policies in our organizations and how they need to be enforced. 

Security architecture development approach

Based on the TOGAF, security concerns are pervasive throughout the architecture domains and in all phases of the architecture development. Security is called out separately because it is infrastructure that is rarely visible to the business function. Its fundamental purpose is to protect the value of the systems and information assets of the enterprise. Often the nature of security in the enterprise is that it is deemed successful if either nothing happens that is visible to the user or other observer, and/or no damage or losses occur to the enterprise.

The generally accepted areas of concern for the security architect are:

• Authentication
• Authorization
• Audit
• Assurance
• Availability
• Asset Protection
• Administration
• Risk Management

When we develop enterprise architecture, security architecture will be all around each phase of the development, security requirements need to be taken into the consideration during the each phase of the development. Here we are talking about creating security architecture not security policies for a special projects. I have experienced a situation during the development of architecture of cloud solution for the organization. When I was working with security specialists on the subject usually I will get a set of policies or even a specific product to use. To me it is different. Some of the general security policies developed based on the past and existing information and technology system may not fit for this architecture. The right approach should be as described by TOGAF, gathering current and emerging security requirements from business for each phase of the architecture development, create a security requirement catalog, perform a baseline analysis to determine the “current state” of  the security effort, identify gaps in the current state, articulate an architecture in functional terms to address the gaps and incorporate emerging business requirements, identify and communicate s “desire state” environment and develop a blueprint for the future architecture. It is important to develop and select standards and policies to implement the security program within the context of the chose architecture. Business requirements should be updated and reassessed during the iteration of the architecture development process.

Reference

TOGAF

https://www.giac.org/paper/gsec/610/building-enterprise-security-architecture/101447

Integrating Security Architecture into Enterprise Architecture

One of the key findings of Gartner research paper “Aligning Security Architecture and Enterprise Architecture: Beast practice” is “The more-closely aligned the security architecture function is to the enterprise architecture (EA), the more effective it is. Complete integration of security into the EA must be the goal.”

In this research paper, Gartner indicates that “The goal is for the security architecture to be completely integrated into an organization's EA.” In reality, as stated in this paper, “many realities mitigate against achieving this in most organizations, including historical organizational and internal political realities. Security architecture has traditionally been practiced separately from the EA. Thus, security architects are often not conversant with EA terminology, principles and practices. Furthermore, the EA tools used by most organizations do not allow for security artifacts to be fully integrated with the EA, simultaneously being able to provide a separate, security perspective where security-only artifacts can be modeled.”

This is what I see in our organization today, in certain degree we don’t even have security architecture in place in general.

EA is difficult to interact with security architecture group, there is no formal communication mechanism in place. While EA creates solution space for the business, security architecture was not in the consideration of the design. Most of the security specialists (I will not call them security architects as most of the them are only concentrating on designing a detailed security solution for the projects rather than creating an architecture) don’t quite understand what enterprise architecture is. Often time this creates conflicts and misunderstanding between EA and security. The security solution, policies for a specific project is isolated from other security policies and solutions even for a similar project.

Gartner provides strategies to improve the level of alignment which I fully agree and I think our organization should adopt.

·        Sending security architects to attend a training course on the EA methodology used in the organization

·        Aligning the structure and methodology of the enterprise information security architecture (EISA) framework with the structure and methodology of the organization's EA approach

·        Adopting EA terminology in the EISA practice

·        Leveraging any focus on IT governance in the organization to support the effective integration of security into the IT services and application life cycles, and thus into the EA process

·        Conducting joint workshops between the EISA and EA teams to develop common processes, process interfaces and terminology

·        Combining EISA and EA in major new projects

·        Placing security architects in the EA team — that is, starting to work toward integrating the EISA team into the EA team

Reference

https://www.gartner.com/doc/790521/aligning-security-architecture-enterprise-architecture