Secured Data Conversion, Migration, Processing and Retrieval between SQL Database and NoSQL Big Data

Abstract

Data collection is presently managed by using classic relational database systems like SQL, MySQL and Oracle. In recent years, data collection has grown up, and it has become more complex than conventional Relational Database Management System (RDBMS); which are incompetent to deal with it. To handle this problem, organizations and large companies like Google, Facebook, Yahoo and others bring up with new data management technique called NoSQL database; which is designed for a large-scale data storage and analysis. In this paper, a new technique is presented and used to convert SQL to NoSQL database, and also it can migrate, process and retrieve data between them. Because of the NoSQL database (Big Data), it sometimes needs to store in an untrusted or semi-trusted third party, the proposed system allow users to protect their database by encrypting database sensitive attributes before performing conversion and migration processes. Furthermore, the proposed system gives users the ability for retrieving NoSQL data from Big Data storage just like retrieve SQL data; that means users can write a SQL query to retrieve NoSQL data. The proposed system used Apache HBase for NoSQL Big Data storage and Apache Sqoop and Hive for data conversion, migration, processing and retrieving processes. The implementation and results of the proposed system are showed the ability of converting, migrating, processing and retrieving data with high efficiency and good performance.