Design and implementation of the web (extract, transform, load) process in data warehouse application

Abstract

Recently, due to an increase size and complexity of data in addition to the management issues, data storage is required a great deal of attention to employ it in realistic applications, so the requirement for designing and implementing a data warehouse system has become an urgent necessity, ETL represents a vital side of data warehouse architecture. This paper, designed data warehouse application contains a web ETL process that divides its work between the input device and server. This system proposed to solve the problem lack of partition of the work between the input device and server. The proposed and designed system can be used and applied in many branches and disciplines for its high performance in adding data, analyzing data using a web server and building queries. The analysis of the results has been proven that a faster time is achieved in cleaning data and transferring it between the remote parts of the system connected to the Internet. Where ETL without Missing for any data took 0.00582 seconds.