What is Datawarehouse and How to Build One
Simply put, it is a system that organizes and stores data in a way that can be efficiently retrieved in a format that helps make business decisions.
I have heard one of the managing director from my client, say “If we use ReportNet, we cannot use Datawarehouse!”. He misunderstood what Datawarehouse really is. At that moment I understood something I took for granted may not be so obvious for other people who are not experts in the field. I don’t blame the managers for confusing reporting tools with data warehousing, since this is how commercial data warehousing tools often promote themselves.
So if you are in management, and you are in the hot spot to put your business intelligence project together, read up!
The basic tasks of building datawarehouse:
- First and foremost, what is your objective? What kind of reports do you need to help your management team to make better business decisions (project term: business requirements)
- Identify where these data come from (project term: data sources, data requirements)
- Identify target database structure for reporting (project term: data modeling)
- Identify how the reports should be run, what kind of software or how business users can access and generate reports (project term: functional requirements)
- Depends on the complexity of transforming the source data to the target data, interim database maybe required (project term: technical requirements, ETL, data modeling, data cleansing)
The project terms given above gives you an idea on the keywords you usually find in a datawarehouse project job description.
Skills and people required to do the above tasks are:
Business analysts – they work with management and business users to identify what reports will be the best fit to the business objective
Data modelers – they are skilled to take the business requirements and construct the best database structure to retrieve data in an efficient manner. This person usually is knowledgeable in both technical and business areas.
ETL developers – also called database developer, sql developer. They are skilled in writing programs and scripts to bring the data from one place to another. ETL means Extract, Transform, and Load.
Reporting developers – usually called “Cognos developer”, “MicroStrategy developer”, “Informatica developer”, etc. Notice they are called according to the business intelligence software name. Most reporting tools can be bought as software, with user friendly interfaces ready for users to generate reports with ease. In most cases in-house reporting tools cost too much to build compared to the cost of integrating third vendor software. They are skilled in integrating the particular software with existing systems and design meaningful reports.
Data architect – like application architects, data architects, they analyze the business objective of the project, analyze the complexity of the project and identify what is needed and how (resources and processes) to build the datawarehouse. They are often experienced data modelers, who have extensive technical and business knowledge.
DBA – database administrators. They are skilled in managing servers that are dedicated to host databases, setting up databases according to data usage and control user access, and on-going backup and disaster recovery planning.
Infrastructure architect / network administrator – They manage all the hardware, server connections and access, server load and maintain all servers across the company. They determine what and how to setup the development, testing and production environment. Most large companies require additional servers and new hardware when building a datawarehouse.
Technical Lead – data architects often take on this role in datawarehouse projects. Technical lead works closer to project managers to ensure technical issues have been properly dealt with in a timely manner.
Project Manager – ensure delivery of the project, on time on budget… if possible.
Of course, I only touched the surface of datawarehouse. The devil is in the details as always.
Need Business Intelligence Solution? We can help. Request a proposal or just general consultation, contact us today at info@wizebiz.ca or call us 416-682-2036.
