5.1 Review & Conclusions
__5.2.1 Complete the Pilot Project
__5.2.2 Extend to Other Subject Areas
5.3 The Future of Data Warehousing
__5.3.1 Integrated Tools
__5.3.2 The World Wide Web
__5.3.3 OLAP constructs in RDBMS
__5.3.4 No Future Without Data Warehousing
I should point out again that at the time of writing this report, the project is languishing somewhat, due in part to the Project Leader (the author) being transferred overseas. Although I have moved to a new country and have had to learn a new job, my involvement in the Avondale Data Warehouse Project has continued from a distance. The other member of the technical team was appointed as my successor at Avondale and has also had to learn a new job, so the project understandably has lost a bit of momentum.
In writing this report, we had hoped to be able to make some observations on the impact of the project on decision making processes at Avondale, but since it has not yet been implemented, that is not possible.
Nevertheless, we believe we can claim some credit for helping to focus attention on planning and decision making at Avondale. It is clear that the models chosen for the pilot project will provide vital strategic information, and other areas are being considered for extending the project. In addition, knowledge workers will begin to get answers to questions they had given up asking because of the IT backlog.
In Chapter 10. (A Data Warehouse Design Review Checklist) of Bill Inmon's book Building the Data Warehouse, the author states that "the attendees at the design review include anyone who has a stake in the development, operation, or usage of the DSS subject area being reviewed."(38) This normally includes the following, and of this group, the most important attendees are the end users and the DSS analysts.
A review questionnaire was distributed to members of the pilot project group, and is reported in full along with responses (in quoted italics) in Appendix H: Review Questionnaire. The quoted responses are from a senior administrator. Parts of the survey and responses are included here for comment.
1. The EIS/Data Warehousing concept
Part 2 in the above response suggests that the role of a data warehouse in supporting management decision making was not fully appreciated by Senior Management. The potential benefit of PowerPlay in data analysis and decision support was overshadowed by the very attractive gains offered by Impromptu as an enquiry and reporting tool for middle management. The shortage of technical resources and the lack of adequate management commitment made it difficult to keep the project focused on "management improvement", which was an integral part of its initial justification.
The training that was delivered by the consulting firm was indeed irrelevant, although if the implementation had proceeded according to schedule, the timing would have been appropriate. The trainer who was sent to provide the PowerPlay training was very new in the job and had a superb technical understanding of the Cognos programming languages, but lacked an understanding of what academic administrators might need to support their decision making. He was able to describe what each menu and command button did, but did not show the users how the product could be used to solve their business problems.
Late in 1996, Avondale College went through a Strategic Planning exercise with a consultant. Lack of a clear plan may have slowed the initial progress of the project, but the subject areas that were chosen were regarded as valuable and achievable, and quite appropriate for a pilot project. The Strategic Planning that has taking place since the project started should prove invaluable for extending the project, once the pilot is implemented.
No comments could be made on many parts of the questionnaire since the project did not proceed far enough. When time and resources are available and the project can be restarted, it is clear that the training will need to be repeated.
The immediate tasks which I have been able to identify for finishing the pilot project are as follows:
A number of other possible subject areas were identified earlier in the project and are reported in Section 4.3 of this report. In addition to retention rates, cohort analysis and student performance indicators, there were:
After the recent Strategic Planning exercise conducted at Avondale College in late 1996, it would be useful to repeat the work reported in Section 4.2 on Establishing Needs. This could be expected to reveal new information requirements and priorities.
Currently, the tools for data warehousing as categorized on The Data Warehousing Institutes "New Roadmap To Data Warehousing" poster are only loosely connected or not connected at all. Single vendor solutions are limited in one way or another in terms of a total data warehousing solution. This almost always means tools from a variety of vendors must be used. Data warehousing standards are needed for things such as:
Application development for the web is seen as having great potential as a truly platform-independent environment. Web browsers are becoming the environments in which people work, and is an avenue for extending the reach of decision support front-ends to existing client software. The data warehousing environment of a read-only database on a separate hardware platform lends itself well to Internet access. New products are appearing at a rapid rate that simplify linking data to web sites. The advent of Java offers an alternative to, or at least variation on, client-server processing for the masses.
For a review of the current state of data warehousing on the web, see the Technology Viewpoint article published by Aberdeen Group. It is available on the web at http://www.aberdeen.com/secure/viewpnts/v9n6/v9n6.htm.
A relational database designed for OLTP will not serve well as a database for data
analysis. Optimization techniques such as aggregating fact tables, partitioning fact
tables, and denormalizing relation tables all provide significant improvements in
performance. However, the RDBMS itself and especially its optimizing algorithm need to do
things differently for OLAP.
Some of the deficiencies in existing OLTP are described in an article by Neil Raden that was published in Information Week (March 18, 1996, Issue: 571, Section: OpenLabs).
"Systems designed for transaction processing are overwhelmed by the volume of data [involved in data warehousing]. A data warehouse also needs special tools that eliminate the extraneous overhead of transaction logging, rollback/commit, and incremental referential integrity checking. In addition, a common approach in data warehousing is to store aggregated information in the database, avoiding slow and costly 'group by' operations by individual queries. Some critical components to look for in a DBMS are auto-aggregation at load time, aggregation scheme advisers, and aggregate navigators to direct the query optimizer to always use the highest level of aggregation available.
"To facilitate the types of queries common in decision support, the indexes in data warehousing are larger and more complex. Rebuilding these indexes can be time-consuming, resulting in update cycles that are too long. To get the best performance from a data warehouse, look for DBMSs that have new index types (bitmaps, join indexes) and index building processes, including incremental indexing-using the existing index rather than the base tables as a starting point-and parallelizing the index processing."(39)
The world of higher education as well as business in general is becoming increasingly competitive. Those institutions and businesses that realize the potential benefit of the information resource first will gain a competitive advantage. As stated in the closing statement of the White Paper by E. F. Codd & Associates entitled Providing OLAP (On-line Analytical Processing) to User-Analysts: An IT Mandate, "The quality of strategic business decisions made as a result of OLAP is significantly higher and more timely than those made traditionally. Ultimately, an enterprise's ability to compete successfully and to grow and prosper will be in direct correlation to the quality, efficiency, effectiveness and pervasiveness of its OLAP capability. It is, therefore, incumbent upon IT organizations within enterprises of all sizes, to prepare for and to provide rigorous OLAP support for their organizations".(40)
(38) Inmon, W.H. Building the Data Warehouse. p295.
(39) Raden, N. Technology Tutorial, Part 1 - Maximizing Your Warehouse. Information Week. March 18, 1996.
(40) Codd E.F., Codd S.B. and Salley C.T.: E. F. Codd & Associates. Providing OLAP (On-line Analytical Processing) to User-Analysts: An IT Mandate. p31.
Next section, Previous section or return to table of contents