One of the common problems that System Managers face is system performance
degradation over time, and eventually the end-users complain that the system is
slow. Frustratingly enough, the System Managers currently have no way of
anticipating this. In most cases, the direct cause is the state of fragmentation
on the disk. The ad hoc solution will always be either an overall resource cleanup
and reorganization of the data, replacement of the entire disk, or even a full
server upgrade.
Fragmentation of data is inherent to the manner of its storage on a disk.
When data is deleted, the space previously occupied by it returns to the general
pool of free space on the disk. When new data is written to the disk, it fills
up the space first available to it, regardless of the ultimate size of the data
file. Data in excess (in size) of the first slot of available space
automatically flows over to the next available slot and so on until the data is
fully recorded. Thus the data is fragmented upon the face of the disk, in a
never ending process of deletion and writing of data.
This phenomenon occurs with files written directly to the disk, referred to
as External Fragmentation. as well as with data handled internally by a
data base system that externally may appear as one single data file, referred
to as Internal Fragmentation. Among the latter one may refer to systems
such as SQL, mail applications etc.
The problem of External Fragmentation is very familiar to members of
the IT community. The never ending process of delete-write of information on
the disk eventually results in an extremely high level of fragmentation. This
is the main reason for performance degradation in computers: instead of just
using the data, the computer spends most of its resources looking for it.
To rearrange the data on the disk so as to correct the state of
fragmentation, there are only a few options, the most popular of which is the defragmentation
of the disk. Defragmentation is an extremely long process, and therefore it is
postponed and performed only when the situation is clearly acute. If it is
clear that it is not possible to rearrange the data within a given time limit
(i.e. the weekend...), System Managers prefer to endure heavy performance
degradation rather than decrease the availability or performance of the system
to the users.
Similarly in the case of Internal Fragmentation, the process of defragmentation
is labeled reorganization, but the consequence as described above is
identical.
An old proverb goes “If you can’t measure it – You can’t Manage it”. This is
true also for Information Technology.
According to a research conducted by IDC, a leading
consulting firm in the computers industry, the performance degradation caused
by disk fragmentation alone, shortens the life of the computer/server by 30% on
average. This translates into a direct increase of 30% in the yearly budget for
hardware upgrades. It is possible to minimize this significant sum utilizing
traditional Defrag and Reorganization utilities. The major setback remains the
timing and scheduling of this extremely long and cumbersome process. We propose
that this is a management task that can, and should, be undertaken.
Suppose that the level of chaos (i.e. fragmentation) of the data in a given
Data Base can be calculated and quantified. Such a factor will alert the System
Manager way before the end-users start experiencing any malfunction. One will
be able to manage IT resources and schedule the reorganization process ahead of
time, before it becomes impractical, and before the complaints start arriving.
Such a factor now exists in the world of Disk Management. It is called “Lacelevel”
And it can be downloaded at http://www.disklace.com/.
This novel tool measures the physical disorder and bulk of the data, and
calculates how far it is from the ideal data structure in which there would be
no fragmentation at all. It
calculates a numeric factor that is based on the specific distribution of free
space and data fragments on the disk. The factor reflects the delay caused by
the data disorder and thus to the computer in whole, and indirectly indicates
the users’ response time and general satisfaction.
This tool can be adjusted for any environment. It is based on a patent
pending algorithm, and it could be used to measure External Fragmentation as
well as Internal Fragmentation.
In Hard Disk environment, there are files, and the location and length of
each file and its extents are physical determined. This information is
sufficient to calculate the Fragmentation Factor of the disk. In Database
environment there is one single file that is internally fragmented. Therefore in
this environment we refer to each DATA and INDEX portion of the table as a separate file for
measurement of the Internal Fragmentation level of the data base. This figure
correlates to the level of satisfaction of your users, because it reflects the
level of data disorder on your disk, and consequently the level of performance
of your system.
We have started adapting
this module to SQL, and we hope to present a demo of LaceLevel adapted
to the database environment in the near future to the SQLServerCentral.com
community.