Band 114.
Workload Characterization in High-Performance
Computing Environments
Proceedings of a Workshop held within MASCOTS98, July 19, 1998
Montreal, Canada
Editor: Günter Haring, Gabriela Kosits, S.V. Raghavan
ca. 200 pages, ATS 210,-/160,-
Workload Characterization in High Performance Computing Environments Editors:
Gabriele Kotsis, S.V. Raghavan, G. unter Haring.Contents Gabriele Kotsis, S.V.
Raghavan, Günter Haring A critical factor in the design and development of high-performance
computing architectures and applications is to understand and to characterize
their performance and the corresponding requirements of computing and network
resources. The traditional approach to evaluate the performance of computer
systems and networks is an \off-line" performance analysis . Starting with a
characterisation of the system under study and a characterisation of the load,
a performance model is built and performance results are obtained by applying
performance evaluation techniques (including analytical, numerical, or simulaton
techniques). Alternatively, performance measurements of the real system can
replace the modelling and evaluation step. In any case, an interpretation of
results follows which can trigger either a refinement of the model (or new measurements)
if the results do not provide the required insight, or which will lead to performance
tuning activities (system side or load side) if performance deficiencies are
detected. It follows quite natural (and has been observed in the history of
performance evaluation) that changes in the computing environment (parallel
and distributed computing, network computing, mobile computing, pervasive computing,
etc.) and new system features (security and reliability mechanisms, agent-based
systems, intelligent networks, adaptive systems, etc.) pose new challenges to
performance evaluation by raising the need for new analysis methodologies. From
a load modelling point of view, the difference in using computing resource has
changed the type of model for workload characterization. While in the early
days of computing (70s) the typical systems were used in batch or interactive
mode, static workload models could adequately represent the user behaviour.
In the 80s, dynamic workload models were introduced, which were able to represent
variabilities in user behavior. Within the last years, generative workload models
have been proposed as a suitable method for bridging the gap between the user's
application oriented view of the load and the actual load (physical, resource
oriented requests) submitted to the system. To discuss those recent trends and
approaches in workload characterization for high performance computing systems,
a one-day workshop was organized in connection with the Sixth International
Symposium on Modeling, Analysis and Simulation of Computer and Telecommunication
Systems (MASCOTS '98), July 1998, Montreal, Canada. This book contains the collection
of papers presented at this workshop. The papers are organized into three sections.
In the methodology part, two papers are presented which discuss innovations
in the workload modelling methodology to be able to adequately present the workload
of high performance computing environments. The first paper, Learning Generators
for Workloads, authored by Raghavan et.al. presents a novel type of workload
models which are able to adapt to and predict changes in the resource requirements,
thus being able to represent and generate user behavior sequences. In contrast
to traditional approaches, where the user behavior is characterized by states
and state transitions with associated, fixed probabilities derived from static
knowledge of the environment under study, the approach presented in this paper
incorporates a learning mechanism which is able to dynamically generate the
sequences of user behavior. The learning capability of the model is based on
Hidden Markov Models. Lüthi introduces in his paper on Histogram-Based Characterisation
of Workload Parameters and its Consequences on Model Analysis a technique to
represent uncertainties and variabilities in the workload. Workload parameters
(device demands) are characterized by amultidimensional histogram instead by
a single mean value as in traditional approaches. He discusses how performance
analysis techniques have to be modifed in order to be able to handle histogrambased
input parameters specifying the workload. In the second part, which is dedicated
to workload mesurements, significant results and new insights into the traffic
patterns of the various types of usage of high-speed (ATM) networks are reported.
Network Traffic Measurements of Frame Relay over ATM, by Williamson, reports
on results obtained from measurements of a commercially overed \IP over Frame
Relay over ATM service. The main workload characteristics observed from coarse-grain,
long-time as well as fine-grain, short time measurements are the distinct daily
usage cycle in the aggregated trac, the strong evidence of self.similarity in
the aggregated trac, and the evidence of self-similarity in some of the individual
IP tradffic streams. In a second paper, Network Traffic Measurements of MBone
over ATM, van Melle and Williamson present workload characteristics derived
from measurements of audio/video traffic in an experimental wide area ATM network.
The objective of the measurements was also to evaluate the capacity of the experimental
network and the performance of MBONE tools in this high speed networking environment.
8.Jerkins and Wang contribute a paper on Cell-Level Measurement Analysis of
Individual ATM Connections, where an analysis of busy-hour traffic on a commercial
ATM network is reported. In particular, individual Virtual Path/Virtual Circuit
Identifier levels are analysed. The results provide evidence that traffic generated
by individual ATM connections can exhibit ON-OFF features, which is consistent
with previously published findings from lower speed packet networks. A detailed
analysis of those VPI/VCIs which dominated traffic, revealed that the service
subscriptions (CBR, VBR) have little to do with the traffic patterns generated
by the individual connections. Customers seem to subscribe to a specific service
class because of expected performance, not because of the traffic patterns they
are expected to generate. In the last part on applications, Waheed an Yan present
a case study on Workload Characterisation of CFD Applications Using Partial
Differential Equation Solvers. For a measurement based characterization of CFD
applications, a subset of the NAS parallel benchmark suite was chosen. These
benchmarks represent different types of PDE solvers that are commonly used in
real CFD applications. The benchmarks were executed on three high performance
computing platforms, an Origin 2000, an SP-2, and a cluster of PCs. Savino-V.
azquez and Puigjaner present A Workbench for Predicting the Performance of OrbixEvents
Applications on Variable Workload Environments. The proposed tool provides end
users with mechanisms to specify the essential characteristics of the application
being conceived and the ability to map software components with the execution
environment. The organizers would like to thank the referees, the authors and
the participants of this workshop for their contributions, comments, and active
discussions. We believe, that this event and the papers presented in this book
will stimulate further research in the area of workload modelling and characterization.
Gabriele Kotsis Günter Haring S.V. Raghavan