Application-Based Benchmark
Business Applications Performance Corporation
(BAPCo®)
Overview
SYSmark 2000 is the latest release in the SYSmark family of benchmarking products cooperatively designed and developed by the members of Business Applications Performance Corporation (BAPCo). In 1992, BAPCo developed the concept of application-based benchmarking using popular business software. Ever since the inception of SYSmark 92, other benchmarks have been developed using the concept pioneered by BAPCo. In 1997, BAPCo, with the help of several Fortune 500 corporations, identified new application areas and benchmarking methodologies that address new evaluation and characterization requirements of corporations. Today, BAPCo breaks new ground again by introducing SYSmark 2000, a new suite of application-based benchmarks that addresses the new computing challenges faced by corporations. Particular emphasis has been given to Internet related operations in today's applications. SYSmark 2000 is an unprecedented suite of twelve application-based benchmarks that accurately evaluate and characterize the performance of personal computers within the new computing paradigms.
Comprehensive and scientifically designed workloads in new and exciting areas such as Internet Content Creation, Speech Recognition, and Video Editing complement new workloads in the traditional Office Productivity category to provide an all-encompassing suite of benchmarks. Information technologists can now use SYSmark 2000 to accurately forecast present and future desktop computing needs as well as properly evaluate the various and sometimes dizzying solutions provided by different vendors. The advantage of using SYSmark 2000 lies in the strength of its workloads, which reflect actual usage of real applications within the new computing paradigms. Moreover BAPCo's benchmarks are cooperatively designed and developed by top performance engineers representing a wide cross-section of industry-leading publications, testing labs, PC manufacturers, software developers and semiconductor manufacturers.
SYSmark 2000 allows comparisons between Intel® Architecture systems based on the performance of real applications running on Windows 2000, Windows NT* 4.0 , Windows* 98, and Windows* 95. This document describes the structure and performance characteristics of SYSmark 2000.
Benchmark Structure
SYSmark 2000 contains twelve application workloads and a "workload manager" application responsible for setting up the workloads, timing their execution, and reporting performance results. Each SYSmark 2000 workload consists of a real application (for example, Adobe Photoshop) and a test script that sends commands to the application.
The workloads are divided into two categories. The Office Productivity category contains CorelDRAW* 9, Microsoft* Excel* 2000, Dragon Systems* NaturallySpeaking* Preferred 4.0, Netscape* Communicator* 4.61, Corel* Paradox* 9, Microsoft* PowerPoint* 2000, and Microsoft* Word* 2000. The Internet Content Creation category contains MetaCreations* Bryce* 4, Avid* Elastic Reality* 3.1, Adobe* Photoshop* 5.5, Adobe* Premiere* 5.1, and Microsoft Windows Media Encoder 4.0.
Rating Methodology
After SYSmark 2000 is run on a system to be evaluated, it assigns the system a performance rating for each application, a rating for each category, and an overall rating. The application ratings are based on a comparison of workload run times between the system being tested and a fixed calibration platform. A rating of 100 indicates the test system has performance equal to that of the calibration platform, 200 indicates twice the performance of the calibration platform, etc. Each category rating is simply a geometric mean of the workload ratings in the category. The overall rating is a weighted geometric mean of the category ratings. The SYSmark 2000 calibration platform has the following configuration:
Motherboard: Based on the Intel® 440BX motherboard
CPU: Intel® Pentium® III processor
Core Frequency: 450 MHz
Memory: 128MB DIMM
Video/Resolution: Diamond Viper V770 Ultra, 32 MB, 1024x768 16 bpp.
Disk: IBM* DJNA 371800
Operating System Windows* 98 Second Edition
*A system that scores a SYSmark 2000 rating of 200 is twice as fast as the Calibration Platform.
Office Productivity
This script first takes an abstract design and applies a Corel "Art Stroke" to it and then runs various filter effects (spheroid, charcoal, etc). It then creates and manipulates a scene composed of vector graphics. Next, it takes a raster image and applies several effects (add noise, blur, etc). It also creates several 3D objects and performs various 3D manipulations. Finally, it collates several of the images it created and publishes them as a web site.
Paradox imports a large text file and creates a database table (tens of thousands of entries). It does some SQL-style queries on this table, and also runs a find duplicates query. Next, it continues to import several other text files, formatting them and then exporting each to HTML. Then, it opens up some query forms and enters some more data and produces reports based on queries. Each of these reports is exported to HTML.
The Microsoft* Word* 2000 workload invokes a range of word processing functions including editing, spell checking, search and replace, font change, copy and paste text, print preview, merge mail fields, insert hyperlinks, background and table formatting and opening and viewing HTML pages.
Operations in the Excel workload include closing and opening spreadsheets, HTML pages and data in text files, spell checks, editing, formula calculation, plotting data in chart or histogram, formatting charts and cells, analyzing data in Pivot tables and naming a cell and inserting hyperlinks.
Operations in the PowerPoint workload include closing and opening PowerPoint slides and HTML pages, spell checking, editing, formatting and moving pictures, applying templates, formatting tables in slides, inserting hyperlinks, applying header and footer information, formatting and rotating charts, applying graphic and sound effects and adding movie files.
Dragon NaturallySpeaking Preferred 4.0
Dragon NaturallySpeaking Preferred v4.0 is a continuous speech recognition application that converts speech into text. The script plays a pre-recorded wave file (a recorded speech) using Dragon's PlayWave utility. The utility feeds this wave file into NaturallySpeaking which then converts the wave file to text.
The Netscape Communicator script simulates a user loading, viewing and navigating common web pages. First, it opens up a web site of Shakespeare plays and selects and loads the entire texts several times. Then, the script loads a page consisting of large tables, thumbnails and images, and cycles through viewing the images. Next, the script loads many pages that contain mixes of graphics, tables, and text and does a lot of text searching through these web pages.
Internet Content Creation
This script composes several pictures, video clips and audio clips into a movie. It creates an animation of about 16 BMP files and some other AVI clips, and puts various transitions between them (like scrolling text, fade outs, etc). It also superimposes two audio tracks and runs the audio through several filters, such as reverb effects, etc. The final video is a compressed AVI file.
The script uses the following operations: Load, resize, zooming out images, applying a number of filters to the images, changing the mode and color settings of images, adjusting image brightness and contrast and saving the image to a JPEG file that is optimized for web use.
Avid* Elastic Reality* 3.1 is an image processing application used to create "morphs" between images. The Elastic Reality workload sets up and renders a morph between two MPEG2-sized images (720x480 pixels, roughly 750 kB). The workload has three phases - image loading, morph definition, and morph rendering. The rendering phase takes the majority of the workload run time.
Bryce 4.0 is a ray tracing application where the user can create still and animated scenes, depicting real or fantasy objects and terrain. Those wire frame scenes are rendered, with realistic effects of light and shadow, as well as realistic interaction in animated scenes between the materials used in the objects and terrain. The Bryce 4.0 script opens an assembled wire frame scene and renders it to the final image. Once completed, the script opens a new image and saves it in Metastream video format, suitable for use as streaming video on the Internet. After saving in Metastream format, the script opens another image and saves it in HTML format, for use in web pages on the Internet.
Microsoft Windows Media Encoder 4.0
Media Encoder encodes audio and video content into an Advanced Streaming Format (ASF) stream. The output from Media Encoder is a stream of information that can be heard or viewed with Microsoft Windows Media Player, or sent to a server for multicasting, unicasting, or storage. The input file is AVI clip which is encoded using MPEG-4 Video Codec.
* Designated trademarks are the property of their respective owners.
BAPCo, and SYSmark are registered trademarks of The Business Applications Performance Corporation. All other names mentioned are trademarks, or registered trademarks of their respective companies.
Courtesy The Business Applications Performance Corporation.