In an effort to create a set of benchmarks for gauging desktop performance I have written up a procedure for gnome-terminal to test the speed that text is sent to the terminal window and written a script to perform the test. I have attached the writeup for the procedure (bench_terminal.txt) and the script to run the test (cattest) to this email. I am interested in hearing people's comments on this test. I know this is only one test and doesn't address desktop issues like program startup time, but we have to start somewhere. We can start to assemble the benchmarks and put them in http://fedora.redhat.com/projects/additional-projects/benchmarks/ -Will ------------------------------------------------------------------------ Benchmark text output performance of gnome-terminal Frequently output is sent to a terminal window. In Gnome the program gnome-terminal handles the display of text on a terminal window. The, one simple benchmark is to determine the amount of time required to output a large text file to a gnome-terminal window. gnome-terminal is a little tricky to benchmark because has a server. When you start gnome-terminal it connects to the server rather than starting a new child process. Thus, timing the gnome-terminal command will count the time required to communicate information to the gnome-terminal server rather than the time required to actually perform the task. To work around this problem a script is executed within the new gnome-terminal window and the information is saved to a file. Procedure: 0) Get system configuration information hardware and software: CPU: cat /proc/cpuinfo Memory: cat /proc/meminfo Kernel: uname -a gnome-terminal: rpm -qa gnome-terminal xserver: rpm -qa xorg-x11 1) Get the test file, the jargon file from Project Gutenberg: http://www.gutenberg.net/etext02/jarg422.txt. 2) Verify that the file is the same with md5sum $ md5sum jarg422.txt ef9b53f52312ee266c98c8e206d9e823 jarg422.txt 3) Place the cattest script in the same directory as jarg422.txt and make it executable. 4) Run the test on the console of the machine with the command below. The script will generate a file "cattime in the directory with the amount of time required to cat the file to the terminal window. gnome-terminal -e "./cattest" 4a) If the system is set up to run oprofile and you have root access, you can run the same script as root with command below to get some additional profiling information in the cattime file: gnome-terminal -e "./cattest --profile" Additional analysis on the oprofile data can be performed after the benchmark completes. ------------------------------------------------------------------------ #! /bin/bash # # Simple test to gather data on where gnome-terminal spends # time This is compilicated by the terminal server model of # gnome-terminal. Time taken for benchmark written to cattime. # # When optional --profile on commandline, oprofile used to get an # overall view of what is happening on the system. PROFILING is only # going to work with kernel that have oprofile support (Red Hat SMP # kernels). # # Will Cohen # 5/27/2004 # BENCHMARK="cattest" VERSION=0.0 OPCONTROL=/usr/bin/opcontrol OPREPORT=/usr/bin/opreport RM=/bin/rm RESULTS_FILE=cattime if test "$1" = "--profile"; then PROFILING=yes else PROFILING=no fi # Setup default oprofile. if test "$PROFILING" = "yes"; then $OPCONTROL --deinit $OPCONTROL --reset # FIXME Command below may use previous event settings for oprofile. $OPCONTROL --setup --no-vmlinux --separate=library $OPCONTROL --start fi # Run the actual experiment $RM -rf $RESULTS_FILE echo "Benchmark: " $BENCHMARK " Version: " $VERSION >> $RESULTS_FILE date >> $RESULTS_FILE # The actual benchmark being timed is below. /usr/bin/time /bin/cat `pwd`/jarg422.txt 2>> $RESULTS_FILE # Shutdown oprofile. if test "$PROFILING" = "yes"; then $OPCONTROL --dump $OPCONTROL --shutdown # If PROFILING, need to do analysis with oprerport after running the test. # May need more details than what is provided by command below. $OPREPORT --threshold 2 --long-filenames >> $RESULTS_FILE fi

On Thu, May 27, 2004 at 04:21:38PM -0400, Will Cohen wrote: ># The actual benchmark being timed is below. >/usr/bin/time /bin/cat `pwd`/jarg422.txt 2>> $RESULTS_FILE You're profiling cat under gnome-terminal, not gnome-terminal + cat. Wouldn't be better to run instead: /usr/bin/time gnome-terminal -x cat $PWD/jarg422.txt

> The net result > with the suggested change on gnome-terminal is that you get the amount > of time it takes to fire off the command to the gnome-terminal server, > not the amount of time to complete the task. The /usr/bin/time will > finish long before the cat is actually done. Oh, yes, I forgot. I usually run gnome-terminal with the options --disable-factory --sm-disable. They should prevent the use of a gnome-terminal server, giving more meaningful results.

On Fri, May 28, 2004 at 02:19:42PM -0400, Owen Taylor wrote: >But is it interesting to time the startup time of gnome-terminal? While >that's perhaps an interesting benchmark as well, it has little >correlation with questions like "how fast will my kernel compile >go when run in gnome-terminal"? But will cat end after all output has been displayed or before? (Due to pipe buffers.)

>If we are measuring wall clock time, timing the cat command seems >to be pretty appropriate. > >If we are measuring CPU usage, then we need to find CPU usage from >the terminal *and* from the X server, so we already need to use a >tool like oprofile that can get a global view. I agree. Regards, Luciano Rocha