4 This directory holds performance testing scripts for git tools. The
5 first part of this document describes the various ways in which you
8 When fixing the tools or adding enhancements, you are strongly
9 encouraged to add tests in this directory to cover what you are
10 trying to fix or enhance. The later part of this short document
11 describes how your test scripts should be organized.
17 The easiest way to run tests is to say "make". This runs all
18 the tests on the current git repository.
20 === Running 2 tests in this tree ===
23 ---------------------------------------------------------
24 0001.1: rev-list --all 0.54(0.51+0.02)
25 0001.2: rev-list --all --objects 6.14(5.99+0.11)
26 7810.1: grep worktree, cheap regex 0.16(0.16+0.35)
27 7810.2: grep worktree, expensive regex 7.90(29.75+0.37)
28 7810.3: grep --cached, cheap regex 3.07(3.02+0.25)
29 7810.4: grep --cached, expensive regex 9.39(30.57+0.24)
31 You can compare multiple repositories and even git revisions with the
34 $ ./run . origin/next /path/to/git-tree p0001-rev-list.sh
36 where . stands for the current git tree. The full invocation is
38 ./run [<revision|directory>...] [--] [<test-script>...]
40 A '.' argument is implied if you do not pass any other
41 revisions/directories.
43 You can also manually test this or another git build tree, and then
44 call the aggregation script to summarize the results:
48 $ GIT_BUILD_DIR=/path/to/other/git ./p0001-rev-list.sh
50 $ ./aggregate.perl . /path/to/other/git ./p0001-rev-list.sh
52 aggregate.perl has the same invocation as 'run', it just does not run
55 You can set the following variables (also in your config.mak):
58 Number of times a test should be repeated for best-of-N
59 measurements. Defaults to 3.
62 Options to use when automatically building a git tree for
63 performance testing. E.g., -j6 would be useful.
67 Repositories to copy for the performance tests. The normal
68 repo should be at least git.git size. The large repo should
69 probably be about linux.git size for optimal results.
70 Both default to the git.git you are running from.
72 You can also pass the options taken by ordinary git tests; the most
76 Create "trash" directories used to store all temporary data during
77 testing under <directory>, instead of the t/ directory.
78 Using this option with a RAM-based filesystem (such as tmpfs)
79 can massively speed up the test suite.
85 The performance test files are named as:
87 pNNNN-commandname-details.sh
89 where N is a decimal digit. The same conventions for choosing NNNN as
90 for normal tests apply.
96 The perf script starts much like a normal test script, except it
101 # Copyright (c) 2005 Junio C Hamano
104 test_description='xxx performance test'
107 After that you will want to use some of the following:
109 test_perf_fresh_repo # sets up an empty repository
110 test_perf_default_repo # sets up a "normal" repository
111 test_perf_large_repo # sets up a "large" repository
113 test_perf_default_repo sub # ditto, in a subdir "sub"
115 test_checkout_worktree # if you need the worktree too
117 At least one of the first two is required!
119 You can use test_expect_success as usual. In both test_expect_success
120 and in test_perf, running "git" points to the version that is being
121 perf-tested. The $MODERN_GIT variable points to the git wrapper for the
122 currently checked-out version (i.e., the one that matches the t/perf
123 scripts you are running). This is useful if your setup uses commands
124 that only work with newer versions of git than what you might want to
125 test (but obviously your new commands must still create a state that can
126 be used by the older version of git you are testing).
128 For actual performance tests, use
130 test_perf 'descriptive string' '
135 test_perf spawns a subshell, for lack of better options. This means
138 * you _must_ export all variables that you need in the subshell
140 * you _must_ flag all variables that you want to persist from the
141 subshell with 'test_export':
143 test_perf 'descriptive string' '
144 foo=$(git rev-parse HEAD) &&
148 The so-exported variables are automatically marked for export in the
149 shell executing the perf test. For your convenience, test_export is
150 the same as export in the main shell.
152 This feature relies on a bit of magic using 'set' and 'source'.
153 While we have tried to make sure that it can cope with embedded
154 whitespace and other special characters, it will not work with