git.oblomov.eu Git

Merge branch 'jk/make-tags-find-sources-tweak' into maint

Update the procedure to generate "tags" for developer support.

* jk/make-tags-find-sources-tweak:
  Makefile: exclude contrib from FIND_SOURCE_FILES
  Makefile: match shell scripts in FIND_SOURCE_FILES
  Makefile: exclude test cruft from FIND_SOURCE_FILES
  Makefile: reformat FIND_SOURCE_FILES

Merge branch 'jc/latin-1' into maint

Some platforms no longer understand "latin-1" that is still seen in
the wild in e-mail headers; replace them with "iso-8859-1" that is
more widely known when conversion fails from/to it.

* jc/latin-1:
utf8: accept "latin-1" as ISO-8859-1
utf8: refactor code to decide fallback encoding

Eighth batch for 2.12

Signed-off-by: Junio C Hamano <gitster@pobox.com>

Merge branch 'jk/fsck-connectivity-check-fix'

"git fsck --connectivity-check" was not working at all.

* jk/fsck-connectivity-check-fix:
  fsck: lazily load types under --connectivity-only
  fsck: move typename() printing to its own function
  t1450: use "mv -f" within loose object directory
  fsck: check HAS_OBJ more consistently
  fsck: do not fallback "git fsck <bogus>" to "git fsck"
  fsck: tighten error-checks of "git fsck <head>"
  fsck: prepare dummy objects for --connectivity-check
  fsck: report trees as dangling
  t1450: clean up sub-objects in duplicate-entry test

Merge branch 'js/difftool-builtin'

Rewrite a scripted porcelain "git difftool" in C.

* js/difftool-builtin:
  difftool: hack around -Wzero-length-format warning
  difftool: retire the scripted version
  difftool: implement the functionality in the builtin
  difftool: add a skeleton for the upcoming builtin

Merge branch 'rs/qsort-s'

A few codepaths had to rely on a global variable when sorting
elements of an array because sort(3) API does not allow extra data
to be passed to the comparison function.  Use qsort_s() when
natively available, and a fallback implementation of it when not,
to eliminate the need, which is a prerequisite for making the
codepath reentrant.

* rs/qsort-s:
  ref-filter: use QSORT_S in ref_array_sort()
  string-list: use QSORT_S in string_list_sort()
  perf: add basic sort performance test
  add QSORT_S
  compat: add qsort_s()

Merge branch 'ls/travis-p4-on-macos'

Update the definition of the MacOSX test environment used by
TravisCI.

* ls/travis-p4-on-macos:
travis-ci: fix Perforce install on macOS

Merge branch 'vp/show-ref-verify-head'

"git show-ref HEAD" used with "--verify" because the user is not
interested in seeing refs/remotes/origin/HEAD, and used with
"--head" because the user does not want HEAD to be filtered out,
i.e. "git show-ref --head --verify HEAD", did not work as expected.

* vp/show-ref-verify-head:
  show-ref: remove a stale comment
  show-ref: remove dead `if (verify)' check
  show-ref: detect dangling refs under --verify as well
  show-ref: move --quiet handling into show_one()
  show-ref: allow -d to work with --verify
  show-ref: accept HEAD with --verify

Merge branch 'sb/retire-convert-objects-from-contrib'

Remove an ancient tool left in contrib/.

* sb/retire-convert-objects-from-contrib:
contrib: remove git-convert-objects

Merge branch 'sb/in-core-index-doc'

Documentation and in-code comments updates.

* sb/in-core-index-doc:
  documentation: retire unfinished documentation
  cache.h: document add_[file_]to_index
  cache.h: document remove_index_entry_at
  cache.h: document index_name_pos

Merge branch 'js/remote-rename-with-half-configured-remote'

With anticipatory tweaking for remotes defined in ~/.gitconfig
(e.g. "remote.origin.prune" set to true, even though there may or
may not actually be "origin" remote defined in a particular Git
repository), "git remote rename" and other commands misinterpreted
and behaved as if such a non-existing remote actually existed.

* js/remote-rename-with-half-configured-remote:
remote rename: more carefully determine whether a remote is configured
remote rename: demonstrate a bogus "remote exists" bug

Merge branch 'jk/clear-delta-base-cache-fix'

A crashing bug introduced in v2.11 timeframe has been found (it is
triggerable only in fast-import) and fixed.

* jk/clear-delta-base-cache-fix:
clear_delta_base_cache(): don't modify hashmap while iterating

Merge branch 'st/verify-tag'

"git tag" and "git verify-tag" learned to put GPG verification
status in their "--format=<placeholders>" output format.

* st/verify-tag:
  t/t7004-tag: Add --format specifier tests
  t/t7030-verify-tag: Add --format specifier tests
  builtin/tag: add --format argument for tag -v
  builtin/verify-tag: add --format to verify-tag
  ref-filter: add function to print single ref_array_item
  gpg-interface, tag: add GPG_VERIFY_OMIT_STATUS flag

Merge branch 'js/mingw-isatty'

An update to a topic that is already in 'master'.

* js/mingw-isatty:
mingw: follow-up to "replace isatty() hack"

Merge branch 'js/sequencer-i-countdown-3'

The sequencer machinery has been further enhanced so that a later
set of patches can start using it to reimplement "rebase -i".

* js/sequencer-i-countdown-3: (38 commits)
  sequencer (rebase -i): write out the final message
  sequencer (rebase -i): write the progress into files
  sequencer (rebase -i): show the progress
  sequencer (rebase -i): suggest --edit-todo upon unknown command
  sequencer (rebase -i): show only failed cherry-picks' output
  sequencer (rebase -i): show only failed `git commit`'s output
  sequencer: use run_command() directly
  sequencer: update reading author-script
  sequencer (rebase -i): differentiate between comments and 'noop'
  sequencer (rebase -i): implement the 'drop' command
  sequencer (rebase -i): allow rescheduling commands
  sequencer (rebase -i): respect strategy/strategy_opts settings
  sequencer (rebase -i): respect the rebase.autostash setting
  sequencer (rebase -i): run the post-rewrite hook, if needed
  sequencer (rebase -i): record interrupted commits in rewritten, too
  sequencer (rebase -i): copy commit notes at end
  sequencer (rebase -i): set the reflog message consistently
  sequencer (rebase -i): refactor setting the reflog message
  sequencer (rebase -i): allow fast-forwarding for edit/reword
  sequencer (rebase -i): implement the 'reword' command
  ...

Merge branch 'jk/coding-guidelines-update'

Developer doc update.

* jk/coding-guidelines-update:
CodingGuidelines: clarify multi-line brace style

Merge branch 'jk/loose-object-fsck'

"git fsck" inspects loose objects more carefully now.

* jk/loose-object-fsck:
  fsck: detect trailing garbage in all object types
  fsck: parse loose object paths directly
  sha1_file: add read_loose_object() function
  t1450: test fsck of packed objects
  sha1_file: fix error message for alternate objects
  t1450: refactor loose-object removal

Merge branch 'js/exec-path-coverity-workaround'

Code cleanup.

* js/exec-path-coverity-workaround:
git_exec_path: do not return the result of getenv()
git_exec_path: avoid Coverity warning about unfree()d result

Merge branch 'bw/push-submodule-only'

"git submodule push" learned "--recurse-submodules=only option to
push submodules out without pushing the top-level superproject.

* bw/push-submodule-only:
  push: add option to push only submodules
  submodules: add RECURSE_SUBMODULES_ONLY value
  transport: reformat flag #defines to be more readable

Merge branch 'jk/vreport-sanitize'

An error message with an ASCII control character like '\r' in it
can alter the message to hide its early part, which is problematic
when a remote side gives such an error message that the local side
will relay with a "remote: " prefix.

* jk/vreport-sanitize:
vreport: sanitize ASCII control chars
Revert "vreportf: avoid intermediate buffer"

Documentation: implement linkgit macro for Asciidoctor

AsciiDoc uses a configuration file to implement macros like linkgit,
while Asciidoctor uses Ruby extensions. Implement a Ruby extension that
implements the linkgit macro for Asciidoctor in the same way that
asciidoc.conf does for AsciiDoc. Adjust the Makefile to use it by
default.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

builtin/commit.c: switch to strbuf, instead of snprintf()

Switch to dynamic allocation with strbuf, so we can avoid dealing
with magic numbers in the code and reduce the cognitive burden from
the programmers. The original code is correct, but programmers no
longer have to count bytes needed for static allocation to know that.

As a side effect of this change, we also reduce the snprintf()
calls, that may silently truncate results if the programmer is not
careful.

Helped-by: René Scharfe <l.s.r@web.de>
Helped-by: Junio C Hamano <gitster@pobox.com>
Helped-by: Jeff King <peff@peff.net>
Signed-off-by: Elia Pinto <gitter.spiros@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

update-ref: add test cases for bare repository

The default behavior of update-ref to create reflogs differs in
repositories with worktree and bare ones. The existing tests cover only
the behavior of repositories with worktree.

This commit adds tests that assert the correct behavior in bare
repositories for update-ref. Two cases are covered:

- If core.logAllRefUpdates is not set, no reflogs should be created
- If core.logAllRefUpdates is true, reflogs should be created

Signed-off-by: Cornelius Weig <cornelius.weig@tngtech.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

refs: add option core.logAllRefUpdates = always

When core.logallrefupdates is true, we only create a new reflog for refs
that are under certain well-known hierarchies. The reason is that we
know that some hierarchies (like refs/tags) are not meant to change, and
that unknown hierarchies might not want reflogs at all (e.g., a
hypothetical refs/foo might be meant to change often and drop old
history immediately).

However, sometimes it is useful to override this decision and simply log
for all refs, because the safety and audit trail is more important than
the performance implications of keeping the log around.

This patch introduces a new "always" mode for the core.logallrefupdates
option which will log updates to everything under refs/, regardless
where in the hierarchy it is (we still will not log things like
ORIG_HEAD and FETCH_HEAD, which are known to be transient).

Based-on-patch-by: Jeff King <peff@peff.net>
Signed-off-by: Cornelius Weig <cornelius.weig@tngtech.com>
Reviewed-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

receive-pack: call string_list_clear() unconditionally

string_list_clear() handles empty lists just fine, so remove the
redundant check.

Signed-off-by: Rene Scharfe <l.s.r@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

Completion: Add support for --submodule=diff

Teach git-completion.bash about the 'diff' option to 'git diff
--submodule=', which was added in Git 2.11.

Signed-off-by: Peter Law <PeterJCLaw@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

checkout: convert post_checkout_hook() to struct object_id

Signed-off-by: Rene Scharfe <l.s.r@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

use oidcpy() for copying hashes between instances of struct object_id

Patch generated by Coccinelle and contrib/coccinelle/object_id.cocci.

Signed-off-by: Rene Scharfe <l.s.r@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

use oid_to_hex_r() for converting struct object_id hashes to hex strings

Patch generated by Coccinelle and contrib/coccinelle/object_id.cocci.

Signed-off-by: Rene Scharfe <l.s.r@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

graph: use SWAP macro

Exchange the values of graph->columns and graph->new_columns using the
macro SWAP instead of hand-rolled code. The result is shorter and
easier to read.

This transformation was not done by the semantic patch swap.cocci
because there's an unrelated statement between the second and the last
step of the exchange, so it didn't match the expected pattern.

Signed-off-by: Rene Scharfe <l.s.r@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

diff: use SWAP macro

Use the macro SWAP to exchange the value of pairs of variables instead
of swapping them manually with the help of a temporary variable. The
resulting code is shorter and easier to read.

The two cases were not transformed by the semantic patch swap.cocci
because it's extra careful and handles only cases where the types of all
variables are the same -- and here we swap two ints and use an unsigned
temporary variable for that. Nevertheless the conversion is safe, as
the value range is preserved with and without the patch.

Signed-off-by: Rene Scharfe <l.s.r@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

use SWAP macro

Apply the semantic patch swap.cocci to convert hand-rolled swaps to use
the macro SWAP. The resulting code is shorter and easier to read, the
object code is effectively unchanged.

The patch for object.c had to be hand-edited in order to preserve the
comment before the change; Coccinelle tried to eat it for some reason.

Signed-off-by: Rene Scharfe <l.s.r@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

apply: use SWAP macro

Use the exported macro SWAP instead of the file-scoped macro swap and
remove the latter's definition.

Signed-off-by: Rene Scharfe <l.s.r@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

add SWAP macro

Add a macro for exchanging the values of variables. It allows users
to avoid repetition and takes care of the temporary variable for them.
It also makes sure that the storage sizes of its two parameters are the
same. Its memcpy(1) calls are optimized away by current compilers.

Also add a conservative semantic patch for transforming only swaps of
variables of the same type.

Signed-off-by: Rene Scharfe <l.s.r@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

t0001: don't let a default ACL interfere with the umask test

The "init creates a new deep directory (umask vs. shared)" test expects
the permissions of newly created files to be based on the umask, which
fails if a default ACL is inherited from the working tree for git.  So
attempt to remove a default ACL if there is one.  Same idea as
8ed0a740dd42bd0724aebed6e3b07c4ea2a2d5e8.  (I guess I'm the only one who
ever runs the test suite with a default ACL set.)

Signed-off-by: Matt McCutchen <matt@mattmccutchen.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

config: add markup to core.logAllRefUpdates doc

Signed-off-by: Cornelius Weig <cornelius.weig@tngtech.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

git-p4: fix git-p4.mapUser on Windows

When running git-p4 on Windows, with multiple git-p4.mapUser entries in
git config - no user mappings are applied to the generated repository.

Reproduction Steps:

1. Add multiple git-p4.mapUser entries to git config on a Windows
machine
2. Attempt to clone a p4 repository

None of the user mappings will be applied.

This issue is actually caused by gitConfigList, using split(os.linesep)
to convert the output of git config --get-all into a list. On Windows,
os.linesep is equal to '\r\n' - however git.exe returns configuration
with a line seperator of '\n'.

This leads to the list returned by gitConfigList containing only one
element - which contains the full output of git config --get-all in
string form, which causes problems for the code introduced to
getUserMapFromPerforceServer in 10d08a149d ("git-p4: map a P4 user to
Git author name and email address", 2016-03-01)

This issue should be caught by the test introduced in 10d08a1, however
would require running on Windows to reproduce.

Using splitlines solves this issue, by splitting config on all
typical delimiters ('\n', '\r\n' etc.)

Signed-off-by: George Vanburgh <gvanburgh@bloomberg.net>
Reviewed-by: Luke Diamand <luke@diamand.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

help: improve is_executable() on Windows

On Windows, executables need to have the file extension `.exe`, or they
are not executables. Hence, to support scripts, Git for Windows also
looks for a she-bang line by opening the file in question, and executing
it via the specified script interpreter.

To figure out whether files in the `PATH` are executable, `git help` has
code that imitates this behavior. With one exception: it *always* opens
the files and looks for a she-bang line *or* an `MZ` tell-tale
(nevermind that files with the magic `MZ` but without file extension
`.exe` would still not be executable).

Opening this many files leads to performance problems that are even more
serious when a virus scanner is running. Therefore, let's change the
code to look for the file extension `.exe` early, and avoid opening the
file altogether if we already know that it is executable.

See the following measurements (in seconds) as an example, where we
execute a simple program that simply lists the directory contents and
calls open() on every listed file:

With virus scanner running (coldcache):

$ ./a.exe /libexec/git-core/
before open (git-add.exe): 0.000000
after open (git-add.exe): 0.412873
before open (git-annotate.exe): 0.000175
after open (git-annotate.exe): 0.397925
before open (git-apply.exe): 0.000243
after open (git-apply.exe): 0.399996
before open (git-archive.exe): 0.000147
after open (git-archive.exe): 0.397783
before open (git-bisect--helper.exe): 0.000160
after open (git-bisect--helper.exe): 0.397700
before open (git-blame.exe): 0.000160
after open (git-blame.exe): 0.399136
...

With virus scanner running (hotcache):

$ ./a.exe /libexec/git-core/
before open (git-add.exe): 0.000000
after open (git-add.exe): 0.000325
before open (git-annotate.exe): 0.000229
after open (git-annotate.exe): 0.000177
before open (git-apply.exe): 0.000167
after open (git-apply.exe): 0.000150
before open (git-archive.exe): 0.000154
after open (git-archive.exe): 0.000156
before open (git-bisect--helper.exe): 0.000132
after open (git-bisect--helper.exe): 0.000180
before open (git-blame.exe): 0.000718
after open (git-blame.exe): 0.000724
...

With this patch I get:

$ time git help git
Launching default browser to display HTML ...

real    0m8.723s
user    0m0.000s
sys     0m0.000s

and without

$ time git help git
Launching default browser to display HTML ...

real    1m37.734s
user    0m0.000s
sys     0m0.031s

both tests with cold cache and giving the machine some time to settle
down after restart.

[jes: adjusted the commit message]

Signed-off-by: Heiko Voigt <heiko.voigt@mahr.de>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

mingw: allow hooks to be .exe files

Executable files in Windows need to have the extension '.exe', otherwise
they do not work. Extend the hooks to not just look at the hard coded
names, but also at the names extended by the custom STRIP_EXTENSION,
which is defined as '.exe' in Windows.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

doc: clarify distinction between sign-off and pgp-signing

The documentation for submission discourages pgp-signing, but demands
a proper sign-off by contributors. However, when skimming the headings,
the wording of the section for sign-off could mistakenly be understood
as concerning pgp-signing. Thus, new contributors could oversee the
necessary sign-off.

This commit improves the wording such that the section about sign-off
cannot be misunderstood as pgp-signing. In addition, the paragraph about
pgp-signing is changed such that it avoids the impression that
pgp-signing could be relevant at later stages of the submission.

Signed-off-by: Cornelius Weig <cornelius.weig@tngtech.com>
Helped-by: Philip Oakley <philipoakley@iee.org>
Helped-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

test-lib: on FreeBSD, look for unzip(1) in /usr/local/bin/

Eric Wong reported that while FreeBSD has a /usr/bin/unzip, it uses
different semantics from those that are needed by Git's tests: When
passing the -a option to Info-Zip, it heeds the text attribute of the
.zip file's central directory, while FreeBSD's unzip ignores that
attribute.

The common work-around is to install Info-Zip on FreeBSD, into
/usr/local/bin/.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Tested-by: Eric Wong <e@80x24.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

t/Makefile: add a rule to re-run previously-failed tests

This patch automates the process of determining which tests failed
previously and re-running them.

While developing patch series, it is a good practice to run the test
suite from time to time, just to make sure that obvious bugs are caught
early. With complex patch series, it is common to run `make -j15 -k
test`, i.e. run the tests in parallel and *not* stop at the first
failing test but continue. This has the advantage of identifying
possibly multiple problems in one big test run.

It is particularly important to reduce the turn-around time thusly on
Windows, where the test suite spends 45 minutes on the computer on which
this patch was developed.

It is the most convenient way to determine which tests failed after
running the entire test suite, in parallel, to look for left-over "trash
directory.t*" subdirectories in the t/ subdirectory. However, those
directories might live outside t/ when overridden using the
--root=<directory> option, to which the Makefile has no access. The next
best method is to grep explicitly for failed tests in the test-results/
directory, which the Makefile *can* access.

Please note that the often-recommended `prove` tool requires Perl, and
that opens a whole new can of worms on Windows. As no native Windows Perl
comes with Subversion bindings, we have to use a Perl in Git for Windows
that uses the POSIX emulation layer named MSYS2 (which is a portable
version of Cygwin). When using this emulation layer under stress, e.g.
when running massively-parallel tests, unexplicable crashes occur quite
frequently, and instead of having a solution to the original problem, the
developer now has an additional, quite huge problem. For that reason, this
developer rejected `prove` as a solution and went with this patch instead.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Reviewed-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

use absolute_pathdup()

Apply the semantic patch for converting callers that duplicate the
result of absolute_path() to call absolute_pathdup() instead, which
avoids an extra string copy to a static buffer.

Signed-off-by: Rene Scharfe <l.s.r@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

abspath: add absolute_pathdup()

Add a function that returns a buffer containing the absolute path of its
argument and a semantic patch for its intended use. It avoids an extra
string copy to a static buffer.

Signed-off-by: Rene Scharfe <l.s.r@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

status: be prepared for not-yet-started interactive rebase

Some developers might want to call `git status` in a working
directory where they just started an interactive rebase, but the
edit script is still opened in the editor.

Let's show a meaningful message in such cases.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

submodule update: run custom update script for initial populating as well

In 1b4735d9f3 (submodule: no [--merge|--rebase] when newly cloned,
2011-02-17), all actions were defaulted to checkout for populating
a submodule initially, because merging or rebasing makes no sense
in that situation.

Other commands however do make sense, such as the custom command
that was added later (6cb5728c43, submodule update: allow custom
command to update submodule working tree, 2013-07-03).

I am unsure about the "none" command, as I can see an initial
checkout there as a useful thing. On the other hand going strictly
by our own documentation, we should do nothing in case of "none"
as well, because the user asked for it.

Reported-by: Han-Wen Nienhuys <hanwen@google.com>
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

submodule absorbing: fix worktree/gitdir pointers recursively for non-moves

Consider having a submodule 'sub' and a nested submodule at 'sub/nested'.
When nested is already absorbed into sub, but sub is not absorbed into
its superproject, then we need to fixup the gitfile and core.worktree
setting for 'nested' when absorbing 'sub', but we do not need to move
its git dir around.

Previously 'nested's gitfile contained "gitdir: ../.git/modules/nested";
it has to be corrected to "gitdir: ../../.git/modules/sub1/modules/nested".

An alternative I considered to do this work lazily, i.e. when resolving
"../.git/modules/nested", we would notice the ".git" being a gitfile
linking to another path. That seemed to be robuster by design, but harder
to get the implementation right. Maybe we have to do that anyway once we
try to have submodules and worktrees working nicely together, but for now
just produce 'correct' (i.e. direct) pointers.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

cache.h: expose the dying procedure for reading gitlinks

In a later patch we want to react to only a subset of errors, defaulting
the rest to die as usual. Separate the block that takes care of dying
into its own function so we have easy access to it.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

setup: add gentle version of resolve_git_dir

This follows a93bedada (setup: add gentle version of read_gitfile,
2015-06-09), and assumes the same reasoning. resolve_git_dir is unsuited
for speculative calls, so we want to use the gentle version to find out
about potential errors.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

fsck: lazily load types under --connectivity-only

The recent fixes to "fsck --connectivity-only" load all of
the objects with their correct types. This keeps the
connectivity-only code path close to the regular one, but it
also introduces some unnecessary inefficiency. While getting
the type of an object is cheap compared to actually opening
and parsing the object (as the non-connectivity-only case
would do), it's still not free.

For reachable non-blob objects, we end up having to parse
them later anyway (to see what they point to), making our
type lookup here redundant.

For unreachable objects, we might never hit them at all in
the reachability traversal, making the lookup completely
wasted. And in some cases, we might have quite a few
unreachable objects (e.g., when alternates are used for
shared object storage between repositories, it's normal for
there to be objects reachable from other repositories but
not the one running fsck).

The comment in mark_object_for_connectivity() claims two
benefits to getting the type up front:

  1. We need to know the types during fsck_walk(). (And not
     explicitly mentioned, but we also need them when
     printing the types of broken or dangling commits).

     We can address this by lazy-loading the types as
     necessary. Most objects never need this lazy-load at
     all, because they fall into one of these categories:

       a. Reachable from our tips, and are coerced into the
  correct type as we traverse (e.g., a parent link
  will call lookup_commit(), which converts OBJ_NONE
  to OBJ_COMMIT).

       b. Unreachable, but not at the tip of a chunk of
          unreachable history. We only mention the tips as
  "dangling", so an unreachable commit which links
  to hundreds of other objects needs only report the
  type of the tip commit.

  2. It serves as a cross-check that the coercion in (1a) is
     correct (i.e., we'll complain about a parent link that
     points to a blob). But we get most of this for free
     already, because right after coercing, we'll parse any
     non-blob objects. So we'd notice then if we expected a
     commit and got a blob.

     The one exception is when we expect a blob, in which
     case we never actually read the object contents.

     So this is a slight weakening, but given that the whole
     point of --connectivity-only is to sacrifice some data
     integrity checks for speed, this seems like an
     acceptable tradeoff.

Here are before and after timings for an extreme case with
~5M reachable objects and another ~12M unreachable (it's the
torvalds/linux repository on GitHub, connected to shared
storage for all of the other kernel forks):

  [before]
  $ time git fsck --no-dangling --connectivity-only
  real 3m4.323s
  user 1m25.121s
  sys 1m38.710s

  [after]
  $ time git fsck --no-dangling --connectivity-only
  real 0m51.497s
  user 0m49.575s
  sys 0m1.776s

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

fsck: move typename() printing to its own function

When an object has a problem, we mention its type. But we do
so by feeding the result of typename() directly to
fprintf(). This is potentially dangerous because typename()
can return NULL for some type values (like OBJ_NONE).

It's doubtful that this can be triggered in practice with
the current code, so this is probably not fixing a bug. But
it future-proofs us against modifications that make things
like OBJ_NONE more likely (and gives future patches a
central point to handle them).

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

relink: really remove the command

The files in contrib/examples are meant to illustrate "you could
combine plumbing commands to implement something like these"; this
is an opposite and is an example of what not to do, e.g. accessing
the object store directly bypassing Git.

Remove it.

Suggested-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

relink: retire the command

Back in the olden days, when all objects were loose and rubber boots were
made out of wood, it made sense to try to share (immutable) objects
between repositories.

Ever since the arrival of pack files, it is but an anachronism.

Let's move the script to the contrib/examples/ directory and no longer
offer it.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

difftool: hack around -Wzero-length-format warning

Building with "gcc -Wall" will complain that the format in:

warning("")

is empty. Which is true, but the warning is over-eager. We
are calling the function for its side effect of printing
"warning:", even with an empty string.

Our DEVELOPER Makefile knob disables the warning, but not
everybody uses it. Let's silence the warning in the code so
that nobody reports it or tries to "fix" it.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

unpack-trees: support super-prefix option

In the future we want to support working tree operations within submodules,
e.g. "git checkout --recurse-submodules", which will update the submodule
to the commit as recorded in its superproject. In the submodule the
unpack-tree operation is carried out as usual, but the reporting to the
user needs to prefix any path with the superproject. The mechanism for
this is the super-prefix. (see 74866d757, git: make super-prefix option)

Add support for the super-prefix option for commands that unpack trees
by wrapping any path output in unpacking trees in the newly introduced
super_prefixed function. This new function prefixes any path with the
super-prefix if there is one. Assuming the submodule case doesn't happen
in the majority of the cases, we'd want to have a fast behavior for no
super prefix, i.e. no reallocation/copying, but just returning path.

Another aspect of introducing the `super_prefixed` function is to consider
who owns the memory and if this is the right place where the path gets
modified. As the super prefix ought to change the output behavior only and
not the actual unpack tree part, it is fine to be that late in the line.
As we get passed in 'const char *path', we cannot change the path itself,
which means in case of a super prefix we have to copy over the path.
We need two static buffers in that function as the error messages
contain at most two paths.

For testing purposes enable it in read-tree, which has no output
of paths other than an unpack-trees.c. These are all converted in
this patch.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

t1450: use "mv -f" within loose object directory

The loose objects are created with mode 0444. That doesn't
prevent them being overwritten by rename(), but some
versions of "mv" will be extra careful and prompt the user,
even without "-i".

Reportedly macOS does this, at least in the Travis builds.
The prompt reads from /dev/null, defaulting to "no", and the
object isn't moved. Then to make matters even more
interesting, it still returns "0" and the rest of the test
proceeds, but with a broken setup.

We can work around it by using "mv -f" to override the
prompt. This should work as it's already used in t5504 for
the same purpose.

Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

show-ref: remove a stale comment

When cf0adba788 ("Store peeled refs in packed-refs file.",
2006-11-19) made the command to die with a message on error even
when --quiet is passed, it left the comment to say it changed the
semantics. But that kind of information belongs to the log message,
not in-code comment. Besides, the behaviour after the change has
been the established one for the past 10 years ;-)

Signed-off-by: Junio C Hamano <gitster@pobox.com>

log --graph: customize the graph lines with config log.graphColors

If you have a 256 colors terminal (or one with true color support), then
the predefined 12 colors seem limited. On the other hand, you don't want
to draw graph lines with every single color in this mode because the two
colors could look extremely similar. This option allows you to hand pick
the colors you want.

Even with standard terminal, if your background color is neither black
or white, then the graph line may match your background and become
hidden. You can exclude your background color (or simply the colors you
hate) with this.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

Seventh batch for 2.12

Signed-off-by: Junio C Hamano <gitster@pobox.com>

Merge branch 'ws/request-pull-code-cleanup'

Code clean-up.

* ws/request-pull-code-cleanup:
request-pull: drop old USAGE stuff

Merge branch 'rh/diff-orderfile-doc'

Documentation fix.

* rh/diff-orderfile-doc:
diff: document the format of the -O (diff.orderFile) file
diff: document behavior of relative diff.orderFile

Merge branch 'sb/cd-then-git-can-be-written-as-git-c'

Test clean-up.

* sb/cd-then-git-can-be-written-as-git-c:
lib-submodule-update.sh: reduce use of subshell by using "git -C"

Merge branch 'ad/bisect-terms'

Documentation fix.

* ad/bisect-terms:
Documentation/bisect: improve on (bad|new) and (good|bad)

Merge branch 'sg/fix-versioncmp-with-common-suffix'

The prereleaseSuffix feature of version comparison that is used in
"git tag -l" did not correctly when two or more prereleases for the
same release were present (e.g. when 2.0, 2.0-beta1, and 2.0-beta2
are there and the code needs to compare 2.0-beta1 and 2.0-beta2).

* sg/fix-versioncmp-with-common-suffix:
  versioncmp: generalize version sort suffix reordering
  versioncmp: factor out helper for suffix matching
  versioncmp: use earliest-longest contained suffix to determine sorting order
  versioncmp: cope with common part overlapping with prerelease suffix
  versioncmp: pass full tagnames to swap_prereleases()
  t7004-tag: add version sort tests to show prerelease reordering issues
  t7004-tag: use test_config helper
  t7004-tag: delete unnecessary tags with test_when_finished

Merge branch 'vn/diff-ihc-config'

"git diff" learned diff.interHunkContext configuration variable
that gives the default value for its --inter-hunk-context option.

* vn/diff-ihc-config:
diff: add interhunk context config option

Merge branch 'sb/submodule-init'

Error message fix.

* sb/submodule-init:
submodule update --init: display correct path from submodule

Merge branch 'sb/submodule-embed-gitdir'

Help-text fix.

* sb/submodule-embed-gitdir:
submodule absorbgitdirs: mention in docstring help

Merge branch 'sb/submodule-config-tests'

Test updates.

* sb/submodule-config-tests:
t7411: test lookup of uninitialized submodules
t7411: quote URLs

Merge branch 'jk/grep-e-could-be-extended-beyond-posix'

Tighten a test to avoid mistaking an extended ERE regexp engine as
a PRE regexp engine.

* jk/grep-e-could-be-extended-beyond-posix:
t7810: avoid assumption about invalid regex syntax

Merge branch 'bw/read-blob-data-does-not-modify-index-state'

Code clean-up.

* bw/read-blob-data-does-not-modify-index-state:
index: improve constness for reading blob data

show-ref: remove dead `if (verify)' check

As show_ref() is only ever called on the path where --verify is not
specified, `verify' can never possibly be true here.

Signed-off-by: Vladimir Panteleev <git@thecybershadow.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

show-ref: detect dangling refs under --verify as well

Move detection of dangling refs into show_one(), so that they are
detected when --verify is present as well as when it is absent.

Signed-off-by: Vladimir Panteleev <git@thecybershadow.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

show-ref: move --quiet handling into show_one()

Do the same with --quiet as was done with -d, to remove the need to
perform this check at show_one()'s call site from the --verify branch.

Signed-off-by: Vladimir Panteleev <git@thecybershadow.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

show-ref: allow -d to work with --verify

Move handling of -d into show_one(), so that it takes effect when
--verify is present as well as when it is absent. This is useful when
the user wishes to avoid the costly iteration of refs.

Signed-off-by: Vladimir Panteleev <git@thecybershadow.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

show-ref: accept HEAD with --verify

Previously, when --verify was specified, show-ref would use a separate
code path which did not handle HEAD and treated it as an invalid
ref. Thus, "git show-ref --verify HEAD" (where "--verify" is used
because the user is not interested in seeing refs/remotes/origin/HEAD)
did not work as expected.

Instead of insisting that the input begins with "refs/", allow "HEAD"
as well in the codepath that handles "--verify", so that all valid
full refnames including HEAD are passed to the same output machinery.

Signed-off-by: Vladimir Panteleev <git@thecybershadow.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

ref-filter: use QSORT_S in ref_array_sort()

Pass the array of sort keys to compare_refs() via the context parameter
of qsort_s() instead of using a global variable; that's cleaner and
simpler. If ref_array_sort() is to be called from multiple parallel
threads then care still needs to be taken that the global variable
used_atom is not modified concurrently.

Signed-off-by: Rene Scharfe <l.s.r@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

string-list: use QSORT_S in string_list_sort()

Pass the comparison function to cmp_items() via the context parameter of
qsort_s() instead of using a global variable.  That allows calling
string_list_sort() from multiple parallel threads.

Our qsort_s() in compat/ is slightly slower than qsort(1) from glibc
2.24 for sorting lots of lines:

Test                         HEAD^             HEAD
---------------------------------------------------------------------
0071.2: sort(1)              0.10(0.22+0.01)   0.09(0.21+0.00) -10.0%
0071.3: string_list_sort()   0.16(0.15+0.01)   0.17(0.15+0.00) +6.3%

GNU sort(1) version 8.26 is significantly faster because it uses
multiple parallel threads; with the unportable option --parallel=1 it
becomes slower:

Test                         HEAD^             HEAD
--------------------------------------------------------------------
0071.2: sort(1)              0.21(0.18+0.01)   0.20(0.18+0.01) -4.8%
0071.3: string_list_sort()   0.16(0.13+0.02)   0.17(0.15+0.01) +6.3%

There is some instability -- the numbers for the sort(1) check shouldn't
be affected by this patch.  Anyway, the performance of our qsort_s()
implementation is apparently good enough, at least for this test.

Signed-off-by: Rene Scharfe <l.s.r@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

perf: add basic sort performance test

Add a sort command to test-string-list that reads lines from stdin,
stores them in a string_list and then sorts it. Use it in a simple
perf test script to measure the performance of string_list_sort().

Signed-off-by: Rene Scharfe <l.s.r@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

add QSORT_S

Add the macro QSORT_S, a convenient wrapper for qsort_s() that infers
the size of the array elements and dies on error.

Basically all possible errors are programming mistakes (passing NULL as
base of a non-empty array, passing NULL as comparison function,
out-of-bounds accesses), so terminating the program should be acceptable
for most callers.

Signed-off-by: Rene Scharfe <l.s.r@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

compat: add qsort_s()

The function qsort_s() was introduced with C11 Annex K; it provides the
ability to pass a context pointer to the comparison function, supports
the convention of using a NULL pointer for an empty array and performs a
few safety checks.

Add an implementation based on compat/qsort.c for platforms that lack a
native standards-compliant qsort_s() (i.e. basically everyone). It
doesn't perform the full range of possible checks: It uses size_t
instead of rsize_t and doesn't check nmemb and size against RSIZE_MAX
because we probably don't have the restricted size type defined. For
the same reason it returns int instead of errno_t.

Signed-off-by: Rene Scharfe <l.s.r@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

Makefile: add a knob to enable the use of Asciidoctor

While Git has traditionally built its documentation using AsciiDoc, some
people wish to use Asciidoctor for speed or other reasons.  Add a
Makefile knob, USE_ASCIIDOCTOR, that sets various options in order to
produce acceptable output.  For HTML output, XHTML5 was chosen, since
the AsciiDoc options also produce XHTML, albeit XHTML 1.1.

Asciidoctor does not have built-in support for the linkgit macro, but it
is available using the Asciidoctor Extensions Lab.  Add a macro to
enable the use of this extension if it is available.  Without it, the
linkgit macros are emitted into the output.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

Documentation: move dblatex arguments into variable

Our dblatex invocation uses several style components from the AsciiDoc
distribution, but those components are not available when building with
Asciidoctor. Move the command line arguments into a variable so it can
be overridden by the user or makefile configuration options.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

Documentation: add XSLT to fix DocBook for Texinfo

There are two ways to create a section in a reference document (i.e.,
manpage) in DocBook 4: refsection elements and refsect, refsect2, and
refsect3 elements.  Either form is acceptable as of DocBook 4.2, but
they cannot be mixed.  Prior to DocBook 4.2, only the numbered forms
were acceptable.

docbook2texi only accepts the numbered forms, and this has not generally
been a problem, since AsciiDoc produces the numbered forms.
Asciidoctor, on the other hand, uses a shared backend for DocBook 4 and
5, and uses the unnumbered refsection elements instead.

If we don't convert the unnumbered form to the numbered form,
docbook2texi omits section headings, which is undesirable.  Add an XSLT
stylesheet to transform the unnumbered forms to the numbered forms
automatically, and preprocess the DocBook XML as part of the
transformation to Texinfo format.

Note that this transformation is only necessary for Texinfo, since
docbook2texi provides its own stylesheets.  The DocBook stylesheets,
which we use for other formats, provide the full range of DocBook 4 and
5 compatibility, and don't have this issue.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

Documentation: sort sources for gitman.texi

Sorting the sources makes it easier to compare the output using diff.
In addition, it aids groups creating reproducible builds, as the order
of the files is no longer dependent on the file system or other
irrelevant factors.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

Documentation: remove unneeded argument in cat-texi.perl

The newly-added use of the warnings pragma exposes that the $menu[0]
argument to printf has long been silently ignored, since there is no
format specifier for it. It doesn't appear that the argument is
actually needed, either: there is no reason to insert the name of one
particular documentation page anywhere in the header that's being
generated.

Remove the unused argument, and since the format specification
functionality is no longer needed, convert the printf to a simple print.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

Documentation: modernize cat-texi.perl

Good style for Perl includes using the strict and warnings pragmas, and
preferring lexical file handles over bareword file handles. Using
lexical file handles necessitates being explicit when $_ is printed, so
that Perl does not get confused and instead print the glob ref.

The benefit of this modernization is that a formerly obscured bug is now
visible, which will be fixed in a followup patch.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

Documentation: fix warning in cat-texi.perl

Newer versions of Perl produce the warning "Unescaped left brace in
regex is deprecated, passed through in regex" when an unescaped left
brace occurs in a regex. Escape the brace to avoid this warning.

Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

travis-ci: fix Perforce install on macOS

The `perforce` and `perforce-server` package were moved from brew [1][2]
to cask [3]. Teach TravisCI the new location.

Perforce updates their binaries without version bumps. That made the
brew install (legitimately!) fail due to checksum mismatches. The
workaround is not necessary anymore as Cask [4] allows to disable the
checksum test for individual formulas.

[1] https://github.com/Homebrew/homebrew-binary/commit/1394e42de04d07445f82f9512627e864ff4ca4c6
[2] https://github.com/Homebrew/homebrew-binary/commit/f8da22d6b8dbcfcfdb2dfa9ac1a5e5d8e05aac2b
[3] https://github.com/caskroom/homebrew-cask/pull/29180
[4] https://caskroom.github.io/

Signed-off-by: Lars Schneider <larsxschneider@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

contrib: remove git-convert-objects

git-convert-objects, originally named git-convert-cache was used in
early 2005 to convert ancient repositories where objects are named
after the hash of their compressed contents to the current object
naming sheme where they are named after the hash of their pre-compression
contents.

By now the need for conversion of the very early repositories is
less relevant, we no longer need to keep it in contrib; remove it.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

remote rename: more carefully determine whether a remote is configured

One of the really nice features of the ~/.gitconfig file is that users
can override defaults by their own preferred settings for all of their
repositories.

One such default that some users like to override is whether the
"origin" remote gets auto-pruned or not. The user would simply call

git config --global remote.origin.prune true

and from now on all "origin" remotes would be pruned automatically when
fetching into the local repository.

There is just one catch: now Git thinks that the "origin" remote is
configured, even if the repository config has no [remote "origin"]
section at all, as it does not realize that the "prune" setting was
configured globally and that there really is no "origin" remote
configured in this repository.

That is a problem e.g. when renaming a remote to a new name, when Git
may be fooled into thinking that there is already a remote of that new
name.

Let's fix this by paying more attention to *where* the remote settings
came from: if they are configured in the local repository config, we
must not overwrite them. If they were configured elsewhere, we cannot
overwrite them to begin with, as we only write the repository config.

There is only one caller of remote_is_configured() (in `git fetch`) that
may want to take remotes into account even if they were configured
outside the repository config; all other callers essentially try to
prevent the Git command from overwriting settings in the repository
config.

To accommodate that fact, the remote_is_configured() function now
requires a parameter that states whether the caller is interested in all
remotes, or only in those that were configured in the repository config.

Many thanks to Jeff King whose tireless review helped with settling for
nothing less than the current strategy.

This fixes https://github.com/git-for-windows/git/issues/888

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

remote rename: demonstrate a bogus "remote exists" bug

Some users like to set `remote.origin.prune = true` in their ~/.gitconfig
so that all of their repositories use that default.

However, our code is ill-prepared for this, mistaking that single entry to
mean that there is already a remote of the name "origin", even if there is
not.

This patch adds a test case demonstrating this issue.

Reported by Andrew Arnott.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

difftool: retire the scripted version

It served its purpose, but now we have a builtin difftool. Time for the
Perl script to enjoy Florida.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

difftool: implement the functionality in the builtin

This patch gives life to the skeleton added in the previous patch.

The motivation for converting the difftool is that Perl scripts are not at
all native on Windows, and that `git difftool` therefore is pretty slow on
that platform, when there is no good reason for it to be slow.

In addition, Perl does not really have access to Git's internals. That
means that any script will always have to jump through unnecessary
hoops, and it will often need to perform unnecessary work (e.g. when
reading the entire config every time `git config` is called to query a
single config value).

The current version of the builtin difftool does not, however, make full
use of the internals but instead chooses to spawn a couple of Git
processes, still, to make for an easier conversion. There remains a lot
of room for improvement, left later.

Note: to play it safe, the original difftool is still called unless the
config setting difftool.useBuiltin is set to true.

The reason: this new, experimental, builtin difftool was shipped as part
of Git for Windows v2.11.0, to allow for easier large-scale testing, but
of course as an opt-in feature.

The speedup is actually more noticable on Linux than on Windows: a quick
test shows that t7800-difftool.sh runs in (2.183s/0.052s/0.108s)
(real/user/sys) in a Linux VM, down from (6.529s/3.112s/0.644s), while on
Windows, it is (36.064s/2.730s/7.194s), down from (47.637s/2.407s/6.863s).
The culprit is most likely the overhead incurred from *still* having to
shell out to mergetool-lib.sh and difftool--helper.sh.

Still, it is an improvement.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

documentation: retire unfinished documentation

When looking for documentation for a specific function, you may be tempted
to run

git -C Documentation grep index_name_pos

only to find the file technical/api-in-core-index.txt, which doesn't
help for understanding the given function. It would be better to not find
these functions in the documentation, such that people directly dive into
the code instead.

In the previous patches we have documented
* index_name_pos()
* remove_index_entry_at()
* add_[file_]to_index()
in cache.h

We already have documentation for:
* add_index_entry()
* read_index()

Which leaves us with a TODO for:
* cache -> the_index macros
* refresh_index()
* discard_index()
* ie_match_stat() and ie_modified(); how they are different and when to
use which.
* write_index() that was renamed to write_locked_index
* cache_tree_invalidate_path()
* cache_tree_update()

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

cache.h: document add_[file_]to_index

Helped-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

cache.h: document remove_index_entry_at

Do this by moving the existing documentation from
read-cache.c to cache.h.

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

cache.h: document index_name_pos

Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

color.c: trim leading spaces in color_parse_mem()

Normally color_parse_mem() is called from config parser which trims the
leading spaces already. The new caller in the next patch won't. Let's be
tidy and trim leading spaces too (we already trim trailing spaces
after a word).

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

color.c: fix color_parse_mem() with value_len == 0

In this code we want to match the word "reset". If len is zero,
strncasecmp() will return zero and we incorrectly assume it's "reset" as
a result.

Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>

clear_delta_base_cache(): don't modify hashmap while iterating

On Thu, Jan 19, 2017 at 03:03:46PM +0100, Ulrich Spörlein wrote:

> > I suspect the patch below may fix things for you. It works around it by
> > walking over the lru list (either is fine, as they both contain all
> > entries, and since we're clearing everything, we don't care about the
> > order).
>
> Confirmed. With the patch applied, I can import the whole 55G in one go
> without any crashes or aborts. Thanks much!

Thanks. Here it is rolled up with a commit message.

-- >8 --
Subject: clear_delta_base_cache(): don't modify hashmap while iterating

Removing entries while iterating causes fast-import to
access an already-freed `struct packed_git`, leading to
various confusing errors.

What happens is that clear_delta_base_cache() drops the
whole contents of the cache by iterating over the hashmap,
calling release_delta_base_cache() on each entry. That
function removes the item from the hashmap. The hashmap code
may then shrink the table, but the hashmap_iter struct
retains an offset from the old table.

As a result, the next call to hashmap_iter_next() may claim
that the iteration is done, even though some items haven't
been visited.

The only caller of clear_delta_base_cache() is fast-import,
which wants to clear the cache because it is discarding the
packed_git struct for its temporary pack. So by failing to
remove all of the entries, we still have references to the
freed packed_git.

To make things even more confusing, this doesn't seem to
trigger with the test suite, because it depends on
complexities like the size of the hash table, which entries
got cleared, whether we try to access them before they're
evicted from the cache, etc.

So I've been able to identify the problem with large
imports like freebsd's svn import, or a fast-export of
linux.git. But nothing that would be reasonable to run as
part of the normal test suite.

We can fix this easily by iterating over the lru linked list
instead of the hashmap. They both contain the same entries,
and we can use the "safe" variant of the list iterator,
which exists for exactly this case.

Let's also add a warning to the hashmap API documentation to
reduce the chances of getting bit by this again.

Reported-by: Ulrich Spörlein <uqs@freebsd.org>
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>