fix(clone): third review of #175: the cases in which an unfinished clone was refused where it used to be completed

The third review confirmed, under real process kills, that a checkout that
was cut off leaves no index, and found no way left to replace or delete a
user's file. It found five narrow cases in which gclo refused, failed or said
done where it used to complete the tree.

- When the branch of an unfinished checkout must move before it is written
  (a tree Windows cannot write, or a history origin rewrote), the commit it
  leaves is recorded, and its files on disk are recognised by it as well.
  A history origin rewrote is followed again, as on main.
- A file is the unfinished checkout's own when it holds what the commit
  stores or what the checkout filters make of it (the 'ident' attribute).
- Where the repository has a symbolic link, a link in the folder is the
  checkout's own.
- A marked folder is taken for finished only when its index and its files
  hold the whole tree of its commit: an index somebody's git wrote over an
  unfinished tree does not make it whole.
- A link where the repository has a submodule is refused; a link is something
  with a target, not anything with a reparse point.
- Tests for each, for a file stored with carriage returns, for a completion
  that fails half-way, and for the order of what following origin's default
  branch writes. The message of an ordinary update that would replace files
  no longer says who changed them.

Refs #174

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Signed-off-by: James Maes <james@kof22.com>
This commit is contained in:
james.maesandClaude Fable 5.2 committed 2026-11-10 22:30:40 -06:00
1 parent 099c2b5ee8
commit 41f3712a8a
5 files changed
+537 -69

No files matched your search

+1 -1
View File
@@ -112,7 +112,7 @@ On launch, gclo shows a brief branded splash, then the **Quick Sync** connect ca
8. **Cancel** at any time: in-flight repos stop cleanly, unstarted repos are marked Canceled, and the summary still appears. After a run, the result bar offers **Retry failed** when something failed (it re-runs exactly the failed rows), otherwise **Open folder**. The folder button in the toolbar opens the target folder in Explorer whenever no sync is running.
**When something goes wrong:** failures are isolated per repository — one broken repo never stops the rest, and a failed row shows its error message inline. Failed clones are cleaned up, and a clone whose checkout never finished (a crash, a kill, a cleanup that could not delete the folder) is flagged inside its `.git`, so a partial working tree is never mistaken for an up-to-date repo on the next run — gclo completes the checkout instead. What it completes is its own. On Windows, a file that is already where the repository has one is replaced only when it is what the unfinished checkout wrote there (its content, or the beginning of it, line endings aside); for anything else the row fails and names the file: move it and sync again. The same check guards a repository that was empty when it was cloned and gets its first files later. Such a clone follows the origin's default branch when nothing has been put into its folder; a clone that has branches of its own is left as it is when its HEAD names a branch without commits (a branch you started there yourself). While a canceled clone or update is still ending, syncing that repository again fails with a message to try again when it has finished. Repositories with Windows-impossible paths are the deliberate exception: gclo checks every path in the incoming tree against Windows file-system rules *before* checkout — invalid characters, reserved device names like `CON`, trailing spaces or dots, names that differ only by case — and when it finds any, the repo is marked Failed with a per-path list of reasons, but the fully fetched `.git` is **kept** (nothing was ever checked out). Recovery happens in place, with no re-download: click the row's **Resolve…** link to rename each offending path (safe names are pre-suggested) or skip it, and gclo materializes the working tree right there; the CLI does the same automatically with [`--sanitize-paths`](docs/CLI.md#windows-invalid-paths-and---sanitize-paths). This applies to a repository gclo could not check out when it cloned it. A repository that was already checked out normally is never rewritten: when an update brings it such a path, it fails and nothing in it is changed, and **Resolve…** (or `--sanitize-paths`) then says how to get a fresh clone with the renames instead of touching it. When WSL is installed with git in its default distribution, the same dialog offers **Clone in WSL instead**: the repository is cloned unchanged into `~/gclo/<org>/<repo>` inside WSL (reachable from Explorer as `\wsl.localhost\<distro>\home\...`), the token travels only through the environment, and the Windows copy is left as it is. Long paths beyond the classic 260-character limit are handled automatically (`core.longpaths`).
**When something goes wrong:** failures are isolated per repository — one broken repo never stops the rest, and a failed row shows its error message inline. Failed clones are cleaned up, and a clone whose checkout never finished (a crash, a kill, a cleanup that could not delete the folder) is flagged inside its `.git`, so a partial working tree is never mistaken for an up-to-date repo on the next run — gclo completes the checkout instead. What it completes is its own. On Windows, a file that is already where the repository has one is replaced only when it is what the unfinished checkout wrote there (its content, or the beginning of it, line endings aside); for anything else the row fails and names the file: move it and sync again. The same check guards a repository that was empty when it was cloned and gets its first files later. Such a clone follows the origin's default branch when nothing has been put into its folder (otherwise it keeps the branch its HEAD names, and stays empty until origin has that branch); a clone that has branches of its own is left as it is when its HEAD names a branch without commits (a branch you started there yourself). If you have worked in an unfinished clone with git in the meantime, gclo finishes it only when every file there is still the repository's own. While a canceled clone or update is still ending, syncing that repository again fails with a message to try again when it has finished. Repositories with Windows-impossible paths are the deliberate exception: gclo checks every path in the incoming tree against Windows file-system rules *before* checkout — invalid characters, reserved device names like `CON`, trailing spaces or dots, names that differ only by case — and when it finds any, the repo is marked Failed with a per-path list of reasons, but the fully fetched `.git` is **kept** (nothing was ever checked out). Recovery happens in place, with no re-download: click the row's **Resolve…** link to rename each offending path (safe names are pre-suggested) or skip it, and gclo materializes the working tree right there; the CLI does the same automatically with [`--sanitize-paths`](docs/CLI.md#windows-invalid-paths-and---sanitize-paths). This applies to a repository gclo could not check out when it cloned it. A repository that was already checked out normally is never rewritten: when an update brings it such a path, it fails and nothing in it is changed, and **Resolve…** (or `--sanitize-paths`) then says how to get a fresh clone with the renames instead of touching it. When WSL is installed with git in its default distribution, the same dialog offers **Clone in WSL instead**: the repository is cloned unchanged into `~/gclo/<org>/<repo>` inside WSL (reachable from Explorer as `\wsl.localhost\<distro>\home\...`), the token travels only through the environment, and the Windows copy is left as it is. Long paths beyond the classic 260-character limit are handled automatically (`core.longpaths`).
**Settings** (File > Settings…) let you set a default GitHub token (stored in the Windows Credential Manager), the default target folder (with a folder picker), the default parallelism, and the theme (System, Light, or Dark); an Advanced section toggles the startup splash and its duration.
+1 -1
View File
@@ -121,4 +121,4 @@ flowchart LR
| 2026-11-10 | v1.1.1 | James Maes (@KofTwentyTwo) | T3/T4 updated: path recovery writes and deletes only through `WorkingTree` (inside the working directory, never in `.git`, never through a link); `.git` and its Windows aliases are invalid path segments; the stored recovery file's paths are validated on load; cleanup of a failed clone does not follow links (#141) |
| 2026-11-10 | v1.1.1 | James Maes (@KofTwentyTwo) | T15 added: text from repositories and servers is shown with control, separator and format characters made visible, in the activity log and in everything the CLI prints as text, including characters a legacy console code page would turn into control bytes; the desktop app's surfaces are named as not covered (#143) |
| 2026-11-10 | v1.1.1 | James Maes (@KofTwentyTwo) | The project's home moved to `git.kof22.com`: the Help menu, the About dialog, the CLI's error text and the documentation link there, and the Scoop bucket is added from there (the manifest that pins the download URL and its hash now comes from that server over HTTPS; T2 is unchanged in kind). Help > Share hands a fixed text (what gclo is, where to download it, the sender's gclo version) to the system's mail handler as a `mailto:` link with no recipient, or shows it and copies it to the clipboard; nothing about the user, their accounts, or their repositories is in it, and the address it is built from must be a plain http(s) host and path (#163, #165). The update feed and the release downloads named in package manifests are still GitHub Releases at this revision. |
| 2026-11-10 | v1.1.1 | James Maes (@KofTwentyTwo) | Working tree (#174; the T3 and T4 rows are unchanged, this adds to what they rely on). On Windows, a forced checkout into a folder whose tree is not finished (the first checkout of a clone that was empty, and the completion of a checkout that was cut off) is preceded by refusals that change nothing: a link on the way or at a destination, a file where a folder is needed (also on the way to a submodule), a folder that is not empty where a file is needed, a file at a destination that is not what the unfinished checkout wrote there (the commit's content or the beginning of it, carriage returns aside), and anything but an empty folder where the repository has a submodule. An unfinished checkout is recognised by its marker and the absence of an index; a marked folder that has an index is updated like any clone, without force. It is completed at the commit its branch is on, before the branch moves. A repository that has local branches is not treated as an empty clone; one without follows origin's default branch only when its folder holds nothing but `.git`. The folder is marked unfinished before its branch is created. One git operation per folder at a time within a process (not across processes, and not across spellings of a path that only the file system can resolve). The WSL clone is made in `~/gclo/.gclo-tmp` and renamed into place when complete (`mv -T`, after a check that nothing is there). Not covered: a file created in the folder between the check and the checkout that follows it; an empty or identical file of the user's at a destination, which is replaced; platforms other than Windows, where the checkout is forced as before. |
| 2026-11-10 | v1.1.1 | James Maes (@KofTwentyTwo) | Working tree (#174; the T3 and T4 rows are unchanged, this adds to what they rely on). On Windows, a forced checkout into a folder whose tree is not finished (the first checkout of a clone that was empty, and the completion of a checkout that was cut off) is preceded by refusals that change nothing: a link on the way or at a destination (except where the repository itself has a symbolic link), a file where a folder is needed (also on the way to a submodule), a folder that is not empty where a file is needed, a file at a destination that is not what the unfinished checkout wrote there, and anything but an empty folder where the repository has a submodule. "What the unfinished checkout wrote" is the content, or the beginning of it, of the commit the branch is on or of a commit recorded in `gclo.checkoutwrittenbefore`, as stored or as the checkout filters make it, carriage returns aside. An unfinished checkout is a marked folder that has no index, or whose index and files do not hold the whole tree of its commit; a marked folder that holds it is updated like any clone, without force. An unfinished checkout is completed at the commit its branch is on, before the branch moves, except where the branch must move first (a tree Windows cannot write, or a history origin rewrote); the commit it leaves is then recorded. A repository that has local branches is not treated as an empty clone; one without follows origin's default branch only when its folder holds nothing but `.git`. The folder is marked unfinished before its branch is created. One git operation per folder at a time within a process (not across processes, and not across spellings of a path that only the file system can resolve). The WSL clone is made in `~/gclo/.gclo-tmp` and renamed into place when complete (`mv -T`, after a check that nothing is there). Not covered: a file created in the folder between the check and the checkout that follows it; a file of the user's at a destination that is empty, or holds the repository's content or the beginning of it, which is replaced; a link of the user's where the repository has a symbolic link, which is replaced (what it leads to is not touched); an unfinished clone in which a folder of an earlier commit became a file, or that was cut off by a power loss leaving files of zeros, which is refused until the folder is cleared; platforms other than Windows, where the checkout is forced as before. |
+341 -1
View File
@@ -394,7 +394,7 @@ public sealed class InterruptedCloneTests : IDisposable
// The checkout that was cut off is finished at its own commit, which has no
// such file; what origin got since is an ordinary update, and that one does
// not replace a file that was added here.
Assert.Contains("The update would replace files in this folder that were changed or added here", refused.Message, StringComparison.Ordinal);
Assert.Contains("The update would replace files in this folder that are not as git recorded them", refused.Message, StringComparison.Ordinal);
Assert.Equal("mine", File.ReadAllText(Path.Combine(clone, "notes.txt")));
Assert.Equal("theirs", File.ReadAllText(Path.Combine(clone, "readme.txt")));
Assert.False(IsCheckoutPending(clone));
@@ -904,4 +904,344 @@ public sealed class InterruptedCloneTests : IDisposable
Assert.False(File.Exists(Path.Combine(clone, "readme.txt")));
Assert.Equal("mine", File.ReadAllText(Path.Combine(clone, "page.html")));
}
// ---------------------------------------------------------------- review of #175, third round
private const string LF = "\n";
private const string CRLF = "\r\n";
/// <summary>
/// Adds a commit to the origin itself (no working copy in between), on the branch
/// its HEAD names: the tree of the commit before it, changed by
/// <paramref name="change"/>; with <paramref name="asANewRoot"/> a commit without
/// a parent, which is what a rewritten history looks like from a clone.
/// </summary>
private static string CommitToOrigin(string origin, Func<Repository, TreeDefinition, TreeDefinition> change, bool asANewRoot = false)
{
using var repo = new Repository(origin);
Commit? before = repo.Head.Tip;
TreeDefinition definition = change(repo, before is null || asANewRoot ? new TreeDefinition() : TreeDefinition.From(before.Tree));
Signature who = MakeSignature();
Commit commit = repo.ObjectDatabase.CreateCommit(
who, who, "written into origin", repo.ObjectDatabase.CreateTree(definition),
before is null || asANewRoot ? [] : [before], prettifyMessage: false);
if(before is null)
{
repo.Refs.Add(repo.Refs.Head.TargetIdentifier, commit.Id);
}
else
{
repo.Refs.UpdateTarget(repo.Refs.Head.ResolveToDirectReference(), commit.Id);
}
return commit.Sha;
}
private static Blob BlobOf(Repository repo, string content)
=> repo.ObjectDatabase.CreateBlob(new MemoryStream(System.Text.Encoding.UTF8.GetBytes(content)));
private static string? WrittenBefore(string workdir)
{
using var repo = new Repository(workdir);
return repo.Config.Get<string>("gclo.checkoutwrittenbefore", ConfigurationLevel.Local)?.Value;
}
[Fact]
public async Task OriginGetsAPathWindowsCannotCreate_AndLosesItAgain_TheUnfinishedCloneIsStillCompleted()
{
if(!OperatingSystem.IsWindows())
{
return; // the paths are Windows' to refuse
}
(string origin, string clone) = await ACloneWhoseFirstCheckoutWasCutOffAsync(("a.txt", "one"), ("b.txt", "b"));
File.WriteAllText(Path.Combine(clone, "a.txt"), "one"); // what the cut-off checkout had written
File.WriteAllText(Path.Combine(clone, "b.txt"), "b");
string first = HeadSha(clone);
CommitToOrigin(origin, (repo, tree) => tree.Add("bad:name.txt", BlobOf(repo, "bad"), Mode.NonExecutableFile));
await Assert.ThrowsAsync<InvalidRepositoryPathsException>(() => _client.PullAsync(clone, Token, CancellationToken.None));
Assert.True(IsCheckoutPending(clone));
Assert.Equal(first, WrittenBefore(clone)); // the branch moved on; the commit whose files are on disk is remembered
string third = CommitToOrigin(
origin, (repo, tree) => tree.Remove("bad:name.txt").Add("a.txt", BlobOf(repo, "two"), Mode.NonExecutableFile));
await _client.PullAsync(clone, Token, CancellationToken.None);
Assert.Equal(third, HeadSha(clone));
Assert.Equal("two", File.ReadAllText(Path.Combine(clone, "a.txt")));
Assert.Equal("b", File.ReadAllText(Path.Combine(clone, "b.txt")));
Assert.False(IsCheckoutPending(clone));
Assert.Null(WrittenBefore(clone));
}
[Fact]
public async Task OriginRewritesItsHistory_BeforeAnUnfinishedCloneIsSyncedAgain_AndTheCloneFollowsIt()
{
(string origin, string clone) = await ACloneWhoseFirstCheckoutWasCutOffAsync(("a.txt", "one"), ("b.txt", "b"));
File.WriteAllText(Path.Combine(clone, "a.txt"), "one");
string rewritten = CommitToOrigin(
origin, (repo, tree) => tree.Add("a.txt", BlobOf(repo, "rewritten"), Mode.NonExecutableFile), asANewRoot: true);
await _client.PullAsync(clone, Token, CancellationToken.None); // nobody committed here: nothing to push or reset
Assert.Equal(rewritten, HeadSha(clone));
Assert.Equal("rewritten", File.ReadAllText(Path.Combine(clone, "a.txt")));
Assert.False(File.Exists(Path.Combine(clone, "b.txt")));
Assert.False(IsCheckoutPending(clone));
Assert.Null(WrittenBefore(clone));
}
/// <summary>
/// A clone into a folder that exists, whose checkout fails when it gets to
/// <c>zzz.txt</c> because another program has that file open: the files before it
/// are written, the marker is set, and there is no index.
/// </summary>
private async Task<string> ACloneWhoseCheckoutFailedHalfWayAsync(string origin, string name)
{
string target = Directory.CreateDirectory(NewPath(name)).FullName;
FileStream? held = null;
_client.AfterFetchForTesting = () =>
held = new FileStream(Path.Combine(target, "zzz.txt"), FileMode.CreateNew, FileAccess.ReadWrite, FileShare.None);
try
{
await Assert.ThrowsAnyAsync<Exception>(() => _client.CloneAsync(origin, target, Token, null, CancellationToken.None));
}
finally
{
held?.Dispose();
_client.AfterFetchForTesting = null;
}
Assert.True(IsCheckoutPending(target));
return target;
}
[Fact]
public async Task AFileTheCheckoutWroteAnObjectIdInto_IsStillItsOwn()
{
// The 'ident' attribute: what is written is not what is stored.
(string origin, _) = await CloneAnEmptyOriginAsync();
CommitToOrigin(origin, (repo, tree) => tree
.Add(".gitattributes", BlobOf(repo, "*.txt ident" + LF), Mode.NonExecutableFile)
.Add("a.txt", BlobOf(repo, "$Id$" + LF + "hello" + LF), Mode.NonExecutableFile)
.Add("zzz.txt", BlobOf(repo, "last" + LF), Mode.NonExecutableFile));
string target = await ACloneWhoseCheckoutFailedHalfWayAsync(origin, "ident");
Assert.Contains("$Id: ", File.ReadAllText(Path.Combine(target, "a.txt")), StringComparison.Ordinal);
await _client.FetchAndPullAsync(origin, target, Token, CancellationToken.None);
Assert.False(IsCheckoutPending(target));
Assert.StartsWith("last", File.ReadAllText(Path.Combine(target, "zzz.txt")), StringComparison.Ordinal);
}
[Theory]
[InlineData(false)] // 'git reset': the index names every file, the folder does not hold them all
[InlineData(true)] // 'git add -A': the index names what is in the folder
public async Task AnIndexSomebodysGitWrote_OverAnUnfinishedTree_DoesNotMakeTheTreeWhole(bool onlyWhatIsThere)
{
(string origin, _) = await CloneAnEmptyOriginAsync();
CommitToOrigin(origin, (repo, tree) => tree
.Add("a.txt", BlobOf(repo, "a"), Mode.NonExecutableFile)
.Add("b.txt", BlobOf(repo, "b"), Mode.NonExecutableFile)
.Add("zzz.txt", BlobOf(repo, "last"), Mode.NonExecutableFile));
string target = await ACloneWhoseCheckoutFailedHalfWayAsync(origin, "then-git");
File.Delete(Path.Combine(target, "zzz.txt"));
using(var repo = new Repository(target))
{
if(onlyWhatIsThere)
{
Commands.Stage(repo, "*");
}
else
{
repo.Index.Replace(repo.Head.Tip);
repo.Index.Write();
}
Assert.True(repo.Index.Count > 0);
}
await _client.FetchAndPullAsync(origin, target, Token, CancellationToken.None);
Assert.Equal("last", File.ReadAllText(Path.Combine(target, "zzz.txt"))); // completed, not released with a file missing
Assert.False(IsCheckoutPending(target));
}
[Theory]
[InlineData("one" + LF + "tw")] // written with other line endings, and cut off
[InlineData("one" + CRLF + "two" + CRLF)] // written as it is stored
public async Task AFileStoredWithCarriageReturns_IsRecognisedTheSameWay(string onDisk)
{
(string origin, string clone) = await CloneAnEmptyOriginAsync();
CommitToOrigin(origin, (repo, tree) => tree.Add("crlf.txt", BlobOf(repo, "one" + CRLF + "two" + CRLF), Mode.NonExecutableFile));
using var cts = new CancellationTokenSource();
_client.BeforeFirstCheckoutForTesting = cts.Cancel;
await Assert.ThrowsAnyAsync<OperationCanceledException>(() => _client.PullAsync(clone, Token, cts.Token));
_client.BeforeFirstCheckoutForTesting = null;
File.WriteAllText(Path.Combine(clone, "crlf.txt"), onDisk);
await _client.PullAsync(clone, Token, CancellationToken.None);
Assert.Equal(["one", "two"], File.ReadAllLines(Path.Combine(clone, "crlf.txt")));
Assert.False(IsCheckoutPending(clone));
}
[Fact]
public async Task ACompletionThatFailsHalfWay_LeavesTheFolderMarked()
{
(string origin, _) = await CloneAnEmptyOriginAsync();
CommitToOrigin(origin, (repo, tree) => tree
.Add("a.txt", BlobOf(repo, "a"), Mode.NonExecutableFile)
.Add("zzz.txt", BlobOf(repo, "last"), Mode.NonExecutableFile));
string target = await ACloneWhoseCheckoutFailedHalfWayAsync(origin, "twice");
// The next sync is the completion, and it fails at the same file: it can be
// read (so it is seen to be the checkout's own), not written.
using(var held = new FileStream(Path.Combine(target, "zzz.txt"), FileMode.Open, FileAccess.Read, FileShare.Read))
{
await Assert.ThrowsAnyAsync<Exception>(() => _client.FetchAndPullAsync(origin, target, Token, CancellationToken.None));
}
Assert.True(IsCheckoutPending(target)); // not taken off before the tree is written
await _client.FetchAndPullAsync(origin, target, Token, CancellationToken.None);
Assert.Equal("last", File.ReadAllText(Path.Combine(target, "zzz.txt")));
Assert.False(IsCheckoutPending(target));
}
[Fact]
public async Task ALinkWhereTheRepositoryHasASymbolicLink_IsTakenForTheCheckoutsOwn_AndWhatItLeadsToIsNotTouched()
{
(string origin, string clone) = await CloneAnEmptyOriginAsync();
CommitToOrigin(origin, (repo, tree) => tree
.Add("readme.txt", BlobOf(repo, "theirs"), Mode.NonExecutableFile)
.Add("alink", BlobOf(repo, "readme.txt"), Mode.SymbolicLink));
using(var repo = new Repository(clone))
{
repo.Config.Set("core.symlinks", false, ConfigurationLevel.Local); // the link is then written as a file that names its target
}
using var cts = new CancellationTokenSource();
_client.BeforeFirstCheckoutForTesting = cts.Cancel;
await Assert.ThrowsAnyAsync<OperationCanceledException>(() => _client.PullAsync(clone, Token, cts.Token));
_client.BeforeFirstCheckoutForTesting = null;
string elsewhere = Directory.CreateDirectory(NewPath("elsewhere")).FullName;
File.WriteAllText(Path.Combine(elsewhere, "kept.txt"), "mine");
string link = Path.Combine(clone, "alink");
WorkingTreeTests.CreateDirectoryLink(link, elsewhere);
try
{
await _client.PullAsync(clone, Token, CancellationToken.None);
Assert.Equal("readme.txt", File.ReadAllText(link));
Assert.Equal("mine", File.ReadAllText(Path.Combine(elsewhere, "kept.txt")));
Assert.False(IsCheckoutPending(clone));
}
finally
{
WorkingTreeTests.RemoveLink(link);
}
}
[Fact]
public async Task ALinkWhereTheRepositoryHasASubmodule_IsNotReplaced()
{
(string origin, string clone) = await CloneAnEmptyOriginAsync();
CommitToOrigin(origin, (repo, tree) => tree
.Add("readme.txt", BlobOf(repo, "theirs"), Mode.NonExecutableFile)
.AddGitLink("sub", new ObjectId("1234567890123456789012345678901234567890")));
string elsewhere = Directory.CreateDirectory(NewPath("elsewhere")).FullName; // empty: only the link itself is in the way
string link = Path.Combine(clone, "sub");
WorkingTreeTests.CreateDirectoryLink(link, elsewhere);
try
{
await Assert.ThrowsAnyAsync<Exception>(() => _client.PullAsync(clone, Token, CancellationToken.None));
Assert.NotNull(new DirectoryInfo(link).LinkTarget);
Assert.False(File.Exists(Path.Combine(clone, "readme.txt")));
}
finally
{
WorkingTreeTests.RemoveLink(link);
}
}
[Fact]
public async Task FollowingOriginsDefaultBranch_ChangesNothing_BeforeItsTreeIsKnownToBeWritable()
{
if(!OperatingSystem.IsWindows())
{
return; // the paths are Windows' to refuse
}
(string origin, string clone) = await CloneAnEmptyOriginAsync();
using(var bare = new Repository(origin))
{
bare.Refs.UpdateTarget("HEAD", "refs/heads/" + OtherBranch);
}
CommitToOrigin(origin, (repo, tree) => tree.Add("bad:name.txt", BlobOf(repo, "bad"), Mode.NonExecutableFile));
string[] before = LocalConfiguration(clone);
string headBefore;
using(var repo = new Repository(clone))
{
headBefore = repo.Refs.Head.TargetIdentifier;
}
await Assert.ThrowsAsync<InvalidRepositoryPathsException>(() => _client.PullAsync(clone, Token, CancellationToken.None));
Assert.Equal(before, LocalConfiguration(clone)); // no marker, no upstream for a branch that was not created
using var after = new Repository(clone);
Assert.Equal(headBefore, after.Refs.Head.TargetIdentifier); // and HEAD names what it named
}
[Fact]
public async Task FollowingOriginsDefaultBranch_TheUpstreamIsConfigured_BeforeTheBranchExists()
{
(string origin, string clone) = await CloneAnEmptyOriginAsync();
await SeedAsync(origin, OtherBranch, ("readme.txt", "from the other branch"));
string heads = Directory.CreateDirectory(Path.Combine(clone, ".git", "refs", "heads")).FullName;
string held = Path.Combine(heads, OtherBranch + ".lock"); // another git holds the lock: the branch cannot be created
File.WriteAllText(held, "");
await Assert.ThrowsAnyAsync<Exception>(() => _client.PullAsync(clone, Token, CancellationToken.None));
using(var stopped = new Repository(clone))
{
Assert.True(stopped.Info.IsHeadUnborn);
Assert.Equal("refs/heads/" + OtherBranch, stopped.Refs.Head.TargetIdentifier);
Assert.Equal("refs/heads/" + OtherBranch, stopped.Config.Get<string>($"branch.{OtherBranch}.merge", ConfigurationLevel.Local)?.Value);
}
Assert.True(IsCheckoutPending(clone));
File.Delete(held);
await _client.PullAsync(clone, Token, CancellationToken.None);
Assert.Equal("from the other branch", File.ReadAllText(Path.Combine(clone, "readme.txt")));
Assert.False(IsCheckoutPending(clone));
}
}
+191 -63
View File
@@ -48,6 +48,13 @@ public sealed class LibGit2GitClient : IGitClient
/// </summary>
private const string CheckoutPendingKey = "gclo.checkoutpending";
/// <summary>
/// Beside the marker: the commits an unfinished checkout here was writing before
/// the branch was moved on (ids, separated by commas). Their files may still be
/// on disk, and are told from a person's by what those commits hold (#174).
/// </summary>
private const string WrittenBeforeKey = "gclo.checkoutwrittenbefore";
/// <summary>
/// Local git config flag meaning "gclo writes this repository's working tree
/// itself, through a path recovery". It outlives the recovery file, so a
@@ -450,8 +457,9 @@ public sealed class LibGit2GitClient : IGitClient
catch(CheckoutConflictException ex) when(File.Exists(Path.Combine(repo.Info.Path, "index")))
{
throw new InvalidOperationException(
"The update would replace files in this folder that were changed or added here, so nothing was updated "
+ $"({ex.Message}). Commit, move or delete them ('git status' lists them), then sync again.", ex);
"The update would replace files in this folder that are not as git recorded them (changed or added here, or "
+ $"left by an update that was interrupted), so nothing was updated ({ex.Message}). 'git status' lists them: "
+ "commit, move or delete them, then sync again.", ex);
}
catch(CheckoutConflictException ex)
{
@@ -879,17 +887,20 @@ public sealed class LibGit2GitClient : IGitClient
/// <para>
/// With <paramref name="aCutOffWriteIsOurs"/> the checkout being completed has
/// written files already. One of them at a destination is recognised by holding
/// what the current commit has for that path, or the beginning of it (the file it
/// was writing when it was cut off), line endings aside: the checkout converts
/// them by rules that the tree itself brings, and those were not all on disk yet
/// when the first files were written.
/// what the current commit has for that path (or a commit of
/// <paramref name="writtenBefore"/>: one the branch was on while the checkout was
/// unfinished), or the beginning of it (the file it was writing when it was cut
/// off), line endings aside: the checkout converts them by rules that the tree
/// itself brings, and those were not all on disk yet when the first files were
/// written.
/// </para>
/// <para>
/// Windows only. The rules about names and links are Windows' rules; elsewhere the
/// checkout is done as it always was.
/// </para>
/// </remarks>
private static void RefuseToWriteOverWhatIsNotOurs(Repository repo, Tree incoming, bool aCutOffWriteIsOurs)
private static void RefuseToWriteOverWhatIsNotOurs(
Repository repo, Tree incoming, bool aCutOffWriteIsOurs, IReadOnlyList<Tree>? writtenBefore = null)
{
if(!OperatingSystem.IsWindows())
{
@@ -898,8 +909,19 @@ public sealed class LibGit2GitClient : IGitClient
string workdir = WorkingDirectoryOf(repo);
List<(string OriginalPath, string EffectivePath, Blob Blob)> entries = CollectEffectiveEntries(incoming, s_noRecovery);
///////////////////////////////////////////////////////////////////////////////
// Where the repository has a symbolic link, a link in the folder is what a //
// checkout writes there (when the user's configuration lets it): it is not //
// looked at further. Replacing a link never touches what it leads to. //
///////////////////////////////////////////////////////////////////////////////
entries.RemoveAll(entry =>
incoming[entry.OriginalPath]?.Mode == Mode.SymbolicLink
&& IsLink(Path.Combine(workdir, entry.EffectivePath.Replace('/', Path.DirectorySeparatorChar))));
var effectiveNow = new HashSet<string>(entries.Select(entry => entry.EffectivePath), StringComparer.Ordinal);
RefuseWhatCannotBeWritten(repo, workdir, entries, ours: null, effectiveNow, replaceOnlyWhatTheCommitHas: true, aCutOffWriteIsOurs);
RefuseWhatCannotBeWritten(
repo, workdir, entries, ours: null, effectiveNow, replaceOnlyWhatTheCommitHas: true, aCutOffWriteIsOurs, writtenBefore);
///////////////////////////////////////////////////////////////////////////////
// What that check lets through because the recovery writes file by file and //
@@ -949,18 +971,13 @@ public sealed class LibGit2GitClient : IGitClient
/// <summary>True when something is at <paramref name="path"/> and it is a link (a junction, a symbolic link).</summary>
/// <summary>
/// True when something is at <paramref name="path"/> and it is a link (a junction, a
/// symbolic link): recognised by having a target, as <see cref="WorkingTree"/> does,
/// not by the reparse-point attribute, which other things carry too.
/// </summary>
private static bool IsLink(string path)
{
try
{
return File.GetAttributes(path).HasFlag(FileAttributes.ReparsePoint);
}
catch(IOException)
{
return false; // nothing is there
}
}
=> new FileInfo(path).LinkTarget is not null || new DirectoryInfo(path).LinkTarget is not null;
@@ -1370,7 +1387,8 @@ public sealed class LibGit2GitClient : IGitClient
HashSet<string>? ours,
HashSet<string> effectiveNow,
bool replaceOnlyWhatTheCommitHas,
bool aCutOffWriteIsOurs = false)
bool aCutOffWriteIsOurs = false,
IReadOnlyList<Tree>? writtenBefore = null)
{
if(File.Exists(Path.Combine(repo.Info.Path, "index.lock")))
{
@@ -1392,7 +1410,10 @@ public sealed class LibGit2GitClient : IGitClient
WorkingTree.Verify(workdir, effectivePath, verifiedFolders);
string full = Path.Combine(workdir, effectivePath.Replace('/', Path.DirectorySeparatorChar));
if(replaceOnlyWhatTheCommitHas && File.Exists(full) && !IsWhatTheCommitHas(full, originalPath, current, aCutOffWriteIsOurs))
if(replaceOnlyWhatTheCommitHas
&& File.Exists(full)
&& !IsWhatTheCommitHas(full, originalPath, current, aCutOffWriteIsOurs)
&& !(writtenBefore ?? []).Any(earlier => IsWhatTheCommitHas(full, originalPath, earlier, aCutOffWriteIsOurs)))
{
throw new InvalidOperationException(
$"'{effectivePath}' is already in this folder, and it is not a file gclo has a record of writing "
@@ -1450,13 +1471,28 @@ public sealed class LibGit2GitClient : IGitClient
return false;
}
using var actual = new FileStream(full, FileMode.Open, FileAccess.Read, FileShare.ReadWrite | FileShare.Delete);
if(orItsBeginning)
{
using Stream stored = blob.GetContentStream();
return HoldsTheBeginningOf(stored, actual);
/////////////////////////////////////////////////////////////////////////////
// What is stored, or what the filters make of it on the way out (the //
// "ident" attribute writes the object's id into the file): either is //
// what a checkout writes, depending on which rules were on disk then. //
/////////////////////////////////////////////////////////////////////////////
using(Stream stored = blob.GetContentStream())
using(var written = new FileStream(full, FileMode.Open, FileAccess.Read, FileShare.ReadWrite | FileShare.Delete))
{
if(HoldsTheBeginningOf(stored, written))
{
return true;
}
}
using Stream filtered = blob.GetContentStream(new FilteringOptions(originalPath));
using var again = new FileStream(full, FileMode.Open, FileAccess.Read, FileShare.ReadWrite | FileShare.Delete);
return HoldsTheBeginningOf(filtered, again);
}
using var actual = new FileStream(full, FileMode.Open, FileAccess.Read, FileShare.ReadWrite | FileShare.Delete);
using Stream expected = blob.GetContentStream(new FilteringOptions(originalPath));
byte[] expectedBuffer = new byte[81920];
byte[] actualBuffer = new byte[81920];
@@ -1622,21 +1658,6 @@ public sealed class LibGit2GitClient : IGitClient
/// <summary>
/// True when the commit a pull would move this branch to has paths Windows cannot
/// create. False when there is no such commit to move to (histories that diverged
/// are the ordinary pull's to report).
/// </summary>
private static bool OriginHasPathsWindowsCannotCreate(Repository repo)
{
Commit? upstream = repo.Head.TrackedBranch?.Tip;
return upstream is not null
&& !string.Equals(upstream.Sha, repo.Head.Tip.Sha, StringComparison.Ordinal)
&& WindowsPathValidator.Validate(upstream.Tree).Count > 0;
}
/// <summary>
/// What a marked folder needs before the ordinary pull: its checkout finished, or
/// only its marker taken off.
@@ -1644,22 +1665,89 @@ public sealed class LibGit2GitClient : IGitClient
/// <remarks>
/// git writes the index when a checkout has reached its end. A marked folder
/// without an index is therefore a tree that was never finished: it is completed,
/// with force, at the commit its branch is on
/// (<see cref="CompletePendingCheckout"/>). A marked folder WITH an index is one
/// whose checkout did finish (gclo's own, whose marker could not be cleared any
/// more, or one a person made with git): its tree is git's to update, and the
/// ordinary pull does that without replacing anything that was changed. Either
/// way the pull that follows brings what origin has got since.
/// with force (<see cref="CompletePendingCheckout"/>). A marked folder whose index
/// and files hold the whole tree of its commit is one whose checkout did finish
/// (gclo's own, whose marker could not be cleared any more, or one a person
/// completed with git): its tree is git's to update, and the ordinary pull does
/// that without replacing anything that was changed. An index that somebody's git
/// wrote over an unfinished tree (a "git reset", a "git add") does not make the
/// tree whole, and that folder is completed like one without an index. Either way
/// the pull that follows brings what origin has got since.
/// </remarks>
private static void FinishOrReleaseAPendingCheckout(Repository repo, CancellationToken ct)
{
if(repo.Index.Count == 0)
if(repo.Index.Count == 0 || !HoldsItsWholeTree(repo))
{
CompletePendingCheckout(repo, ct);
}
else
{
repo.Config.Set(CheckoutPendingKey, false, ConfigurationLevel.Local);
MarkFinished(repo);
}
}
/// <summary>True when every file of the current commit is in the index and in the folder.</summary>
private static bool HoldsItsWholeTree(Repository repo)
{
string workdir = WorkingDirectoryOf(repo);
foreach((string originalPath, _, _) in CollectEffectiveEntries(repo.Head.Tip.Tree, s_noRecovery))
{
string full = Path.Combine(workdir, originalPath.Replace('/', Path.DirectorySeparatorChar));
if(repo.Index[originalPath] is null || !(File.Exists(full) || IsLink(full)))
{
return false;
}
}
return true;
}
/// <summary>The folder's tree is written: the marker and what goes with it are taken off.</summary>
private static void MarkFinished(Repository repo)
{
repo.Config.Set(CheckoutPendingKey, false, ConfigurationLevel.Local);
repo.Config.Unset(WrittenBeforeKey, ConfigurationLevel.Local);
}
/// <summary>The trees of the commits recorded under <see cref="WrittenBeforeKey"/> that are still in the repository.</summary>
private static List<Tree> WrittenBefore(Repository repo)
{
string ids = repo.Config.Get<string>(WrittenBeforeKey, ConfigurationLevel.Local)?.Value ?? "";
var trees = new List<Tree>();
foreach(string id in ids.Split(',', StringSplitOptions.RemoveEmptyEntries | StringSplitOptions.TrimEntries))
{
if(ObjectId.TryParse(id, out ObjectId parsed) && repo.Lookup<Commit>(parsed) is { } commit)
{
trees.Add(commit.Tree);
}
}
return trees;
}
/// <summary>
/// Records, before the branch of an unfinished checkout is moved, the commit it was
/// on. The latest sixteen are kept: a folder that was cut off more often than that
/// while origin kept moving has to be moved out of the way by hand.
/// </summary>
private static void RememberAsWrittenBefore(Repository repo, Commit commit)
{
const int Kept = 16;
List<string> ids =
[
.. (repo.Config.Get<string>(WrittenBeforeKey, ConfigurationLevel.Local)?.Value ?? "")
.Split(',', StringSplitOptions.RemoveEmptyEntries | StringSplitOptions.TrimEntries),
];
if(!ids.Contains(commit.Sha, StringComparer.Ordinal))
{
ids.Add(commit.Sha);
repo.Config.Set(WrittenBeforeKey, string.Join(',', ids.TakeLast(Kept)), ConfigurationLevel.Local);
}
}
@@ -1673,33 +1761,41 @@ public sealed class LibGit2GitClient : IGitClient
/// </summary>
/// <remarks>
/// <para>
/// The branch is not moved first: every attempt to finish writes the same commit,
/// so what an earlier attempt left on disk can be told from what somebody put
/// there since (<see cref="RefuseToWriteOverWhatIsNotOurs"/>, #174). What origin
/// has got meanwhile is brought by the ordinary pull that follows, over a tree
/// that then has an index.
/// Where origin's branch has only got ahead, the branch is not moved first: every
/// attempt to finish writes the same commit, so what an earlier attempt left on
/// disk can be told from what somebody put there since
/// (<see cref="RefuseToWriteOverWhatIsNotOurs"/>, #174). What origin has got
/// meanwhile is brought by the ordinary pull that follows, over a tree that then
/// has an index.
/// </para>
/// <para>
/// The exception is a tree Windows cannot write: the commit the branch is on, or
/// the one origin has now. Then the branch moves to origin's first, as it always
/// did, and nothing is written here: if origin renamed what was in the way the
/// checkout goes ahead at its commit, and if not, the failure names the paths of
/// the newest tree, which is the one the recovery that follows will write.
/// The branch has to move first in two cases, as it always did here: a tree
/// Windows cannot write (the commit the branch is on, or origin's; the failure
/// then names the paths of the newest tree, which is the one the recovery that
/// follows will write), and a history origin has rewritten (there is nothing to
/// fast-forward from). The commit the branch leaves is recorded then
/// (<see cref="WrittenBeforeKey"/>), because its files may be on disk and are
/// judged against it as well.
/// </para>
/// </remarks>
private static void CompletePendingCheckout(Repository repo, CancellationToken ct)
{
if(OperatingSystem.IsWindows() && (HasPathsWindowsCannotCreate(repo) || OriginHasPathsWindowsCannotCreate(repo)))
Commit head = repo.Head.Tip;
if(MustMoveBeforeItIsWritten(repo, head))
{
RememberAsWrittenBefore(repo, head);
AdvanceHeadToTrackedTip(repo);
IReadOnlyList<InvalidPathInfo> invalidPaths = WindowsPathValidator.Validate(repo.Head.Tip.Tree);
if(invalidPaths.Count > 0)
if(OperatingSystem.IsWindows())
{
throw new InvalidRepositoryPathsException(invalidPaths);
IReadOnlyList<InvalidPathInfo> invalidPaths = WindowsPathValidator.Validate(repo.Head.Tip.Tree);
if(invalidPaths.Count > 0)
{
throw new InvalidRepositoryPathsException(invalidPaths);
}
}
}
RefuseToWriteOverWhatIsNotOurs(repo, repo.Head.Tip.Tree, aCutOffWriteIsOurs: true);
RefuseToWriteOverWhatIsNotOurs(repo, repo.Head.Tip.Tree, aCutOffWriteIsOurs: true, WrittenBefore(repo));
ct.ThrowIfCancellationRequested();
try
@@ -1716,7 +1812,39 @@ public sealed class LibGit2GitClient : IGitClient
throw new OperationCanceledException(ct);
}
repo.Config.Set(CheckoutPendingKey, false, ConfigurationLevel.Local);
MarkFinished(repo);
}
/// <summary>
/// True when an unfinished checkout cannot be completed at the commit its branch
/// is on: Windows cannot write that commit's tree or the one origin has now, or
/// origin's branch is not a continuation of it (origin rewrote its history, and
/// by <see cref="TargetOf"/> nobody committed here). Histories that diverged
/// because of commits made here are not such a case: the commit here is completed,
/// and the ordinary pull says what is wrong.
/// </summary>
private static bool MustMoveBeforeItIsWritten(Repository repo, Commit head)
{
Commit target;
try
{
target = TargetOf(repo);
}
catch(InvalidOperationException)
{
target = head;
}
if(OperatingSystem.IsWindows()
&& (WindowsPathValidator.Validate(head.Tree).Count > 0 || WindowsPathValidator.Validate(target.Tree).Count > 0))
{
return true;
}
return !string.Equals(target.Sha, head.Sha, StringComparison.Ordinal)
&& !string.Equals(repo.ObjectDatabase.FindMergeBase(head, target)?.Sha, head.Sha, StringComparison.Ordinal);
}
+3 -3
View File
@@ -19,6 +19,9 @@ public sealed class WslCloner : IWslCloner
/// <summary>Environment variable the inline credential helper reads the token from.</summary>
public const string TokenVariable = "GCLO_GIT_TOKEN";
/// <summary>The folder under <c>~/gclo</c> in which clones are made before they are moved into place.</summary>
internal const string TemporaryFolderName = ".gclo-tmp";
/// <summary>
/// The clone script. $1 organization, $2 repository, $3 clone URL, $4 the URL's
/// host (with a non-default port). Prints the distribution name and the destination
@@ -40,9 +43,6 @@ public sealed class WslCloner : IWslCloner
/// another server by the repository's own configuration) gets the token for that
/// host only (#139).
/// </summary>
/// <summary>The folder under <c>~/gclo</c> in which clones are made before they are moved into place.</summary>
internal const string TemporaryFolderName = ".gclo-tmp";
internal const string CloneScript =
"set -e\n"
+ "org=\"$1\"; repo=\"$2\"; url=\"$3\"; host=\"$4\"\n"