KofTwentyTwo/gclo
2
0 Fork 0
Code Issues 28 Pull Requests 1 Actions Packages Projects Releases 22 Wiki Activity

fix(clone): fourth review of #175: finishing a checkout removes nothing that was staged and writes through no link

Browse Source 
The fourth review looked only at the changes of the third and found two ways,
both opened by those changes, in which finishing an unfinished checkout cost
the user something, and a few smaller things.

- A file somebody added to the index with git, and that the commit does not
  have, was removed by the forced checkout. The completion is now refused,
  names the file, and says how to go on ('git reset', or move the folder).
- Where the repository has a symbolic link, the folders on the way to it were
  no longer checked for links, so a link behind a linked folder could be
  replaced outside the clone. The way is checked again, and a link in the
  folder is the checkout's own only when it leads where the repository's does
  (also on a first checkout, where any link was taken before).
- Whether a marked folder holds its whole tree is asked of the files, not of
  the index: what the index says is whoever's git wrote it.
- A path recovery of an unfinished checkout whose branch had to move accepts
  the files of the commit it left (whole content), and takes the list of those
  commits off when it has written the tree.
- Tests for each, for a list that holds what is not a commit, for the list's
  cap and its entries being recorded once, for a file written before the
  attributes that change it, and for a link to a folder in a finished tree.
  The test of a rewritten history now writes the file it makes a statement
  about, and states what happens to it: it stays, untracked.
- README and threat model say what the code does, and name what is not
  covered: a marked folder whose index somebody's git wrote and whose last
  file was cut off is taken for finished; files of a commit the branch left
  stay in the folder.
- The tests that run the WSL clone script use the shell Git for Windows'
  launcher starts, not the launcher: it puts git's folders in front of the
  path, so the stand-in for git was never found when the tests were started
  from PowerShell (as CI starts them), and two of them failed there.

Refs #174

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Signed-off-by: James Maes <james@kof22.com>
This commit is contained in:
james.maesandClaude Fable 5.2 committed 2026-11-10 23:22:43 -06:00
1 parent 41f3712a8a
commit 5124ed3a59
5 files changed
+425 -41
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

README.md
+1 -1
View File
@@ -112,7 +112,7 @@ On launch, gclo shows a brief branded splash, then the **Quick Sync** connect ca
8. **Cancel** at any time: in-flight repos stop cleanly, unstarted repos are marked Canceled, and the summary still appears. After a run, the result bar offers **Retry failed** when something failed (it re-runs exactly the failed rows), otherwise **Open folder**. The folder button in the toolbar opens the target folder in Explorer whenever no sync is running.
**When something goes wrong:** failures are isolated per repository — one broken repo never stops the rest, and a failed row shows its error message inline. Failed clones are cleaned up, and a clone whose checkout never finished (a crash, a kill, a cleanup that could not delete the folder) is flagged inside its `.git`, so a partial working tree is never mistaken for an up-to-date repo on the next run — gclo completes the checkout instead. What it completes is its own. On Windows, a file that is already where the repository has one is replaced only when it is what the unfinished checkout wrote there (its content, or the beginning of it, line endings aside); for anything else the row fails and names the file: move it and sync again. The same check guards a repository that was empty when it was cloned and gets its first files later. Such a clone follows the origin's default branch when nothing has been put into its folder (otherwise it keeps the branch its HEAD names, and stays empty until origin has that branch); a clone that has branches of its own is left as it is when its HEAD names a branch without commits (a branch you started there yourself). If you have worked in an unfinished clone with git in the meantime, gclo finishes it only when every file there is still the repository's own. While a canceled clone or update is still ending, syncing that repository again fails with a message to try again when it has finished. Repositories with Windows-impossible paths are the deliberate exception: gclo checks every path in the incoming tree against Windows file-system rules *before* checkout — invalid characters, reserved device names like `CON`, trailing spaces or dots, names that differ only by case — and when it finds any, the repo is marked Failed with a per-path list of reasons, but the fully fetched `.git` is **kept** (nothing was ever checked out). Recovery happens in place, with no re-download: click the row's **Resolve…** link to rename each offending path (safe names are pre-suggested) or skip it, and gclo materializes the working tree right there; the CLI does the same automatically with [`--sanitize-paths`](docs/CLI.md#windows-invalid-paths-and---sanitize-paths). This applies to a repository gclo could not check out when it cloned it. A repository that was already checked out normally is never rewritten: when an update brings it such a path, it fails and nothing in it is changed, and **Resolve…** (or `--sanitize-paths`) then says how to get a fresh clone with the renames instead of touching it. When WSL is installed with git in its default distribution, the same dialog offers **Clone in WSL instead**: the repository is cloned unchanged into `~/gclo/<org>/<repo>` inside WSL (reachable from Explorer as `\wsl.localhost\<distro>\home\...`), the token travels only through the environment, and the Windows copy is left as it is. Long paths beyond the classic 260-character limit are handled automatically (`core.longpaths`).
**When something goes wrong:** failures are isolated per repository — one broken repo never stops the rest, and a failed row shows its error message inline. Failed clones are cleaned up, and a clone whose checkout never finished (a crash, a kill, a cleanup that could not delete the folder) is flagged inside its `.git`, so a partial working tree is never mistaken for an up-to-date repo on the next run — gclo completes the checkout instead. What it completes is its own. On Windows, a file that is already where the repository has one is replaced only when it is what the unfinished checkout wrote there (its content, or the beginning of it, line endings aside); for anything else the row fails and names the file: move it and sync again. The same check guards a repository that was empty when it was cloned and gets its first files later. Such a clone follows the origin's default branch when nothing has been put into its folder (otherwise it keeps the branch its HEAD names, and stays empty until origin has that branch); a clone that has branches of its own is left as it is when its HEAD names a branch without commits (a branch you started there yourself). If you have worked in an unfinished clone with git in the meantime, gclo finishes it only when every file there is still the repository's own and nothing you added with `git add` would be removed; otherwise the row fails and names the file. While a canceled clone or update is still ending, syncing that repository again fails with a message to try again when it has finished. Repositories with Windows-impossible paths are the deliberate exception: gclo checks every path in the incoming tree against Windows file-system rules *before* checkout — invalid characters, reserved device names like `CON`, trailing spaces or dots, names that differ only by case — and when it finds any, the repo is marked Failed with a per-path list of reasons, but the fully fetched `.git` is **kept** (nothing was ever checked out). Recovery happens in place, with no re-download: click the row's **Resolve…** link to rename each offending path (safe names are pre-suggested) or skip it, and gclo materializes the working tree right there; the CLI does the same automatically with [`--sanitize-paths`](docs/CLI.md#windows-invalid-paths-and---sanitize-paths). This applies to a repository gclo could not check out when it cloned it. A repository that was already checked out normally is never rewritten: when an update brings it such a path, it fails and nothing in it is changed, and **Resolve…** (or `--sanitize-paths`) then says how to get a fresh clone with the renames instead of touching it. When WSL is installed with git in its default distribution, the same dialog offers **Clone in WSL instead**: the repository is cloned unchanged into `~/gclo/<org>/<repo>` inside WSL (reachable from Explorer as `\wsl.localhost\<distro>\home\...`), the token travels only through the environment, and the Windows copy is left as it is. Long paths beyond the classic 260-character limit are handled automatically (`core.longpaths`).
**Settings** (File > Settings…) let you set a default GitHub token (stored in the Windows Credential Manager), the default target folder (with a folder picker), the default parallelism, and the theme (System, Light, or Dark); an Advanced section toggles the startup splash and its duration.
docs/security/threat-model.md
+1 -1
View File
@@ -121,4 +121,4 @@ flowchart LR
| 2026-11-10 | v1.1.1 | James Maes (@KofTwentyTwo) | T3/T4 updated: path recovery writes and deletes only through `WorkingTree` (inside the working directory, never in `.git`, never through a link); `.git` and its Windows aliases are invalid path segments; the stored recovery file's paths are validated on load; cleanup of a failed clone does not follow links (#141) |
| 2026-11-10 | v1.1.1 | James Maes (@KofTwentyTwo) | T15 added: text from repositories and servers is shown with control, separator and format characters made visible, in the activity log and in everything the CLI prints as text, including characters a legacy console code page would turn into control bytes; the desktop app's surfaces are named as not covered (#143) |
| 2026-11-10 | v1.1.1 | James Maes (@KofTwentyTwo) | The project's home moved to `git.kof22.com`: the Help menu, the About dialog, the CLI's error text and the documentation link there, and the Scoop bucket is added from there (the manifest that pins the download URL and its hash now comes from that server over HTTPS; T2 is unchanged in kind). Help > Share hands a fixed text (what gclo is, where to download it, the sender's gclo version) to the system's mail handler as a `mailto:` link with no recipient, or shows it and copies it to the clipboard; nothing about the user, their accounts, or their repositories is in it, and the address it is built from must be a plain http(s) host and path (#163, #165). The update feed and the release downloads named in package manifests are still GitHub Releases at this revision. |
| 2026-11-10 | v1.1.1 | James Maes (@KofTwentyTwo) | Working tree (#174; the T3 and T4 rows are unchanged, this adds to what they rely on). On Windows, a forced checkout into a folder whose tree is not finished (the first checkout of a clone that was empty, and the completion of a checkout that was cut off) is preceded by refusals that change nothing: a link on the way or at a destination (except where the repository itself has a symbolic link), a file where a folder is needed (also on the way to a submodule), a folder that is not empty where a file is needed, a file at a destination that is not what the unfinished checkout wrote there, and anything but an empty folder where the repository has a submodule. "What the unfinished checkout wrote" is the content, or the beginning of it, of the commit the branch is on or of a commit recorded in `gclo.checkoutwrittenbefore`, as stored or as the checkout filters make it, carriage returns aside. An unfinished checkout is a marked folder that has no index, or whose index and files do not hold the whole tree of its commit; a marked folder that holds it is updated like any clone, without force. An unfinished checkout is completed at the commit its branch is on, before the branch moves, except where the branch must move first (a tree Windows cannot write, or a history origin rewrote); the commit it leaves is then recorded. A repository that has local branches is not treated as an empty clone; one without follows origin's default branch only when its folder holds nothing but `.git`. The folder is marked unfinished before its branch is created. One git operation per folder at a time within a process (not across processes, and not across spellings of a path that only the file system can resolve). The WSL clone is made in `~/gclo/.gclo-tmp` and renamed into place when complete (`mv -T`, after a check that nothing is there). Not covered: a file created in the folder between the check and the checkout that follows it; a file of the user's at a destination that is empty, or holds the repository's content or the beginning of it, which is replaced; a link of the user's where the repository has a symbolic link, which is replaced (what it leads to is not touched); an unfinished clone in which a folder of an earlier commit became a file, or that was cut off by a power loss leaving files of zeros, which is refused until the folder is cleared; platforms other than Windows, where the checkout is forced as before. |
| 2026-11-10 | v1.1.1 | James Maes (@KofTwentyTwo) | Working tree (#174; the T3 and T4 rows are unchanged, this adds to what they rely on). On Windows, a forced checkout into a folder whose tree is not finished (the first checkout of a clone that was empty, and the completion of a checkout that was cut off) is preceded by refusals that change nothing: a link on the way or at a destination (except, at a destination where the repository itself has a symbolic link, a link that leads where the repository's does), a file where a folder is needed (also on the way to a submodule), a folder that is not empty where a file is needed, a file at a destination that is not what the unfinished checkout wrote there, and anything but an empty folder where the repository has a submodule. "What the unfinished checkout wrote" is the content, or the beginning of it, of the commit the branch is on or of a commit recorded in `gclo.checkoutwrittenbefore`, as stored or as the checkout filters make it, carriage returns aside. An unfinished checkout is a marked folder that has no index, or that does not hold every file of its commit; a marked folder that has an index and every file is updated like any clone, without force. A completion is refused when the index names a file the commit does not have (somebody added it with git, and the forced checkout would remove it). An unfinished checkout is completed at the commit its branch is on, before the branch moves, except where the branch must move first (a tree Windows cannot write, or a history origin rewrote); the commit it leaves is then recorded, and a path recovery of that folder accepts a file that is, whole, what a recorded commit has. A repository that has local branches is not treated as an empty clone; one without follows origin's default branch only when its folder holds nothing but `.git`. The folder is marked unfinished before its branch is created. One git operation per folder at a time within a process (not across processes, and not across spellings of a path that only the file system can resolve). The WSL clone is made in `~/gclo/.gclo-tmp` and renamed into place when complete (`mv -T`, after a check that nothing is there). Not covered: a file created in the folder between the check and the checkout that follows it; a file of the user's at a destination that is empty, or holds the repository's content or the beginning of it, which is replaced; a link of the user's that leads exactly where the repository's symbolic link at that path does, which is replaced by the repository's; a marked folder whose index somebody's git wrote and whose last file was cut off, which is taken for finished (git shows that file as changed); files of a commit the branch left that the new commit does not have, which stay in the folder untracked; an unfinished clone in which a folder of an earlier commit became a file, or that was cut off by a power loss leaving files of zeros, which is refused until the folder is cleared; platforms other than Windows, where the checkout is forced as before. |
gclo.Engine.Tests/InterruptedCloneTests.cs
+326 -14
View File
@@ -990,7 +990,8 @@ public sealed class InterruptedCloneTests : IDisposable
public async Task OriginRewritesItsHistory_BeforeAnUnfinishedCloneIsSyncedAgain_AndTheCloneFollowsIt()
{
(string origin, string clone) = await ACloneWhoseFirstCheckoutWasCutOffAsync(("a.txt", "one"), ("b.txt", "b"));
File.WriteAllText(Path.Combine(clone, "a.txt"), "one");
File.WriteAllText(Path.Combine(clone, "a.txt"), "one"); // what the cut-off checkout had written
File.WriteAllText(Path.Combine(clone, "b.txt"), "b");
string rewritten = CommitToOrigin(
origin, (repo, tree) => tree.Add("a.txt", BlobOf(repo, "rewritten"), Mode.NonExecutableFile), asANewRoot: true);
@@ -998,7 +999,7 @@ public sealed class InterruptedCloneTests : IDisposable
Assert.Equal(rewritten, HeadSha(clone));
Assert.Equal("rewritten", File.ReadAllText(Path.Combine(clone, "a.txt")));
Assert.False(File.Exists(Path.Combine(clone, "b.txt")));
Assert.Equal("b", File.ReadAllText(Path.Combine(clone, "b.txt"))); // not in the new history: it stays, as a file git does not track
Assert.False(IsCheckoutPending(clone));
Assert.Null(WrittenBefore(clone));
}
@@ -1130,32 +1131,51 @@ public sealed class InterruptedCloneTests : IDisposable
[Fact]
public async Task ALinkWhereTheRepositoryHasASymbolicLink_IsTakenForTheCheckoutsOwn_AndWhatItLeadsToIsNotTouched()
[Theory]
[InlineData(true, true)]
[InlineData(true, false)]
[InlineData(false, true)]
[InlineData(false, false)] // a link of the user's, in a folder nothing was ever checked out into
public async Task ALinkWhereTheRepositoryHasASymbolicLink_IsTheCheckoutsOwn_OnlyWhenItLeadsWhereTheRepositorysDoes(
bool afterACutOff, bool leadsThere)
{
(string origin, string clone) = await CloneAnEmptyOriginAsync();
string elsewhere = Directory.CreateDirectory(NewPath("elsewhere")).FullName;
File.WriteAllText(Path.Combine(elsewhere, "kept.txt"), "mine");
// A repository names the target of a link with '/'; a link on Windows has '\\'.
string named = leadsThere ? elsewhere.Replace('\\', '/') : "readme.txt";
CommitToOrigin(origin, (repo, tree) => tree
.Add("readme.txt", BlobOf(repo, "theirs"), Mode.NonExecutableFile)
.Add("alink", BlobOf(repo, "readme.txt"), Mode.SymbolicLink));
.Add("alink", BlobOf(repo, named), Mode.SymbolicLink));
using(var repo = new Repository(clone))
{
repo.Config.Set("core.symlinks", false, ConfigurationLevel.Local); // the link is then written as a file that names its target
}
using var cts = new CancellationTokenSource();
_client.BeforeFirstCheckoutForTesting = cts.Cancel;
await Assert.ThrowsAnyAsync<OperationCanceledException>(() => _client.PullAsync(clone, Token, cts.Token));
_client.BeforeFirstCheckoutForTesting = null;
string elsewhere = Directory.CreateDirectory(NewPath("elsewhere")).FullName;
File.WriteAllText(Path.Combine(elsewhere, "kept.txt"), "mine");
if(afterACutOff)
{
await CutOffBeforeTheFirstFileAsync(clone);
}
string link = Path.Combine(clone, "alink");
WorkingTreeTests.CreateDirectoryLink(link, elsewhere);
try
{
await _client.PullAsync(clone, Token, CancellationToken.None);
if(leadsThere)
{
await _client.PullAsync(clone, Token, CancellationToken.None);
Assert.Equal("readme.txt", File.ReadAllText(link));
Assert.Equal(named, File.ReadAllText(link));
Assert.False(IsCheckoutPending(clone));
}
else
{
InvalidOperationException refused = await Assert.ThrowsAsync<InvalidOperationException>(
() => _client.PullAsync(clone, Token, CancellationToken.None));
Assert.Contains("'alink' is a link", refused.Message, StringComparison.Ordinal);
Assert.NotNull(new DirectoryInfo(link).LinkTarget);
Assert.False(File.Exists(Path.Combine(clone, "readme.txt")));
}
Assert.Equal("mine", File.ReadAllText(Path.Combine(elsewhere, "kept.txt")));
Assert.False(IsCheckoutPending(clone));
}
finally
{
@@ -1244,4 +1264,296 @@ public sealed class InterruptedCloneTests : IDisposable
Assert.Equal("from the other branch", File.ReadAllText(Path.Combine(clone, "readme.txt")));
Assert.False(IsCheckoutPending(clone));
}
// ---------------------------------------------------------------- review of #175, fourth round
/// <summary>Cuts the first checkout of an empty clone off once its branch exists and before a file is written.</summary>
private async Task CutOffBeforeTheFirstFileAsync(string clone)
{
using var cts = new CancellationTokenSource();
_client.BeforeFirstCheckoutForTesting = cts.Cancel;
await Assert.ThrowsAnyAsync<OperationCanceledException>(() => _client.PullAsync(clone, Token, cts.Token));
_client.BeforeFirstCheckoutForTesting = null;
Assert.True(IsCheckoutPending(clone));
}
/// <summary>An origin with <c>a.txt</c>, <c>b.txt</c> and <c>zzz.txt</c> (the file a checkout that fails half-way stops at).</summary>
private async Task<string> AnOriginOfThreeFilesAsync()
{
(string origin, _) = await CloneAnEmptyOriginAsync();
CommitToOrigin(origin, (repo, tree) => tree
.Add("a.txt", BlobOf(repo, "a"), Mode.NonExecutableFile)
.Add("b.txt", BlobOf(repo, "b"), Mode.NonExecutableFile)
.Add("zzz.txt", BlobOf(repo, "last"), Mode.NonExecutableFile));
return origin;
}
[Theory]
[InlineData("mine.txt", false)] // 'git add mine.txt'
[InlineData("mywork/staged.txt", false)]
[InlineData("mine.txt", true)] // 'git add -A'
public async Task AFileSomebodyStaged_InAnUnfinishedTree_IsNotRemovedByTheCompletion(string path, bool everything)
{
string origin = await AnOriginOfThreeFilesAsync();
string target = await ACloneWhoseCheckoutFailedHalfWayAsync(origin, "staged");
File.Delete(Path.Combine(target, "zzz.txt")); // the tree is not whole
string mine = Path.Combine(target, path.Replace('/', Path.DirectorySeparatorChar));
Directory.CreateDirectory(Path.GetDirectoryName(mine)!);
File.WriteAllText(mine, "my notes");
using(var repo = new Repository(target))
{
Commands.Stage(repo, everything ? "*" : path);
}
// A forced checkout removes what the index names and the commit does not have.
InvalidOperationException refused = await Assert.ThrowsAsync<InvalidOperationException>(
() => _client.FetchAndPullAsync(origin, target, Token, CancellationToken.None));
Assert.Contains($"'{path}'", refused.Message, StringComparison.Ordinal);
Assert.Contains("git reset", refused.Message, StringComparison.Ordinal);
Assert.Equal("my notes", File.ReadAllText(mine));
Assert.True(IsCheckoutPending(target));
// Out of the index it is a file beside the checkout, and the checkout is finished.
using(var repo = new Repository(target))
{
repo.Index.Replace(repo.Head.Tip); // 'git reset'
repo.Index.Write();
}
await _client.FetchAndPullAsync(origin, target, Token, CancellationToken.None);
Assert.Equal("my notes", File.ReadAllText(mine));
Assert.Equal("last", File.ReadAllText(Path.Combine(target, "zzz.txt")));
Assert.False(IsCheckoutPending(target));
}
[Fact]
public async Task ARenameStagedWithGit_InAFinishedCheckoutWhoseMarkerStayed_IsNotUndone()
{
string origin = await AnOriginOfThreeFilesAsync();
string target = NewPath("renamed-here");
await _client.CloneAsync(origin, target, Token, null, CancellationToken.None);
string renamed = Path.Combine(target, "renamed.txt");
using(var repo = new Repository(target))
{
repo.Config.Set("gclo.checkoutpending", true, ConfigurationLevel.Local);
File.Move(Path.Combine(target, "b.txt"), renamed);
Commands.Stage(repo, "b.txt");
Commands.Stage(repo, "renamed.txt"); // 'git mv b.txt renamed.txt'
}
File.WriteAllText(renamed, "b, and an afternoon of work on it");
await Assert.ThrowsAsync<InvalidOperationException>(() => _client.FetchAndPullAsync(origin, target, Token, CancellationToken.None));
Assert.Equal("b, and an afternoon of work on it", File.ReadAllText(renamed));
Assert.False(File.Exists(Path.Combine(target, "b.txt")));
}
[Fact]
public async Task ALinkOnTheWayToASymbolicLinkOfTheRepository_IsRefused_AndNothingBehindItIsTouched()
{
(string origin, string clone) = await CloneAnEmptyOriginAsync();
string elsewhere = Directory.CreateDirectory(NewPath("elsewhere")).FullName;
string further = Directory.CreateDirectory(NewPath("further")).FullName;
CommitToOrigin(origin, (repo, tree) => tree
.Add("readme.txt", BlobOf(repo, "theirs"), Mode.NonExecutableFile)
.Add("dir/alink", BlobOf(repo, further), Mode.SymbolicLink));
await CutOffBeforeTheFirstFileAsync(clone);
// The user's own link, outside the clone, which leads where the repository's
// does; and the clone's "dir" leads to the folder that holds it.
string outside = Path.Combine(elsewhere, "alink");
WorkingTreeTests.CreateDirectoryLink(outside, further);
string onTheWay = Path.Combine(clone, "dir");
WorkingTreeTests.CreateDirectoryLink(onTheWay, elsewhere);
try
{
InvalidOperationException refused = await Assert.ThrowsAsync<InvalidOperationException>(
() => _client.PullAsync(clone, Token, CancellationToken.None));
Assert.Contains("is a link to another location", refused.Message, StringComparison.Ordinal);
Assert.NotNull(new DirectoryInfo(outside).LinkTarget);
Assert.NotNull(new DirectoryInfo(onTheWay).LinkTarget);
Assert.True(IsCheckoutPending(clone));
}
finally
{
WorkingTreeTests.RemoveLink(onTheWay);
WorkingTreeTests.RemoveLink(outside);
}
}
[Fact]
public async Task TheLinksACheckoutItselfWrote_AreItsOwn_WhateverThisMachineLetsItWrite()
{
// With the right to create symbolic links a checkout writes links (the target
// with this system's separators); without it, files that name their targets.
(string origin, _) = await CloneAnEmptyOriginAsync();
CommitToOrigin(origin, (repo, tree) => tree
.Add("alink", BlobOf(repo, "readme.txt"), Mode.SymbolicLink)
.Add("docs/up", BlobOf(repo, "../readme.txt"), Mode.SymbolicLink)
.Add("readme.txt", BlobOf(repo, "theirs"), Mode.NonExecutableFile)
.Add("zzz.txt", BlobOf(repo, "last"), Mode.NonExecutableFile));
string target = await ACloneWhoseCheckoutFailedHalfWayAsync(origin, "links");
await _client.FetchAndPullAsync(origin, target, Token, CancellationToken.None);
Assert.Equal("last", File.ReadAllText(Path.Combine(target, "zzz.txt")));
Assert.False(IsCheckoutPending(target));
}
[Fact]
public async Task AFinishedCheckoutWhoseMarkerStayed_IsWhole_AlsoWhereItHoldsALinkToAFolder()
{
(string origin, _) = await CloneAnEmptyOriginAsync();
CommitToOrigin(origin, (repo, tree) => tree
.Add("alink", BlobOf(repo, "docs"), Mode.SymbolicLink)
.Add("docs/guide.txt", BlobOf(repo, "guide"), Mode.NonExecutableFile)
.Add("other.txt", BlobOf(repo, "other"), Mode.NonExecutableFile));
string target = NewPath("finished-with-a-link");
await _client.CloneAsync(origin, target, Token, null, CancellationToken.None);
string link = Path.Combine(target, "alink");
if(File.GetAttributes(link).HasFlag(FileAttributes.Directory))
{
Directory.Delete(link);
}
else
{
File.Delete(link);
}
WorkingTreeTests.CreateDirectoryLink(link, Path.Combine(target, "docs")); // a link to a folder: no file is there
using(var repo = new Repository(target))
{
repo.Config.Set("gclo.checkoutpending", true, ConfigurationLevel.Local);
}
File.WriteAllText(Path.Combine(target, "other.txt"), "edited here"); // a completion would refuse this, or undo it
try
{
await _client.FetchAndPullAsync(origin, target, Token, CancellationToken.None);
Assert.Equal("edited here", File.ReadAllText(Path.Combine(target, "other.txt")));
Assert.False(IsCheckoutPending(target));
}
finally
{
WorkingTreeTests.RemoveLink(link);
}
}
[Fact]
public async Task AFileWrittenBeforeTheRulesThatChangeIt_IsStillTheCheckoutsOwn()
{
// "-early.txt" is written before ".gitattributes": as it is stored, not as the
// 'ident' rule in that file makes it.
(string origin, string clone) = await CloneAnEmptyOriginAsync();
CommitToOrigin(origin, (repo, tree) => tree
.Add("-early.txt", BlobOf(repo, "$Id$" + LF + "hello" + LF), Mode.NonExecutableFile)
.Add(".gitattributes", BlobOf(repo, "*.txt ident" + LF), Mode.NonExecutableFile));
await CutOffBeforeTheFirstFileAsync(clone);
File.WriteAllText(Path.Combine(clone, "-early.txt"), "$Id$" + LF + "hello" + LF);
File.WriteAllText(Path.Combine(clone, ".gitattributes"), "*.txt ident" + LF);
await _client.PullAsync(clone, Token, CancellationToken.None);
Assert.False(IsCheckoutPending(clone));
}
[Fact]
public async Task ARecoveryAfterTheBranchMoved_KnowsTheFilesOfTheCommitItLeft_AndTakesTheListOff()
{
if(!OperatingSystem.IsWindows())
{
return; // the paths are Windows' to refuse
}
(string origin, string clone) = await ACloneWhoseFirstCheckoutWasCutOffAsync(("a.txt", "one"));
File.WriteAllText(Path.Combine(clone, "a.txt"), "one"); // what the cut-off checkout had written
CommitToOrigin(origin, (repo, tree) => tree
.Add("a.txt", BlobOf(repo, "two"), Mode.NonExecutableFile)
.Add("bad:name.txt", BlobOf(repo, "bad"), Mode.NonExecutableFile));
await Assert.ThrowsAsync<InvalidRepositoryPathsException>(() => _client.PullAsync(clone, Token, CancellationToken.None));
Assert.NotNull(WrittenBefore(clone));
await _client.ApplyRecoveryAsync(
clone,
new PathRecovery(
new Dictionary<string, string>(StringComparer.Ordinal) { ["bad:name.txt"] = "bad_name.txt" },
new HashSet<string>(StringComparer.Ordinal)),
CancellationToken.None);
Assert.Equal("two", File.ReadAllText(Path.Combine(clone, "a.txt")));
Assert.Equal("bad", File.ReadAllText(Path.Combine(clone, "bad_name.txt")));
Assert.False(IsCheckoutPending(clone));
Assert.Null(WrittenBefore(clone));
}
[Fact]
public async Task AListOfEarlierCommits_ThatHoldsWhatIsNotACommitHere_IsReadWithoutFailing()
{
(_, string clone) = await ACloneWhoseFirstCheckoutWasCutOffAsync(("a.txt", "one"));
File.WriteAllText(Path.Combine(clone, "a.txt"), "on"); // the file the checkout was writing
using(var repo = new Repository(clone))
{
string blob = repo.Head.Tip["a.txt"].Target.Sha;
string tree = repo.Head.Tip.Tree.Sha;
repo.Config.Set(
"gclo.checkoutwrittenbefore",
$"garbage,0123456789012345678901234567890123456789,{blob},{tree}, ,abc,{new string('f', 40)},",
ConfigurationLevel.Local);
}
await _client.PullAsync(clone, Token, CancellationToken.None);
Assert.Equal("one", File.ReadAllText(Path.Combine(clone, "a.txt")));
Assert.False(IsCheckoutPending(clone));
Assert.Null(WrittenBefore(clone));
}
[Fact]
public async Task TheCommitsAnUnfinishedCheckoutLeft_AreRecordedOnce_AndTheLatestSixteenAreKept()
{
if(!OperatingSystem.IsWindows())
{
return; // the paths are Windows' to refuse
}
(string origin, string clone) = await ACloneWhoseFirstCheckoutWasCutOffAsync(("a.txt", "one"));
var left = new List<string> { HeadSha(clone) };
for(int round = 0; round < 17; round++)
{
string content = new('x', round + 1);
string added = CommitToOrigin(origin, (repo, tree) => tree.Add("bad:name.txt", BlobOf(repo, content), Mode.NonExecutableFile));
// The first sync moves the branch off the commit before; the second, with
// nothing new at origin, finds the branch on a commit Windows cannot write
// and records that one; a third would record nothing new.
await Assert.ThrowsAsync<InvalidRepositoryPathsException>(() => _client.PullAsync(clone, Token, CancellationToken.None));
await Assert.ThrowsAsync<InvalidRepositoryPathsException>(() => _client.PullAsync(clone, Token, CancellationToken.None));
await Assert.ThrowsAsync<InvalidRepositoryPathsException>(() => _client.PullAsync(clone, Token, CancellationToken.None));
left.Add(added);
}
Assert.Equal(18, left.Count);
Assert.Equal(string.Join(',', left.TakeLast(16)), WrittenBefore(clone));
}
}
gclo.Engine.Tests/WslCloneScriptRunTests.cs
+25 -6
View File
@@ -64,6 +64,14 @@ public sealed class WslCloneScriptRunTests : IDisposable
/// on the path and, because Git for Windows puts only its <c>cmd</c> folder there,
/// beside every <c>git.exe</c> on the path and where Git for Windows installs.
/// </summary>
/// <remarks>
/// Git for Windows' <c>bin\sh.exe</c> is a launcher: it puts git's own folders in
/// front of the path before it starts the shell, and a stand-in for git that a
/// test puts on the path is then never found. The shell it starts
/// (<c>usr\bin\sh.exe</c>) is used instead, with its own folder on the path for
/// the tools a script needs (<see cref="PathWith"/>). Which of the two a test run
/// finds first depends on where it was started from.
/// </remarks>
private static (string Sh, string Git)? FindTools()
{
string shell = OperatingSystem.IsWindows() ? "sh.exe" : "sh";
@@ -79,11 +87,14 @@ public sealed class WslCloneScriptRunTests : IDisposable
candidates.Add(Path.Combine(Environment.GetFolderPath(programs), "Git", "bin", shell));
}
foreach(string sh in candidates.Where(File.Exists))
foreach(string found in candidates.Where(File.Exists))
{
string started = Path.GetFullPath(Path.Combine(Path.GetDirectoryName(found)!, "..", "usr", "bin", shell));
string sh = OperatingSystem.IsWindows() && File.Exists(started) ? started : found;
try
{
var ask = new ProcessStartInfo(sh) { RedirectStandardOutput = true, RedirectStandardError = true, UseShellExecute = false };
ask.Environment["PATH"] = PathWith(sh, null);
ask.ArgumentList.Add("-c");
ask.ArgumentList.Add("command -v git");
using Process process = Process.Start(ask)!;
@@ -104,6 +115,17 @@ public sealed class WslCloneScriptRunTests : IDisposable
/// <summary>
/// The path a shell is run with: <paramref name="first"/> (a test's stand-ins,
/// when it has any), the shell's own folder, then the path of this process.
/// </summary>
private static string PathWith(string sh, string? first)
=> string.Join(
Path.PathSeparator,
new[] { first, Path.GetDirectoryName(sh), Environment.GetEnvironmentVariable("PATH") }.Where(part => !string.IsNullOrEmpty(part)));
/// <summary>False, with a line in the test's output, when this machine has no shell to run the script with.</summary>
private bool CanRun()
{
@@ -147,10 +169,7 @@ public sealed class WslCloneScriptRunTests : IDisposable
start.Environment["WSL_DISTRO_NAME"] = "test";
start.Environment["GCLO_TEST_REAL_GIT"] = s_tools.Value.Git;
start.Environment["GCLO_TEST_APPEAR"] = (appear ?? "").Replace('\\', '/');
if(pathPrefix is not null)
{
start.Environment["PATH"] = pathPrefix + Path.PathSeparator + Environment.GetEnvironmentVariable("PATH");
}
start.Environment["PATH"] = PathWith(s_tools.Value.Sh, pathPrefix);
using Process process = Process.Start(start)!;
process.StandardInput.Write(WslCloner.CloneScript);
@@ -311,7 +330,7 @@ public sealed class WslCloneScriptRunTests : IDisposable
start.Environment["GCLO_TEST_REAL_GIT"] = s_tools.Value.Git;
start.Environment["GCLO_TEST_SCRIPT"] = script.Replace('\\', '/');
start.Environment["GCLO_TEST_URL"] = _origin.Replace('\\', '/');
start.Environment["PATH"] = wrappers + Path.PathSeparator + Environment.GetEnvironmentVariable("PATH");
start.Environment["PATH"] = PathWith(s_tools.Value.Sh, wrappers);
using Process process = Process.Start(start)!;
Task<string> errorText = process.StandardError.ReadToEndAsync();
gclo.Engine/LibGit2GitClient.cs
+72 -19
View File
@@ -878,11 +878,12 @@ public sealed class LibGit2GitClient : IGitClient
/// <para>
/// The rules are the ones the path recovery follows
/// (<see cref="RefuseWhatCannotBeWritten"/>), and a few it has no need for: a link
/// on the way or at a destination, a file where a folder is needed (also on the way
/// to a submodule), a folder with anything in it where a file is needed, a file at
/// a destination that is not what the current commit has there, and anything but an
/// empty folder where the repository has a submodule (the checkout makes an empty
/// folder of it).
/// on the way or at a destination (at a destination where the repository has a
/// symbolic link, only a link that leads elsewhere), a file where a folder is
/// needed (also on the way to a submodule), a folder with anything in it where a
/// file is needed, a file at a destination that is not what the current commit has
/// there, and anything but an empty folder where the repository has a submodule
/// (the checkout makes an empty folder of it).
/// </para>
/// <para>
/// With <paramref name="aCutOffWriteIsOurs"/> the checkout being completed has
@@ -911,13 +912,13 @@ public sealed class LibGit2GitClient : IGitClient
List<(string OriginalPath, string EffectivePath, Blob Blob)> entries = CollectEffectiveEntries(incoming, s_noRecovery);
///////////////////////////////////////////////////////////////////////////////
// Where the repository has a symbolic link, a link in the folder is what a //
// checkout writes there (when the user's configuration lets it): it is not //
// looked at further. Replacing a link never touches what it leads to. //
// Where the repository has a symbolic link, a link in the folder that leads //
// where the repository's does is what a checkout writes there (when the //
// user's configuration lets it), or the same thing made by hand: it is not //
// looked at further. The folders on the way to it are, as for every entry. //
///////////////////////////////////////////////////////////////////////////////
entries.RemoveAll(entry =>
incoming[entry.OriginalPath]?.Mode == Mode.SymbolicLink
&& IsLink(Path.Combine(workdir, entry.EffectivePath.Replace('/', Path.DirectorySeparatorChar))));
incoming[entry.OriginalPath]?.Mode == Mode.SymbolicLink && IsTheLinkTheRepositoryHas(workdir, entry.EffectivePath, entry.Blob));
var effectiveNow = new HashSet<string>(entries.Select(entry => entry.EffectivePath), StringComparer.Ordinal);
RefuseWhatCannotBeWritten(
@@ -981,6 +982,24 @@ public sealed class LibGit2GitClient : IGitClient
/// <summary>
/// True when a link is at <paramref name="effectivePath"/> and it leads where the
/// repository's symbolic link there does (<paramref name="link"/>, whose content
/// is the target). A link on the way to it is refused, as on the way to anything
/// that is written.
/// </summary>
private static bool IsTheLinkTheRepositoryHas(string workdir, string effectivePath, Blob link)
{
WorkingTree.Verify(workdir, effectivePath);
string full = Path.Combine(workdir, effectivePath.Replace('/', Path.DirectorySeparatorChar));
string? target = new FileInfo(full).LinkTarget ?? new DirectoryInfo(full).LinkTarget;
return target is not null
&& string.Equals(target.Replace('/', '\\'), link.GetContentText().Replace('/', '\\'), StringComparison.Ordinal);
}
/// <summary>The paths of the submodule entries (gitlinks) of a tree, at any depth.</summary>
private static List<string> SubmodulePaths(Tree tree)
{
@@ -1104,7 +1123,8 @@ public sealed class LibGit2GitClient : IGitClient
HashSet<string>? ours = stored?.MaterializedPaths is null ? null : new HashSet<string>(stored.MaterializedPaths, StringComparer.Ordinal);
var effectiveNow = new HashSet<string>(entries.Select(e => e.EffectivePath), StringComparer.Ordinal);
RefuseWhatCannotBeWritten(repo, workdir, entries, ours, effectiveNow, replaceOnlyWhatTheCommitHas: !managedByRecord);
RefuseWhatCannotBeWritten(
repo, workdir, entries, ours, effectiveNow, replaceOnlyWhatTheCommitHas: !managedByRecord, writtenBefore: WrittenBefore(repo));
/////////////////////////////////////////////////////////////////////////////
// The point of no return. Record first that this tree is being rewritten, //
@@ -1147,7 +1167,7 @@ public sealed class LibGit2GitClient : IGitClient
}
SaveRecovery(repo, recovery, effectiveNow, tip.Sha);
repo.Config.Set(CheckoutPendingKey, false, ConfigurationLevel.Local);
MarkFinished(repo);
}
@@ -1665,14 +1685,14 @@ public sealed class LibGit2GitClient : IGitClient
/// <remarks>
/// git writes the index when a checkout has reached its end. A marked folder
/// without an index is therefore a tree that was never finished: it is completed,
/// with force (<see cref="CompletePendingCheckout"/>). A marked folder whose index
/// and files hold the whole tree of its commit is one whose checkout did finish
/// with force (<see cref="CompletePendingCheckout"/>). A marked folder that has an
/// index and holds every file of its commit is one whose checkout did finish
/// (gclo's own, whose marker could not be cleared any more, or one a person
/// completed with git): its tree is git's to update, and the ordinary pull does
/// that without replacing anything that was changed. An index that somebody's git
/// wrote over an unfinished tree (a "git reset", a "git add") does not make the
/// tree whole, and that folder is completed like one without an index. Either way
/// the pull that follows brings what origin has got since.
/// wrote over an unfinished tree (a "git reset", a "git add") does not put the
/// missing files there, and that folder is completed like one without an index.
/// Either way the pull that follows brings what origin has got since.
/// </remarks>
private static void FinishOrReleaseAPendingCheckout(Repository repo, CancellationToken ct)
{
@@ -1688,14 +1708,18 @@ public sealed class LibGit2GitClient : IGitClient
/// <summary>True when every file of the current commit is in the index and in the folder.</summary>
/// <summary>
/// True when every file of the current commit is in the folder (as a file, or as a
/// link: the repository may have a symbolic link there). What the index says is
/// not asked: it is whoever's git wrote it.
/// </summary>
private static bool HoldsItsWholeTree(Repository repo)
{
string workdir = WorkingDirectoryOf(repo);
foreach((string originalPath, _, _) in CollectEffectiveEntries(repo.Head.Tip.Tree, s_noRecovery))
{
string full = Path.Combine(workdir, originalPath.Replace('/', Path.DirectorySeparatorChar));
if(repo.Index[originalPath] is null || !(File.Exists(full) || IsLink(full)))
if(!(File.Exists(full) || IsLink(full)))
{
return false;
}
@@ -1777,6 +1801,13 @@ public sealed class LibGit2GitClient : IGitClient
/// (<see cref="WrittenBeforeKey"/>), because its files may be on disk and are
/// judged against it as well.
/// </para>
/// <para>
/// A file somebody added to the index with git, and that the commit does not have,
/// would be removed from the folder by the forced checkout: the completion is
/// refused instead (<see cref="RefuseToRemoveWhatIsStaged"/>). A file of a commit
/// the branch left that the new commit does not have is not looked for: it stays
/// in the folder, as a file git does not track.
/// </para>
/// </remarks>
private static void CompletePendingCheckout(Repository repo, CancellationToken ct)
{
@@ -1795,6 +1826,7 @@ public sealed class LibGit2GitClient : IGitClient
}
}
RefuseToRemoveWhatIsStaged(repo);
RefuseToWriteOverWhatIsNotOurs(repo, repo.Head.Tip.Tree, aCutOffWriteIsOurs: true, WrittenBefore(repo));
ct.ThrowIfCancellationRequested();
@@ -1817,6 +1849,27 @@ public sealed class LibGit2GitClient : IGitClient
/// <summary>
/// Before a forced checkout into a folder that has an index: refuses, with nothing
/// changed, when the index names a file the commit does not have. A forced checkout
/// removes such a file from the folder, and it is there because somebody added it
/// with git (#174).
/// </summary>
private static void RefuseToRemoveWhatIsStaged(Repository repo)
{
Tree tree = repo.Head.Tip.Tree;
IndexEntry? staged = repo.Index.FirstOrDefault(entry => tree[entry.Path] is null);
if(staged is not null)
{
throw new InvalidOperationException(
$"'{staged.Path}' has been added to the index of this folder with git, and the commit whose checkout gclo has to finish "
+ "here does not have it: finishing would remove the file. Take it out of the index ('git reset'), or move the folder "
+ "away, then sync again.");
}
}
/// <summary>
/// True when an unfinished checkout cannot be completed at the commit its branch
/// is on: Windows cannot write that commit's tree or the one origin has now, or