fix(clone): clones that were empty, cut off, or still ending are handled safely by the next sync (#175)
- A forced checkout into a folder whose tree is not finished (the first checkout of a clone that was empty, the completion of one that was cut off) refuses, before anything changes, when something that is not gclo's is where the repository has a file, needs a folder, or has a submodule; when a link is on the way; and when the index names a file the commit does not have. - The folder is marked unfinished before its branch is created. An unfinished checkout is a marked folder without an index, or one that does not hold every file of its commit: it is completed at the commit its branch is on, and what an earlier attempt wrote is recognised by its content (or the beginning of it, line endings aside). A marked folder that has an index and every file is updated like any clone, without force. - Where the branch of an unfinished checkout must move first (a tree Windows cannot write, a history origin rewrote), the commit it leaves is recorded and its files are recognised as well, also by the path recovery. - An empty clone follows origin's default branch only when its folder holds nothing but .git and it has no branches of its own; a branch somebody started there is left alone. - One git operation per folder at a time in a process. A canceled sync stops waiting for its operations without stopping them; a second operation in the same folder is refused until the first has ended. - A clone without .git/index says so and how to rebuild it, instead of git's count of conflicts. - Clone in WSL clones under ~/gclo/.gclo-tmp and renames the folder into place when it is complete; leftovers of killed runs are removed once a day old. - README, docs/CLI.md and the threat model describe the behaviour and name what is not covered. Four independent reviews; every finding fixed or named as a limit in the threat model. Closes #174 Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com> Signed-off-by: James Maes <james@kof22.com>
This commit was merged in pull request #175.
This commit is contained in:
james.maesandClaude Fable 5.2 committed
1 parent
48b3691828
commit
388b57ea94
9 files changed
+2678
-65
No files matched your search
@@ -112,7 +112,7 @@ On launch, gclo shows a brief branded splash, then the **Quick Sync** connect ca
|
||||
|
||||
8. **Cancel** at any time: in-flight repos stop cleanly, unstarted repos are marked Canceled, and the summary still appears. After a run, the result bar offers **Retry failed** when something failed (it re-runs exactly the failed rows), otherwise **Open folder**. The folder button in the toolbar opens the target folder in Explorer whenever no sync is running.
|
||||
|
||||
**When something goes wrong:** failures are isolated per repository — one broken repo never stops the rest, and a failed row shows its error message inline. Failed clones are cleaned up, and a clone whose checkout never finished (a crash, a kill, a cleanup that could not delete the folder) is flagged inside its `.git`, so a partial working tree is never mistaken for an up-to-date repo on the next run — gclo completes the checkout instead. Repositories with Windows-impossible paths are the deliberate exception: gclo checks every path in the incoming tree against Windows file-system rules *before* checkout — invalid characters, reserved device names like `CON`, trailing spaces or dots, names that differ only by case — and when it finds any, the repo is marked Failed with a per-path list of reasons, but the fully fetched `.git` is **kept** (nothing was ever checked out). Recovery happens in place, with no re-download: click the row's **Resolve…** link to rename each offending path (safe names are pre-suggested) or skip it, and gclo materializes the working tree right there; the CLI does the same automatically with [`--sanitize-paths`](docs/CLI.md#windows-invalid-paths-and---sanitize-paths). This applies to a repository gclo could not check out when it cloned it. A repository that was already checked out normally is never rewritten: when an update brings it such a path, it fails and nothing in it is changed, and **Resolve…** (or `--sanitize-paths`) then says how to get a fresh clone with the renames instead of touching it. When WSL is installed with git in its default distribution, the same dialog offers **Clone in WSL instead**: the repository is cloned unchanged into `~/gclo/<org>/<repo>` inside WSL (reachable from Explorer as `\wsl.localhost\<distro>\home\...`), the token travels only through the environment, and the Windows copy is left as it is. Long paths beyond the classic 260-character limit are handled automatically (`core.longpaths`).
|
||||
**When something goes wrong:** failures are isolated per repository — one broken repo never stops the rest, and a failed row shows its error message inline. Failed clones are cleaned up, and a clone whose checkout never finished (a crash, a kill, a cleanup that could not delete the folder) is flagged inside its `.git`, so a partial working tree is never mistaken for an up-to-date repo on the next run — gclo completes the checkout instead. What it completes is its own. On Windows, a file that is already where the repository has one is replaced only when it is what the unfinished checkout wrote there (its content, or the beginning of it, line endings aside); for anything else the row fails and names the file: move it and sync again. The same check guards a repository that was empty when it was cloned and gets its first files later. Such a clone follows the origin's default branch when nothing has been put into its folder (otherwise it keeps the branch its HEAD names, and stays empty until origin has that branch); a clone that has branches of its own is left as it is when its HEAD names a branch without commits (a branch you started there yourself). If you have worked in an unfinished clone with git in the meantime, gclo finishes it only when every file there is still the repository's own and nothing you added with `git add` would be removed; otherwise the row fails and names the file. While a canceled clone or update is still ending, syncing that repository again fails with a message to try again when it has finished. Repositories with Windows-impossible paths are the deliberate exception: gclo checks every path in the incoming tree against Windows file-system rules *before* checkout — invalid characters, reserved device names like `CON`, trailing spaces or dots, names that differ only by case — and when it finds any, the repo is marked Failed with a per-path list of reasons, but the fully fetched `.git` is **kept** (nothing was ever checked out). Recovery happens in place, with no re-download: click the row's **Resolve…** link to rename each offending path (safe names are pre-suggested) or skip it, and gclo materializes the working tree right there; the CLI does the same automatically with [`--sanitize-paths`](docs/CLI.md#windows-invalid-paths-and---sanitize-paths). This applies to a repository gclo could not check out when it cloned it. A repository that was already checked out normally is never rewritten: when an update brings it such a path, it fails and nothing in it is changed, and **Resolve…** (or `--sanitize-paths`) then says how to get a fresh clone with the renames instead of touching it. When WSL is installed with git in its default distribution, the same dialog offers **Clone in WSL instead**: the repository is cloned unchanged into `~/gclo/<org>/<repo>` inside WSL (reachable from Explorer as `\wsl.localhost\<distro>\home\...`), the token travels only through the environment, and the Windows copy is left as it is. Long paths beyond the classic 260-character limit are handled automatically (`core.longpaths`).
|
||||
|
||||
**Settings** (File > Settings…) let you set a default GitHub token (stored in the Windows Credential Manager), the default target folder (with a folder picker), the default parallelism, and the theme (System, Light, or Dark); an Advanced section toggles the startup splash and its duration.
|
||||
|
||||
|
||||
@@ -49,6 +49,10 @@ Options may be written as `--name value` or `--name=value`.
|
||||
|
||||
Clones every repository of `--org` into `<target>\<repo>`. Repositories that are
|
||||
already valid git repositories locally are fetched and fast-forwarded instead.
|
||||
When a repository gets its first checkout into a folder that already holds something
|
||||
(it was empty when it was cloned, or its first checkout was interrupted), gclo on
|
||||
Windows replaces a file there only if it is what its own unfinished checkout wrote;
|
||||
otherwise that repository fails and the message names what is in the way.
|
||||
Repositories fail independently — one failure never stops the rest. Up to
|
||||
`--parallel` git operations run at once (default 8, maximum 64 — the same
|
||||
ceiling the desktop app applies, because more would trip GitHub's secondary
|
||||
|
||||
@@ -121,3 +121,4 @@ flowchart LR
|
||||
| 2026-11-10 | v1.1.1 | James Maes (@KofTwentyTwo) | T3/T4 updated: path recovery writes and deletes only through `WorkingTree` (inside the working directory, never in `.git`, never through a link); `.git` and its Windows aliases are invalid path segments; the stored recovery file's paths are validated on load; cleanup of a failed clone does not follow links (#141) |
|
||||
| 2026-11-10 | v1.1.1 | James Maes (@KofTwentyTwo) | T15 added: text from repositories and servers is shown with control, separator and format characters made visible, in the activity log and in everything the CLI prints as text, including characters a legacy console code page would turn into control bytes; the desktop app's surfaces are named as not covered (#143) |
|
||||
| 2026-11-10 | v1.1.1 | James Maes (@KofTwentyTwo) | The project's home moved to `git.kof22.com`: the Help menu, the About dialog, the CLI's error text and the documentation link there, and the Scoop bucket is added from there (the manifest that pins the download URL and its hash now comes from that server over HTTPS; T2 is unchanged in kind). Help > Share hands a fixed text (what gclo is, where to download it, the sender's gclo version) to the system's mail handler as a `mailto:` link with no recipient, or shows it and copies it to the clipboard; nothing about the user, their accounts, or their repositories is in it, and the address it is built from must be a plain http(s) host and path (#163, #165). The update feed and the release downloads named in package manifests are still GitHub Releases at this revision. |
|
||||
| 2026-11-10 | v1.1.1 | James Maes (@KofTwentyTwo) | Working tree (#174; the T3 and T4 rows are unchanged, this adds to what they rely on). On Windows, a forced checkout into a folder whose tree is not finished (the first checkout of a clone that was empty, and the completion of a checkout that was cut off) is preceded by refusals that change nothing: a link on the way or at a destination (except, at a destination where the repository itself has a symbolic link, a link that leads where the repository's does), a file where a folder is needed (also on the way to a submodule), a folder that is not empty where a file is needed, a file at a destination that is not what the unfinished checkout wrote there, and anything but an empty folder where the repository has a submodule. "What the unfinished checkout wrote" is the content, or the beginning of it, of the commit the branch is on or of a commit recorded in `gclo.checkoutwrittenbefore`, as stored or as the checkout filters make it, carriage returns aside. An unfinished checkout is a marked folder that has no index, or that does not hold every file of its commit; a marked folder that has an index and every file is updated like any clone, without force. A completion is refused when the index names a file the commit does not have (somebody added it with git, and the forced checkout would remove it). An unfinished checkout is completed at the commit its branch is on, before the branch moves, except where the branch must move first (a tree Windows cannot write, or a history origin rewrote); the commit it leaves is then recorded, and a path recovery of that folder accepts a file that is, whole, what a recorded commit has. A repository that has local branches is not treated as an empty clone; one without follows origin's default branch only when its folder holds nothing but `.git`. The folder is marked unfinished before its branch is created. One git operation per folder at a time within a process (not across processes, and not across spellings of a path that only the file system can resolve). The WSL clone is made in `~/gclo/.gclo-tmp` and renamed into place when complete (`mv -T`, after a check that nothing is there). Not covered: a file created in the folder between the check and the checkout that follows it; a file of the user's at a destination that is empty, or holds the repository's content or the beginning of it, which is replaced; a link of the user's that leads exactly where the repository's symbolic link at that path does, which is replaced by the repository's; a marked folder whose index somebody's git wrote and whose last file was cut off, which is taken for finished (git shows that file as changed); files of a commit the branch left that the new commit does not have, which stay in the folder untracked; an unfinished clone in which a folder of an earlier commit became a file, or that was cut off by a power loss leaving files of zeros, which is refused until the folder is cleared; platforms other than Windows, where the checkout is forced as before. |
|
||||
File diff suppressed because it is too large.
Load diff
@@ -544,10 +544,21 @@ public sealed class LibGit2GitClientTests : IDisposable
|
||||
|
||||
|
||||
|
||||
/// <summary>
|
||||
/// Sets or clears the marker. Set, the folder is also left without an index, as a
|
||||
/// checkout that was cut off leaves it: git writes the index when a checkout has
|
||||
/// reached its end, and a marked folder that has one is taken for finished.
|
||||
/// </summary>
|
||||
private static void SetCheckoutPending(string workdir, bool value)
|
||||
{
|
||||
using var repo = new Repository(workdir);
|
||||
repo.Config.Set("gclo.checkoutpending", value, ConfigurationLevel.Local);
|
||||
using(var repo = new Repository(workdir))
|
||||
{
|
||||
repo.Config.Set("gclo.checkoutpending", value, ConfigurationLevel.Local);
|
||||
}
|
||||
if(value)
|
||||
{
|
||||
File.Delete(Path.Combine(workdir, ".git", "index"));
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
@@ -0,0 +1,345 @@
|
||||
/*
|
||||
* Copyright (c) 2026 James Maes (KofTwentyTwo)
|
||||
* SPDX-License-Identifier: MIT
|
||||
*/
|
||||
|
||||
using System.Diagnostics;
|
||||
using LibGit2Sharp;
|
||||
using Xunit.Abstractions;
|
||||
using static gclo.Engine.Tests.GitTestHelpers;
|
||||
|
||||
|
||||
namespace gclo.Engine.Tests;
|
||||
|
||||
|
||||
/// <summary>
|
||||
/// Runs <see cref="WslCloner.CloneScript"/> for real, with a POSIX shell and local
|
||||
/// repositories, where the machine has one (Git for Windows brings <c>sh</c>): what
|
||||
/// the script leaves on disk is what matters, and matching its text cannot show
|
||||
/// that. On a machine with no <c>sh</c> that can find <c>git</c> the tests have
|
||||
/// nothing to run: they pass, and say so in their output.
|
||||
/// </summary>
|
||||
public sealed class WslCloneScriptRunTests : IDisposable
|
||||
{
|
||||
private static readonly (string Sh, string Git)? s_tools = FindTools();
|
||||
|
||||
private readonly string _root = Path.Combine(Path.GetTempPath(), "gclo-tests", Guid.NewGuid().ToString("N"));
|
||||
|
||||
private readonly string _home;
|
||||
|
||||
private readonly string _origin;
|
||||
|
||||
|
||||
|
||||
private readonly ITestOutputHelper _output;
|
||||
|
||||
|
||||
|
||||
public WslCloneScriptRunTests(ITestOutputHelper output)
|
||||
{
|
||||
_output = output;
|
||||
_home = Path.Combine(_root, "home");
|
||||
Directory.CreateDirectory(_home);
|
||||
|
||||
// An origin with one commit, made through a working copy.
|
||||
_origin = Path.Combine(_root, "origin.git");
|
||||
Repository.Init(_origin, isBare: true);
|
||||
string seeder = Repository.Clone(_origin, Path.Combine(_root, "seeder"));
|
||||
using var repo = new Repository(seeder);
|
||||
File.WriteAllText(Path.Combine(repo.Info.WorkingDirectory, "readme.txt"), "theirs");
|
||||
Commands.Stage(repo, "readme.txt");
|
||||
Signature who = MakeSignature();
|
||||
repo.Commit("first commit", who, who);
|
||||
repo.Network.Push(repo.Network.Remotes["origin"], repo.Head.CanonicalName, new PushOptions());
|
||||
}
|
||||
|
||||
|
||||
|
||||
public void Dispose() => TryDeleteDirectory(_root);
|
||||
|
||||
|
||||
|
||||
/// <summary>
|
||||
/// A POSIX shell that can find git, or null when this machine has none. Looked for
|
||||
/// on the path and, because Git for Windows puts only its <c>cmd</c> folder there,
|
||||
/// beside every <c>git.exe</c> on the path and where Git for Windows installs.
|
||||
/// </summary>
|
||||
/// <remarks>
|
||||
/// Git for Windows' <c>bin\sh.exe</c> is a launcher: it puts git's own folders in
|
||||
/// front of the path before it starts the shell, and a stand-in for git that a
|
||||
/// test puts on the path is then never found. The shell it starts
|
||||
/// (<c>usr\bin\sh.exe</c>) is used instead, with its own folder on the path for
|
||||
/// the tools a script needs (<see cref="PathWith"/>). Which of the two a test run
|
||||
/// finds first depends on where it was started from.
|
||||
/// </remarks>
|
||||
private static (string Sh, string Git)? FindTools()
|
||||
{
|
||||
string shell = OperatingSystem.IsWindows() ? "sh.exe" : "sh";
|
||||
string[] path = (Environment.GetEnvironmentVariable("PATH") ?? "").Split(Path.PathSeparator, StringSplitOptions.RemoveEmptyEntries);
|
||||
var candidates = new List<string>(path.Select(folder => Path.Combine(folder, shell)));
|
||||
foreach(string folder in path.Where(folder => File.Exists(Path.Combine(folder, "git.exe"))))
|
||||
{
|
||||
candidates.Add(Path.GetFullPath(Path.Combine(folder, "..", "bin", shell)));
|
||||
candidates.Add(Path.GetFullPath(Path.Combine(folder, "..", "usr", "bin", shell)));
|
||||
}
|
||||
foreach(Environment.SpecialFolder programs in (Environment.SpecialFolder[])[Environment.SpecialFolder.ProgramFiles, Environment.SpecialFolder.ProgramFilesX86])
|
||||
{
|
||||
candidates.Add(Path.Combine(Environment.GetFolderPath(programs), "Git", "bin", shell));
|
||||
}
|
||||
|
||||
foreach(string found in candidates.Where(File.Exists))
|
||||
{
|
||||
string started = Path.GetFullPath(Path.Combine(Path.GetDirectoryName(found)!, "..", "usr", "bin", shell));
|
||||
string sh = OperatingSystem.IsWindows() && File.Exists(started) ? started : found;
|
||||
try
|
||||
{
|
||||
var ask = new ProcessStartInfo(sh) { RedirectStandardOutput = true, RedirectStandardError = true, UseShellExecute = false };
|
||||
ask.Environment["PATH"] = PathWith(sh, null);
|
||||
ask.ArgumentList.Add("-c");
|
||||
ask.ArgumentList.Add("command -v git");
|
||||
using Process process = Process.Start(ask)!;
|
||||
string git = process.StandardOutput.ReadToEnd().Trim();
|
||||
process.WaitForExit(30000);
|
||||
if(process.ExitCode == 0 && git.Length > 0)
|
||||
{
|
||||
return (sh, git);
|
||||
}
|
||||
}
|
||||
catch(Exception)
|
||||
{
|
||||
// Not a shell that runs here: try the next one.
|
||||
}
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
|
||||
|
||||
/// <summary>
|
||||
/// The path a shell is run with: <paramref name="first"/> (a test's stand-ins,
|
||||
/// when it has any), the shell's own folder, then the path of this process.
|
||||
/// </summary>
|
||||
private static string PathWith(string sh, string? first)
|
||||
=> string.Join(
|
||||
Path.PathSeparator,
|
||||
new[] { first, Path.GetDirectoryName(sh), Environment.GetEnvironmentVariable("PATH") }.Where(part => !string.IsNullOrEmpty(part)));
|
||||
|
||||
|
||||
|
||||
/// <summary>False, with a line in the test's output, when this machine has no shell to run the script with.</summary>
|
||||
private bool CanRun()
|
||||
{
|
||||
if(s_tools is null)
|
||||
{
|
||||
_output.WriteLine("Not run: no 'sh' that can find 'git' on this machine (Git for Windows brings one).");
|
||||
}
|
||||
return s_tools is not null;
|
||||
}
|
||||
|
||||
|
||||
|
||||
private string Destination(string org, string repo) => Path.Combine(_home, "gclo", org, repo);
|
||||
|
||||
|
||||
|
||||
private string[] Leftovers()
|
||||
{
|
||||
string tmp = Path.Combine(_home, "gclo", ".gclo-tmp");
|
||||
return Directory.Exists(tmp) ? [.. Directory.GetFileSystemEntries(tmp).Select(Path.GetFileName).Order(StringComparer.Ordinal)!] : [];
|
||||
}
|
||||
|
||||
|
||||
|
||||
/// <summary>Runs the script as wsl.exe would hand it to the shell: on standard input, with its four arguments.</summary>
|
||||
private (int ExitCode, string Output, string Error) Run(string org, string repo, string? url = null, string? pathPrefix = null, string? appear = null)
|
||||
{
|
||||
var start = new ProcessStartInfo(s_tools!.Value.Sh)
|
||||
{
|
||||
RedirectStandardInput = true,
|
||||
RedirectStandardOutput = true,
|
||||
RedirectStandardError = true,
|
||||
UseShellExecute = false,
|
||||
};
|
||||
foreach(string argument in (string[])["-s", "--", org, repo, (url ?? _origin).Replace('\\', '/'), "example.test"])
|
||||
{
|
||||
start.ArgumentList.Add(argument);
|
||||
}
|
||||
start.Environment["HOME"] = _home.Replace('\\', '/');
|
||||
start.Environment[WslCloner.TokenVariable] = "not-a-token";
|
||||
start.Environment["WSL_DISTRO_NAME"] = "test";
|
||||
start.Environment["GCLO_TEST_REAL_GIT"] = s_tools.Value.Git;
|
||||
start.Environment["GCLO_TEST_APPEAR"] = (appear ?? "").Replace('\\', '/');
|
||||
start.Environment["PATH"] = PathWith(s_tools.Value.Sh, pathPrefix);
|
||||
|
||||
using Process process = Process.Start(start)!;
|
||||
process.StandardInput.Write(WslCloner.CloneScript);
|
||||
process.StandardInput.Close();
|
||||
Task<string> error = process.StandardError.ReadToEndAsync();
|
||||
string output = process.StandardOutput.ReadToEnd();
|
||||
Assert.True(process.WaitForExit(120000), "the script did not end");
|
||||
return (process.ExitCode, output, error.GetAwaiter().GetResult());
|
||||
}
|
||||
|
||||
|
||||
|
||||
[Fact]
|
||||
public void ANewClone_EndsUpAtTheDestination_AndLeavesNothingBehind_AndASecondRunUpdatesIt()
|
||||
{
|
||||
if(!CanRun())
|
||||
{
|
||||
return;
|
||||
}
|
||||
|
||||
(int exitCode, string output, string error) = Run("acme", "repo");
|
||||
|
||||
Assert.True(exitCode == 0, error);
|
||||
Assert.Equal("theirs", File.ReadAllText(Path.Combine(Destination("acme", "repo"), "readme.txt")));
|
||||
Assert.Empty(Leftovers());
|
||||
Assert.EndsWith("/gclo/acme/repo", output.TrimEnd().Split('\n')[^1].TrimEnd(), StringComparison.Ordinal);
|
||||
|
||||
(int again, _, string secondError) = Run("acme", "repo");
|
||||
Assert.True(again == 0, secondError);
|
||||
}
|
||||
|
||||
|
||||
|
||||
[Fact]
|
||||
public void ACloneThatFails_LeavesNoFolderThatLooksLikeAClone()
|
||||
{
|
||||
if(!CanRun())
|
||||
{
|
||||
return;
|
||||
}
|
||||
|
||||
(int exitCode, _, _) = Run("acme", "repo", url: Path.Combine(_root, "no-such-origin.git"));
|
||||
|
||||
Assert.NotEqual(0, exitCode);
|
||||
Assert.False(Directory.Exists(Destination("acme", "repo")));
|
||||
Assert.Empty(Leftovers());
|
||||
}
|
||||
|
||||
|
||||
|
||||
[Fact]
|
||||
public void AFolderWithSomethingInIt_AtTheDestination_IsRefused_AndAnEmptyOneIsUsed()
|
||||
{
|
||||
if(!CanRun())
|
||||
{
|
||||
return;
|
||||
}
|
||||
Directory.CreateDirectory(Destination("acme", "full"));
|
||||
File.WriteAllText(Path.Combine(Destination("acme", "full"), "mine.txt"), "mine");
|
||||
Directory.CreateDirectory(Destination("acme", "empty"));
|
||||
|
||||
(int refused, _, string error) = Run("acme", "full");
|
||||
(int used, _, string usedError) = Run("acme", "empty");
|
||||
|
||||
Assert.Equal(1, refused);
|
||||
Assert.Contains("exists but is not a repository", error, StringComparison.Ordinal);
|
||||
Assert.Equal("mine", File.ReadAllText(Path.Combine(Destination("acme", "full"), "mine.txt")));
|
||||
Assert.False(Directory.Exists(Path.Combine(Destination("acme", "full"), ".git")));
|
||||
Assert.True(used == 0, usedError);
|
||||
Assert.Equal("theirs", File.ReadAllText(Path.Combine(Destination("acme", "empty"), "readme.txt")));
|
||||
}
|
||||
|
||||
|
||||
|
||||
[Fact]
|
||||
public void ADestinationThatAppearsWhileTheCloneRuns_IsNotMovedInto()
|
||||
{
|
||||
if(!CanRun())
|
||||
{
|
||||
return;
|
||||
}
|
||||
|
||||
// A stand-in for git that runs the real one and then, after a clone, makes
|
||||
// the destination appear: what another gclo cloning the same repository does.
|
||||
string wrappers = Path.Combine(_root, "wrappers");
|
||||
Directory.CreateDirectory(wrappers);
|
||||
File.WriteAllText(
|
||||
Path.Combine(wrappers, "git"),
|
||||
"#!/bin/sh\n\"$GCLO_TEST_REAL_GIT\" \"$@\"\nrc=$?\n"
|
||||
+ "case \" $* \" in *\" clone \"*) mkdir -p \"$GCLO_TEST_APPEAR\"; echo theirs-too > \"$GCLO_TEST_APPEAR/other.txt\";; esac\n"
|
||||
+ "exit $rc\n");
|
||||
string destination = Destination("acme", "repo");
|
||||
|
||||
(int exitCode, _, string error) = Run("acme", "repo", pathPrefix: wrappers, appear: destination);
|
||||
|
||||
Assert.Equal(1, exitCode);
|
||||
Assert.Contains("appeared while the repository was being cloned", error, StringComparison.Ordinal);
|
||||
Assert.Equal(["other.txt"], Directory.GetFileSystemEntries(destination).Select(Path.GetFileName), StringComparer.Ordinal); // no clone nested inside it
|
||||
Assert.Empty(Leftovers());
|
||||
}
|
||||
|
||||
|
||||
|
||||
[Fact]
|
||||
public void OnlyOldLeftovers_InGclosOwnTemporaryFolder_AreRemoved()
|
||||
{
|
||||
if(!CanRun())
|
||||
{
|
||||
return;
|
||||
}
|
||||
string tmp = Path.Combine(_home, "gclo", ".gclo-tmp");
|
||||
string old = Directory.CreateDirectory(Path.Combine(tmp, "acme.other.111")).FullName;
|
||||
File.WriteAllText(Path.Combine(old, "half.txt"), "x");
|
||||
Directory.SetLastWriteTimeUtc(old, DateTime.UtcNow.AddDays(-2));
|
||||
Directory.CreateDirectory(Path.Combine(tmp, "acme.other.222")); // young: perhaps a clone in progress
|
||||
|
||||
// A real clone whose name looks like a leftover's, and is old: not gclo's to remove.
|
||||
string sibling = Directory.CreateDirectory(Destination("acme", "repo.gclo-clone.keep")).FullName;
|
||||
File.WriteAllText(Path.Combine(sibling, "kept.txt"), "mine");
|
||||
Directory.SetLastWriteTimeUtc(sibling, DateTime.UtcNow.AddDays(-2));
|
||||
|
||||
(int exitCode, _, string error) = Run("acme", "repo");
|
||||
|
||||
Assert.True(exitCode == 0, error);
|
||||
Assert.Equal(["acme.other.222"], Leftovers(), StringComparer.Ordinal);
|
||||
Assert.Equal("mine", File.ReadAllText(Path.Combine(sibling, "kept.txt")));
|
||||
}
|
||||
|
||||
|
||||
|
||||
[Fact]
|
||||
public async Task AScriptThatIsEndedByASignal_LeavesNoTemporaryFolder_AndNoClone()
|
||||
{
|
||||
if(!CanRun())
|
||||
{
|
||||
return;
|
||||
}
|
||||
|
||||
// A stand-in for git that clones and then stays busy for a moment, so that the
|
||||
// signal arrives while the script is waiting for it. A shell runs its handler
|
||||
// when the command it waits for has ended; the handler ends the script, and
|
||||
// ending it removes the temporary folder.
|
||||
string wrappers = Path.Combine(_root, "wrappers");
|
||||
Directory.CreateDirectory(wrappers);
|
||||
File.WriteAllText(
|
||||
Path.Combine(wrappers, "git"),
|
||||
"#!/bin/sh\n\"$GCLO_TEST_REAL_GIT\" \"$@\"\nrc=$?\ncase \" $* \" in *\" clone \"*) sleep 3;; esac\nexit $rc\n");
|
||||
string script = Path.Combine(_root, "script.sh");
|
||||
File.WriteAllText(script, WslCloner.CloneScript);
|
||||
|
||||
var start = new ProcessStartInfo(s_tools!.Value.Sh) { RedirectStandardOutput = true, RedirectStandardError = true, UseShellExecute = false };
|
||||
start.ArgumentList.Add("-c");
|
||||
start.ArgumentList.Add(
|
||||
"sh \"$GCLO_TEST_SCRIPT\" acme repo \"$GCLO_TEST_URL\" example.test & pid=$!; sleep 1; kill -TERM $pid; wait $pid; echo \"exit=$?\"");
|
||||
start.Environment["HOME"] = _home.Replace('\\', '/');
|
||||
start.Environment[WslCloner.TokenVariable] = "not-a-token";
|
||||
start.Environment["WSL_DISTRO_NAME"] = "test";
|
||||
start.Environment["GCLO_TEST_REAL_GIT"] = s_tools.Value.Git;
|
||||
start.Environment["GCLO_TEST_SCRIPT"] = script.Replace('\\', '/');
|
||||
start.Environment["GCLO_TEST_URL"] = _origin.Replace('\\', '/');
|
||||
start.Environment["PATH"] = PathWith(s_tools.Value.Sh, wrappers);
|
||||
|
||||
using Process process = Process.Start(start)!;
|
||||
Task<string> errorText = process.StandardError.ReadToEndAsync();
|
||||
string output = await process.StandardOutput.ReadToEndAsync();
|
||||
string error = await errorText;
|
||||
await process.WaitForExitAsync().WaitAsync(TimeSpan.FromMinutes(2));
|
||||
|
||||
Assert.Contains("exit=143", output, StringComparison.Ordinal); // ended by the signal, not by finishing
|
||||
Assert.False(Directory.Exists(Destination("acme", "repo")), error);
|
||||
Assert.Empty(Leftovers());
|
||||
}
|
||||
}
|
||||
@@ -278,8 +278,25 @@ public sealed class WslClonerTests
|
||||
Assert.Equal(2, CountOf(script, "-c credential.helper= -c \"credential.https://$host.helper=$helper\""));
|
||||
Assert.Equal(0, CountOf(script, "\"credential.helper=$helper\""));
|
||||
Assert.Contains("host=\"$4\"", script);
|
||||
// "--" ends option parsing, so a URL can never be read as an option.
|
||||
Assert.Contains("clone --quiet -- \"$url\" \"$dest\"", script);
|
||||
// "--" ends option parsing, so a URL can never be read as an option. The
|
||||
// clone is made beside the destination and moved into place when it is
|
||||
// complete: a clone that is cut off leaves no folder that looks like one.
|
||||
Assert.Contains("clone --quiet -- \"$url\" \"$tmp\"", script);
|
||||
Assert.Contains("tmp=\"$HOME/gclo/.gclo-tmp/$org.$repo.$$\"", script);
|
||||
Assert.Contains("trap 'rm -rf \"$tmp\"' EXIT", script);
|
||||
Assert.True(
|
||||
script.IndexOf("clone --quiet", StringComparison.Ordinal) < script.IndexOf("mv -T \"$tmp\" \"$dest\"", StringComparison.Ordinal),
|
||||
"the folder is moved into place after the clone");
|
||||
Assert.True(
|
||||
script.IndexOf("mv -T \"$tmp\" \"$dest\"", StringComparison.Ordinal) < script.IndexOf("trap - EXIT", StringComparison.Ordinal),
|
||||
"the cleanup stays armed until the folder is in place");
|
||||
Assert.Contains("/gclo/" + WslCloner.TemporaryFolderName + "/", script);
|
||||
// What a killed run left is removed only from gclo's own temporary folder, and
|
||||
// only when it is old: a young one may be another gclo's clone in progress.
|
||||
Assert.Contains("find \"$HOME/gclo/.gclo-tmp\" -mindepth 1 -maxdepth 1 -mmin +1440 -exec rm -rf {} +", script);
|
||||
Assert.DoesNotContain("rm -rf \"$dest\"", script);
|
||||
// Something that is in the way and is not a repository is not cloned into.
|
||||
Assert.Contains("exists but is not a repository", script);
|
||||
// An existing destination is only fast-forwarded when it is a clone of this URL.
|
||||
// The stored URL is compared, not "remote get-url": that one applies the user's
|
||||
// url.*.insteadOf rewrites and would refuse a destination gclo itself cloned.
|
||||
@@ -324,6 +341,21 @@ public sealed class WslClonerTests
|
||||
|
||||
|
||||
|
||||
[Theory]
|
||||
[InlineData(".gclo-tmp")]
|
||||
[InlineData(".GCLO-TMP")]
|
||||
public async Task Clone_IntoGclosOwnTemporaryFolder_IsRefused_WithoutStartingWsl(string organization)
|
||||
{
|
||||
// Unfinished clones are made in ~/gclo/.gclo-tmp, and old ones there are removed.
|
||||
WslCloneException ex = await Assert.ThrowsAsync<WslCloneException>(
|
||||
() => Create().CloneAsync("https://github.com/acme/repo.git", organization, "repo", Token, CancellationToken.None));
|
||||
|
||||
Assert.Contains("gclo's own temporary folder", ex.Message);
|
||||
Assert.Empty(_runner.Calls);
|
||||
}
|
||||
|
||||
|
||||
|
||||
[Theory]
|
||||
[InlineData("http://github.com/acme/repo.git")]
|
||||
[InlineData("ssh://git@github.com/acme/repo.git")]
|
||||
|
||||
+679
-57
@@ -3,6 +3,7 @@
|
||||
* SPDX-License-Identifier: MIT
|
||||
*/
|
||||
|
||||
using System.Collections.Concurrent;
|
||||
using System.Diagnostics.CodeAnalysis;
|
||||
using System.Text.Json;
|
||||
using LibGit2Sharp;
|
||||
@@ -47,6 +48,13 @@ public sealed class LibGit2GitClient : IGitClient
|
||||
/// </summary>
|
||||
private const string CheckoutPendingKey = "gclo.checkoutpending";
|
||||
|
||||
/// <summary>
|
||||
/// Beside the marker: the commits an unfinished checkout here was writing before
|
||||
/// the branch was moved on (ids, separated by commas). Their files may still be
|
||||
/// on disk, and are told from a person's by what those commits hold (#174).
|
||||
/// </summary>
|
||||
private const string WrittenBeforeKey = "gclo.checkoutwrittenbefore";
|
||||
|
||||
/// <summary>
|
||||
/// Local git config flag meaning "gclo writes this repository's working tree
|
||||
/// itself, through a path recovery". It outlives the recovery file, so a
|
||||
@@ -94,6 +102,13 @@ public sealed class LibGit2GitClient : IGitClient
|
||||
/// </summary>
|
||||
internal Action? AfterFetchForTesting { get; set; }
|
||||
|
||||
/// <summary>
|
||||
/// Test seam: runs in the first checkout of an empty clone, after its branch was
|
||||
/// created and before any file is written. A test cancels there to leave exactly
|
||||
/// what a cut-off checkout leaves.
|
||||
/// </summary>
|
||||
internal Action? BeforeFirstCheckoutForTesting { get; set; }
|
||||
|
||||
|
||||
|
||||
/// <inheritdoc/>
|
||||
@@ -107,7 +122,7 @@ public sealed class LibGit2GitClient : IGitClient
|
||||
/// <inheritdoc/>
|
||||
public Task CloneAsync(string url, string path, string token, Action<double>? onProgress, CancellationToken cancellationToken)
|
||||
=> Task.Factory.StartNew(
|
||||
() => Clone(url, path, token, onProgress, cancellationToken),
|
||||
() => OneAtATime(path, () => Clone(url, path, token, onProgress, cancellationToken)),
|
||||
cancellationToken, TaskCreationOptions.LongRunning, TaskScheduler.Default);
|
||||
|
||||
|
||||
@@ -115,7 +130,7 @@ public sealed class LibGit2GitClient : IGitClient
|
||||
/// <inheritdoc/>
|
||||
public Task FetchAndPullAsync(string url, string path, string token, CancellationToken cancellationToken)
|
||||
=> Task.Factory.StartNew(
|
||||
() => FetchAndPull(url, path, token, cancellationToken),
|
||||
() => OneAtATime(path, () => FetchAndPull(url, path, token, cancellationToken)),
|
||||
cancellationToken, TaskCreationOptions.LongRunning, TaskScheduler.Default);
|
||||
|
||||
|
||||
@@ -123,11 +138,71 @@ public sealed class LibGit2GitClient : IGitClient
|
||||
/// <inheritdoc/>
|
||||
public Task ApplyRecoveryAsync(string path, PathRecovery recovery, CancellationToken cancellationToken)
|
||||
=> Task.Factory.StartNew(
|
||||
() => ApplyRecovery(path, recovery, cancellationToken),
|
||||
() => OneAtATime(path, () => ApplyRecovery(path, recovery, cancellationToken)),
|
||||
cancellationToken, TaskCreationOptions.LongRunning, TaskScheduler.Default);
|
||||
|
||||
|
||||
|
||||
/// <summary>
|
||||
/// The folders a git operation of this process is working in right now. A caller
|
||||
/// that stops waiting for an operation (a canceled sync does) does not stop the
|
||||
/// operation: it goes on until git notices, and then cleans up after itself, which
|
||||
/// for a clone means deleting the folder. Starting the next operation on that
|
||||
/// folder before then would have the two working against each other.
|
||||
/// </summary>
|
||||
private static readonly ConcurrentDictionary<string, byte> s_inFlight =
|
||||
new(OperatingSystem.IsWindows() ? StringComparer.OrdinalIgnoreCase : StringComparer.Ordinal);
|
||||
|
||||
|
||||
|
||||
/// <summary>
|
||||
/// Runs <paramref name="operation"/> as the only one in <paramref name="path"/>;
|
||||
/// refuses at once when an earlier one there has not ended yet.
|
||||
/// </summary>
|
||||
private static void OneAtATime(string path, Action operation)
|
||||
{
|
||||
///////////////////////////////////////////////////////////////////////////////
|
||||
// The device and long-path spellings of a path name the same folder, and //
|
||||
// GetFullPath leaves them as they are: they are taken off first. Spellings //
|
||||
// only the file system can resolve (a share on this machine, a substituted //
|
||||
// drive, a link in a parent) are not recognised as the same folder. //
|
||||
///////////////////////////////////////////////////////////////////////////////
|
||||
string folder = path;
|
||||
foreach(string prefix in (string[])[@"\\?\UNC\", @"\\.\UNC\", @"\??\UNC\"])
|
||||
{
|
||||
if(folder.StartsWith(prefix, StringComparison.OrdinalIgnoreCase))
|
||||
{
|
||||
folder = @"\\" + folder[prefix.Length..];
|
||||
}
|
||||
}
|
||||
foreach(string prefix in (string[])[@"\\?\", @"\\.\", @"\??\"])
|
||||
{
|
||||
if(folder.StartsWith(prefix, StringComparison.Ordinal))
|
||||
{
|
||||
folder = folder[prefix.Length..];
|
||||
}
|
||||
}
|
||||
folder = Path.TrimEndingDirectorySeparator(Path.GetFullPath(folder));
|
||||
if(!s_inFlight.TryAdd(folder, 0))
|
||||
{
|
||||
throw new InvalidOperationException(
|
||||
"Another git operation of gclo is still working in this folder: one that was canceled and has not ended yet "
|
||||
+ "(it ends on its own a moment after it was canceled), or another workspace syncing the same folder. "
|
||||
+ "Sync again when it has finished.");
|
||||
}
|
||||
|
||||
try
|
||||
{
|
||||
operation();
|
||||
}
|
||||
finally
|
||||
{
|
||||
s_inFlight.TryRemove(folder, out _);
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
private void Clone(string url, string path, string token, Action<double>? onProgress, CancellationToken ct)
|
||||
{
|
||||
bool existedBefore = Directory.Exists(path);
|
||||
@@ -202,9 +277,14 @@ public sealed class LibGit2GitClient : IGitClient
|
||||
ct.ThrowIfCancellationRequested();
|
||||
Commands.Checkout(repo, repo.Head, new CheckoutOptions
|
||||
{
|
||||
CheckoutModifiers = CheckoutModifiers.Force, // materialize the fresh working tree
|
||||
// The only checkout that lacked the hook: a canceled clone used to
|
||||
// write its whole tree before the token was ever observed (#31).
|
||||
////////////////////////////////////////////////////////////////////////
|
||||
// Force: materialize the fresh working tree. The hook lets a //
|
||||
// cancellation stop the checkout before it has written much; once //
|
||||
// libgit2 is writing, it may run to the end, and the cancellation is //
|
||||
// then the caller's to see (the engine stops waiting; the folder //
|
||||
// stays marked as busy until this returns). //
|
||||
////////////////////////////////////////////////////////////////////////
|
||||
CheckoutModifiers = CheckoutModifiers.Force,
|
||||
CheckoutNotifyFlags = CheckoutNotifyFlags.Updated,
|
||||
OnCheckoutNotify = (_, _) => !ct.IsCancellationRequested,
|
||||
});
|
||||
@@ -289,10 +369,14 @@ public sealed class LibGit2GitClient : IGitClient
|
||||
return;
|
||||
}
|
||||
|
||||
if(IsCheckoutPending(repo))
|
||||
if(IsCheckoutPending(repo) && !repo.Info.IsHeadUnborn)
|
||||
{
|
||||
CompletePendingCheckout(repo, ct);
|
||||
return;
|
||||
/////////////////////////////////////////////////////////////////////
|
||||
// With an unborn HEAD there is no commit to complete: the first //
|
||||
// checkout was cut off before its branch existed, and starts over //
|
||||
// below. //
|
||||
/////////////////////////////////////////////////////////////////////
|
||||
FinishOrReleaseAPendingCheckout(repo, ct);
|
||||
}
|
||||
|
||||
if(repo.Info.IsHeadDetached)
|
||||
@@ -307,7 +391,7 @@ public sealed class LibGit2GitClient : IGitClient
|
||||
// Empty clone — or a clone that was killed before its first checkout ran.
|
||||
// If the remote branch exists now, materialize it instead of silently
|
||||
// reporting an empty working tree as up to date forever.
|
||||
MaterializeUnbornHead(repo, ct);
|
||||
MaterializeUnbornHead(repo, url, token, ct);
|
||||
return;
|
||||
}
|
||||
|
||||
@@ -370,6 +454,25 @@ public sealed class LibGit2GitClient : IGitClient
|
||||
$"'{repo.Head.FriendlyName}' has local commits that origin does not have; gclo only fast-forwards and never merges. "
|
||||
+ "Push or reset the local commits, or delete the folder to re-clone.", ex);
|
||||
}
|
||||
catch(CheckoutConflictException ex) when(File.Exists(Path.Combine(repo.Info.Path, "index")))
|
||||
{
|
||||
throw new InvalidOperationException(
|
||||
"The update would replace files in this folder that are not as git recorded them (changed or added here, or "
|
||||
+ $"left by an update that was interrupted), so nothing was updated ({ex.Message}). 'git status' lists them: "
|
||||
+ "commit, move or delete them, then sync again.", ex);
|
||||
}
|
||||
catch(CheckoutConflictException ex)
|
||||
{
|
||||
///////////////////////////////////////////////////////////////////////////
|
||||
// Without an index git cannot tell which files are unchanged, takes //
|
||||
// every one of them for a conflict, and says only how many. Say what //
|
||||
// is wrong and what clears it; nothing was changed. //
|
||||
///////////////////////////////////////////////////////////////////////////
|
||||
throw new InvalidOperationException(
|
||||
"This clone has no index ('.git/index' is missing), so git cannot tell which files are unchanged and "
|
||||
+ $"refuses to update any of them ({ex.Message}). Run 'git -C \"{Path.TrimEndingDirectorySeparator(repo.Info.WorkingDirectory)}\" reset' "
|
||||
+ "to rebuild the index, or delete the folder to re-clone.", ex);
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -542,9 +645,8 @@ public sealed class LibGit2GitClient : IGitClient
|
||||
const string Prefix = "refs/heads/";
|
||||
string oldName = repo.Head.FriendlyName;
|
||||
|
||||
string? defaultRef = repo.Network.ListReferences(url, credentials)
|
||||
.FirstOrDefault(r => string.Equals(r.CanonicalName, "HEAD", StringComparison.Ordinal))?.TargetIdentifier;
|
||||
if(defaultRef is null || !defaultRef.StartsWith(Prefix, StringComparison.Ordinal))
|
||||
string? defaultRef = RemoteDefaultBranch(repo, url, credentials);
|
||||
if(defaultRef is null)
|
||||
{
|
||||
throw new InvalidOperationException(
|
||||
$"'{oldName}' no longer exists on origin, and origin has no default branch to follow. "
|
||||
@@ -601,11 +703,42 @@ public sealed class LibGit2GitClient : IGitClient
|
||||
|
||||
|
||||
/// <summary>
|
||||
/// HEAD symbolically points at a branch with no commits. If the fetch brought that
|
||||
/// branch's remote counterpart, create the local branch there, wire its upstream,
|
||||
/// and check it out to populate the working tree.
|
||||
/// The branch origin says is its default ("refs/heads/..."), asked of origin itself;
|
||||
/// null when origin does not name one (an empty repository).
|
||||
/// </summary>
|
||||
private static void MaterializeUnbornHead(Repository repo, CancellationToken ct)
|
||||
private static string? RemoteDefaultBranch(Repository repo, string url, CredentialsHandler credentials)
|
||||
{
|
||||
string? head = repo.Network.ListReferences(url, credentials)
|
||||
.FirstOrDefault(r => string.Equals(r.CanonicalName, "HEAD", StringComparison.Ordinal))?.TargetIdentifier;
|
||||
return head is not null && head.StartsWith("refs/heads/", StringComparison.Ordinal) ? head : null;
|
||||
}
|
||||
|
||||
|
||||
|
||||
/// <summary>
|
||||
/// HEAD symbolically points at a branch with no commits: an empty clone, or a clone
|
||||
/// that was cut off before its first checkout. If the fetch brought a branch to
|
||||
/// check out, create the local branch there, wire its upstream, and check it out
|
||||
/// to populate the working tree.
|
||||
/// </summary>
|
||||
/// <remarks>
|
||||
/// <para>
|
||||
/// Only a repository without any branch of its own is treated as an empty
|
||||
/// clone; one that has local branches is left as it is. Which branch is then
|
||||
/// checked out: the one HEAD names when origin has it. When origin does not, and
|
||||
/// the folder holds nothing but its <c>.git</c> (nobody has put anything into it),
|
||||
/// HEAD still names the branch a new repository starts with here, which need not
|
||||
/// be origin's: origin is asked for its default branch, and that one is checked
|
||||
/// out. A folder that somebody has put files into keeps the branch HEAD names.
|
||||
/// </para>
|
||||
/// <para>
|
||||
/// Nothing changes before everything was checked: the branch to check out, the
|
||||
/// paths Windows cannot create, and what is already in the folder
|
||||
/// (<see cref="RefuseToWriteOverWhatIsNotOurs"/>). Only then is HEAD pointed at
|
||||
/// the branch, the folder marked unfinished, and the branch created.
|
||||
/// </para>
|
||||
/// </remarks>
|
||||
private void MaterializeUnbornHead(Repository repo, string url, string token, CancellationToken ct)
|
||||
{
|
||||
const string Prefix = "refs/heads/";
|
||||
string targetRef = repo.Refs.Head.TargetIdentifier;
|
||||
@@ -615,10 +748,44 @@ public sealed class LibGit2GitClient : IGitClient
|
||||
}
|
||||
|
||||
string branchName = targetRef[Prefix.Length..];
|
||||
|
||||
/////////////////////////////////////////////////////////////////////////////////
|
||||
// A repository with branches of its own is not an empty clone. Its unborn //
|
||||
// HEAD is a branch a person started ("git switch --orphan"), and it is //
|
||||
// theirs, whatever origin has under that name. (With files staged it is the //
|
||||
// refusal further down that says so.) //
|
||||
/////////////////////////////////////////////////////////////////////////////////
|
||||
bool hasBranchesOfItsOwn = repo.Branches.Any(b => !b.IsRemote);
|
||||
Branch remoteBranch = repo.Branches["origin/" + branchName];
|
||||
bool followOriginsDefault = false;
|
||||
if(remoteBranch?.Tip is null)
|
||||
{
|
||||
return; // the remote is still empty; nothing to pull
|
||||
if(hasBranchesOfItsOwn || repo.Index.Count > 0 || !repo.Branches.Any(b => b.IsRemote) || !HoldsNothingButItsGitFolder(repo))
|
||||
{
|
||||
/////////////////////////////////////////////////////////////////////////
|
||||
// Origin is still empty; or this is not the untouched folder of an //
|
||||
// empty clone: it has branches, files are staged, or something was //
|
||||
// put into it. Then the branch HEAD names may be one somebody chose, //
|
||||
// and it is left. //
|
||||
/////////////////////////////////////////////////////////////////////////
|
||||
return;
|
||||
}
|
||||
|
||||
/////////////////////////////////////////////////////////////////////////////
|
||||
// A provider of its own: the fetch already spent one of the two requests //
|
||||
// the replay guard allows. //
|
||||
/////////////////////////////////////////////////////////////////////////////
|
||||
string? defaultRef = RemoteDefaultBranch(repo, url, MakeCredentialsProvider(token, url));
|
||||
Branch? defaultBranch = defaultRef is null ? null : repo.Branches["origin/" + defaultRef[Prefix.Length..]];
|
||||
if(defaultBranch?.Tip is null)
|
||||
{
|
||||
return; // origin names no default branch, or it was not fetched
|
||||
}
|
||||
|
||||
followOriginsDefault = true;
|
||||
targetRef = defaultRef!;
|
||||
branchName = targetRef[Prefix.Length..];
|
||||
remoteBranch = defaultBranch;
|
||||
}
|
||||
|
||||
///////////////////////////////////////////////////////////////////////////////
|
||||
@@ -627,6 +794,11 @@ public sealed class LibGit2GitClient : IGitClient
|
||||
// the branch it was started from. Checking out over that, with force, would //
|
||||
// replace someone's files. //
|
||||
///////////////////////////////////////////////////////////////////////////////
|
||||
if(hasBranchesOfItsOwn && repo.Index.Count == 0)
|
||||
{
|
||||
return;
|
||||
}
|
||||
|
||||
if(repo.Index.Count > 0)
|
||||
{
|
||||
throw new InvalidOperationException(
|
||||
@@ -644,19 +816,34 @@ public sealed class LibGit2GitClient : IGitClient
|
||||
}
|
||||
}
|
||||
|
||||
RefuseToWriteOverWhatIsNotOurs(repo, remoteBranch.Tip.Tree, aCutOffWriteIsOurs: false);
|
||||
|
||||
///////////////////////////////////////////////////////////////////////////////
|
||||
// From here on the repository changes, in an order in which every point it //
|
||||
// can be cut off at is one the next pull recovers from: HEAD names the //
|
||||
// branch (still unborn: this method runs again); the branch's upstream is //
|
||||
// configured before the branch exists; the folder is marked unfinished //
|
||||
// before the branch exists, because from then on it looks like any clone //
|
||||
// and only the marker says that its tree is not written yet. //
|
||||
///////////////////////////////////////////////////////////////////////////////
|
||||
if(followOriginsDefault)
|
||||
{
|
||||
repo.Refs.UpdateTarget("HEAD", targetRef);
|
||||
}
|
||||
repo.Config.Set($"branch.{branchName}.remote", "origin", ConfigurationLevel.Local);
|
||||
repo.Config.Set($"branch.{branchName}.merge", targetRef, ConfigurationLevel.Local);
|
||||
repo.Config.Set(CheckoutPendingKey, true, ConfigurationLevel.Local);
|
||||
repo.Refs.Add(targetRef, remoteBranch.Tip.Id);
|
||||
Branch local = repo.Branches[branchName];
|
||||
repo.Branches.Update(local, b =>
|
||||
{
|
||||
b.Remote = "origin";
|
||||
b.UpstreamBranch = targetRef;
|
||||
});
|
||||
BeforeFirstCheckoutForTesting?.Invoke();
|
||||
|
||||
ct.ThrowIfCancellationRequested();
|
||||
try
|
||||
{
|
||||
// Force: the working tree of a killed half-clone may hold partial files;
|
||||
// recovering means making it match the branch tip exactly.
|
||||
////////////////////////////////////////////////////////////////////////////
|
||||
// Force: nothing that is not gclo's is in the way (checked above), and //
|
||||
// nothing was ever checked out here, so there is no state to merge. //
|
||||
////////////////////////////////////////////////////////////////////////////
|
||||
Commands.Checkout(repo, local, new CheckoutOptions
|
||||
{
|
||||
CheckoutModifiers = CheckoutModifiers.Force,
|
||||
@@ -668,6 +855,174 @@ public sealed class LibGit2GitClient : IGitClient
|
||||
{
|
||||
throw new OperationCanceledException(ct);
|
||||
}
|
||||
|
||||
repo.Config.Set(CheckoutPendingKey, false, ConfigurationLevel.Local);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/// <summary>True when the working directory holds nothing but the repository's own <c>.git</c>.</summary>
|
||||
private static bool HoldsNothingButItsGitFolder(Repository repo)
|
||||
=> Directory.EnumerateFileSystemEntries(WorkingDirectoryOf(repo))
|
||||
.All(entry => string.Equals(Path.GetFileName(entry), ".git", StringComparison.OrdinalIgnoreCase));
|
||||
|
||||
|
||||
|
||||
/// <summary>
|
||||
/// Before a forced checkout into a folder whose tree gclo has not finished writing:
|
||||
/// refuses, with nothing changed, when something that is not gclo's is where the
|
||||
/// incoming tree has a file, needs a folder, or has a submodule. A forced checkout
|
||||
/// replaces all of those without asking (#174).
|
||||
/// </summary>
|
||||
/// <remarks>
|
||||
/// <para>
|
||||
/// The rules are the ones the path recovery follows
|
||||
/// (<see cref="RefuseWhatCannotBeWritten"/>), and a few it has no need for: a link
|
||||
/// on the way or at a destination (at a destination where the repository has a
|
||||
/// symbolic link, only a link that leads elsewhere), a file where a folder is
|
||||
/// needed (also on the way to a submodule), a folder with anything in it where a
|
||||
/// file is needed, a file at a destination that is not what the current commit has
|
||||
/// there, and anything but an empty folder where the repository has a submodule
|
||||
/// (the checkout makes an empty folder of it).
|
||||
/// </para>
|
||||
/// <para>
|
||||
/// With <paramref name="aCutOffWriteIsOurs"/> the checkout being completed has
|
||||
/// written files already. One of them at a destination is recognised by holding
|
||||
/// what the current commit has for that path (or a commit of
|
||||
/// <paramref name="writtenBefore"/>: one the branch was on while the checkout was
|
||||
/// unfinished), or the beginning of it (the file it was writing when it was cut
|
||||
/// off), line endings aside: the checkout converts them by rules that the tree
|
||||
/// itself brings, and those were not all on disk yet when the first files were
|
||||
/// written.
|
||||
/// </para>
|
||||
/// <para>
|
||||
/// Windows only. The rules about names and links are Windows' rules; elsewhere the
|
||||
/// checkout is done as it always was.
|
||||
/// </para>
|
||||
/// </remarks>
|
||||
private static void RefuseToWriteOverWhatIsNotOurs(
|
||||
Repository repo, Tree incoming, bool aCutOffWriteIsOurs, IReadOnlyList<Tree>? writtenBefore = null)
|
||||
{
|
||||
if(!OperatingSystem.IsWindows())
|
||||
{
|
||||
return;
|
||||
}
|
||||
|
||||
string workdir = WorkingDirectoryOf(repo);
|
||||
List<(string OriginalPath, string EffectivePath, Blob Blob)> entries = CollectEffectiveEntries(incoming, s_noRecovery);
|
||||
|
||||
///////////////////////////////////////////////////////////////////////////////
|
||||
// Where the repository has a symbolic link, a link in the folder that leads //
|
||||
// where the repository's does is what a checkout writes there (when the //
|
||||
// user's configuration lets it), or the same thing made by hand: it is not //
|
||||
// looked at further. The folders on the way to it are, as for every entry. //
|
||||
///////////////////////////////////////////////////////////////////////////////
|
||||
entries.RemoveAll(entry =>
|
||||
incoming[entry.OriginalPath]?.Mode == Mode.SymbolicLink && IsTheLinkTheRepositoryHas(workdir, entry.EffectivePath, entry.Blob));
|
||||
|
||||
var effectiveNow = new HashSet<string>(entries.Select(entry => entry.EffectivePath), StringComparer.Ordinal);
|
||||
RefuseWhatCannotBeWritten(
|
||||
repo, workdir, entries, ours: null, effectiveNow, replaceOnlyWhatTheCommitHas: true, aCutOffWriteIsOurs, writtenBefore);
|
||||
|
||||
///////////////////////////////////////////////////////////////////////////////
|
||||
// What that check lets through because the recovery writes file by file and //
|
||||
// a forced checkout does not: a link at a destination (replaced by the //
|
||||
// file), and a folder there that holds no file but something else (links, //
|
||||
// empty folders), which the checkout removes with what is in it. //
|
||||
///////////////////////////////////////////////////////////////////////////////
|
||||
foreach((_, string effectivePath, _) in entries)
|
||||
{
|
||||
string full = Path.Combine(workdir, effectivePath.Replace('/', Path.DirectorySeparatorChar));
|
||||
if(IsLink(full))
|
||||
{
|
||||
throw new InvalidOperationException(
|
||||
$"'{effectivePath}' is a link in this folder, where the repository has a file. Move or delete the link, then try again.");
|
||||
}
|
||||
if(Directory.Exists(full) && Directory.EnumerateFileSystemEntries(full).Any())
|
||||
{
|
||||
throw new InvalidOperationException(
|
||||
$"'{effectivePath}' is a folder that is not empty, where the repository has a file. Move or delete the folder, then try again.");
|
||||
}
|
||||
}
|
||||
|
||||
var checkedFolders = new HashSet<string>(StringComparer.Ordinal);
|
||||
foreach(string submodule in SubmodulePaths(incoming))
|
||||
{
|
||||
WorkingTree.Verify(workdir, submodule);
|
||||
for(int slash = submodule.IndexOf('/', StringComparison.Ordinal); slash > 0; slash = submodule.IndexOf('/', slash + 1))
|
||||
{
|
||||
string folder = submodule[..slash];
|
||||
if(checkedFolders.Add(folder) && File.Exists(Path.Combine(workdir, folder.Replace('/', Path.DirectorySeparatorChar))))
|
||||
{
|
||||
throw new InvalidOperationException(
|
||||
$"'{folder}' is a file in this folder, and the repository has a folder of that name. "
|
||||
+ "Move or delete the file, then try again.");
|
||||
}
|
||||
}
|
||||
|
||||
string full = Path.Combine(workdir, submodule.Replace('/', Path.DirectorySeparatorChar));
|
||||
if(IsLink(full) || File.Exists(full) || (Directory.Exists(full) && Directory.EnumerateFileSystemEntries(full).Any()))
|
||||
{
|
||||
throw new InvalidOperationException(
|
||||
$"'{submodule}' is already in this folder, where the repository has a submodule, and it is not an empty folder. "
|
||||
+ "Move or delete it, then try again.");
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/// <summary>
|
||||
/// True when something is at <paramref name="path"/> and it is a link (a junction, a
|
||||
/// symbolic link): recognised by having a target, as <see cref="WorkingTree"/> does,
|
||||
/// not by the reparse-point attribute, which other things carry too.
|
||||
/// </summary>
|
||||
private static bool IsLink(string path)
|
||||
=> new FileInfo(path).LinkTarget is not null || new DirectoryInfo(path).LinkTarget is not null;
|
||||
|
||||
|
||||
|
||||
/// <summary>
|
||||
/// True when a link is at <paramref name="effectivePath"/> and it leads where the
|
||||
/// repository's symbolic link there does (<paramref name="link"/>, whose content
|
||||
/// is the target). A link on the way to it is refused, as on the way to anything
|
||||
/// that is written.
|
||||
/// </summary>
|
||||
private static bool IsTheLinkTheRepositoryHas(string workdir, string effectivePath, Blob link)
|
||||
{
|
||||
WorkingTree.Verify(workdir, effectivePath);
|
||||
|
||||
string full = Path.Combine(workdir, effectivePath.Replace('/', Path.DirectorySeparatorChar));
|
||||
string? target = new FileInfo(full).LinkTarget ?? new DirectoryInfo(full).LinkTarget;
|
||||
return target is not null
|
||||
&& string.Equals(target.Replace('/', '\\'), link.GetContentText().Replace('/', '\\'), StringComparison.Ordinal);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/// <summary>The paths of the submodule entries (gitlinks) of a tree, at any depth.</summary>
|
||||
private static List<string> SubmodulePaths(Tree tree)
|
||||
{
|
||||
var found = new List<string>();
|
||||
var stack = new Stack<(Tree Tree, string Prefix)>();
|
||||
stack.Push((tree, ""));
|
||||
while(stack.Count > 0)
|
||||
{
|
||||
(Tree current, string prefix) = stack.Pop();
|
||||
foreach(TreeEntry entry in current)
|
||||
{
|
||||
string path = prefix.Length == 0 ? entry.Name : $"{prefix}/{entry.Name}";
|
||||
if(entry.TargetType == TreeEntryTargetType.Tree)
|
||||
{
|
||||
stack.Push(((Tree)entry.Target, path));
|
||||
}
|
||||
else if(entry.TargetType == TreeEntryTargetType.GitLink)
|
||||
{
|
||||
found.Add(path);
|
||||
}
|
||||
}
|
||||
}
|
||||
return found;
|
||||
}
|
||||
|
||||
|
||||
@@ -768,7 +1123,8 @@ public sealed class LibGit2GitClient : IGitClient
|
||||
|
||||
HashSet<string>? ours = stored?.MaterializedPaths is null ? null : new HashSet<string>(stored.MaterializedPaths, StringComparer.Ordinal);
|
||||
var effectiveNow = new HashSet<string>(entries.Select(e => e.EffectivePath), StringComparer.Ordinal);
|
||||
RefuseWhatCannotBeWritten(repo, workdir, entries, ours, effectiveNow, replaceOnlyWhatTheCommitHas: !managedByRecord);
|
||||
RefuseWhatCannotBeWritten(
|
||||
repo, workdir, entries, ours, effectiveNow, replaceOnlyWhatTheCommitHas: !managedByRecord, writtenBefore: WrittenBefore(repo));
|
||||
|
||||
/////////////////////////////////////////////////////////////////////////////
|
||||
// The point of no return. Record first that this tree is being rewritten, //
|
||||
@@ -811,7 +1167,7 @@ public sealed class LibGit2GitClient : IGitClient
|
||||
}
|
||||
|
||||
SaveRecovery(repo, recovery, effectiveNow, tip.Sha);
|
||||
repo.Config.Set(CheckoutPendingKey, false, ConfigurationLevel.Local);
|
||||
MarkFinished(repo);
|
||||
}
|
||||
|
||||
|
||||
@@ -1050,7 +1406,9 @@ public sealed class LibGit2GitClient : IGitClient
|
||||
List<(string OriginalPath, string EffectivePath, Blob Blob)> entries,
|
||||
HashSet<string>? ours,
|
||||
HashSet<string> effectiveNow,
|
||||
bool replaceOnlyWhatTheCommitHas)
|
||||
bool replaceOnlyWhatTheCommitHas,
|
||||
bool aCutOffWriteIsOurs = false,
|
||||
IReadOnlyList<Tree>? writtenBefore = null)
|
||||
{
|
||||
if(File.Exists(Path.Combine(repo.Info.Path, "index.lock")))
|
||||
{
|
||||
@@ -1072,7 +1430,10 @@ public sealed class LibGit2GitClient : IGitClient
|
||||
WorkingTree.Verify(workdir, effectivePath, verifiedFolders);
|
||||
|
||||
string full = Path.Combine(workdir, effectivePath.Replace('/', Path.DirectorySeparatorChar));
|
||||
if(replaceOnlyWhatTheCommitHas && File.Exists(full) && !IsWhatTheCommitHas(full, originalPath, current))
|
||||
if(replaceOnlyWhatTheCommitHas
|
||||
&& File.Exists(full)
|
||||
&& !IsWhatTheCommitHas(full, originalPath, current, aCutOffWriteIsOurs)
|
||||
&& !(writtenBefore ?? []).Any(earlier => IsWhatTheCommitHas(full, originalPath, earlier, aCutOffWriteIsOurs)))
|
||||
{
|
||||
throw new InvalidOperationException(
|
||||
$"'{effectivePath}' is already in this folder, and it is not a file gclo has a record of writing "
|
||||
@@ -1118,16 +1479,41 @@ public sealed class LibGit2GitClient : IGitClient
|
||||
/// True when the file at <paramref name="full"/> holds exactly what a checkout of
|
||||
/// <paramref name="commitTree"/> writes for <paramref name="originalPath"/>
|
||||
/// (line-ending conversion included). False when the commit has no such file.
|
||||
/// With <paramref name="orItsBeginning"/> the question is whether an unfinished
|
||||
/// checkout wrote the file: then it holds the commit's content for the path, or
|
||||
/// the beginning of it, line endings aside
|
||||
/// (<see cref="HoldsTheBeginningOf"/>).
|
||||
/// </summary>
|
||||
private static bool IsWhatTheCommitHas(string full, string originalPath, Tree? commitTree)
|
||||
private static bool IsWhatTheCommitHas(string full, string originalPath, Tree? commitTree, bool orItsBeginning = false)
|
||||
{
|
||||
if(commitTree?[originalPath]?.Target is not Blob blob)
|
||||
{
|
||||
return false;
|
||||
}
|
||||
|
||||
using Stream expected = blob.GetContentStream(new FilteringOptions(originalPath));
|
||||
if(orItsBeginning)
|
||||
{
|
||||
/////////////////////////////////////////////////////////////////////////////
|
||||
// What is stored, or what the filters make of it on the way out (the //
|
||||
// "ident" attribute writes the object's id into the file): either is //
|
||||
// what a checkout writes, depending on which rules were on disk then. //
|
||||
/////////////////////////////////////////////////////////////////////////////
|
||||
using(Stream stored = blob.GetContentStream())
|
||||
using(var written = new FileStream(full, FileMode.Open, FileAccess.Read, FileShare.ReadWrite | FileShare.Delete))
|
||||
{
|
||||
if(HoldsTheBeginningOf(stored, written))
|
||||
{
|
||||
return true;
|
||||
}
|
||||
}
|
||||
|
||||
using Stream filtered = blob.GetContentStream(new FilteringOptions(originalPath));
|
||||
using var again = new FileStream(full, FileMode.Open, FileAccess.Read, FileShare.ReadWrite | FileShare.Delete);
|
||||
return HoldsTheBeginningOf(filtered, again);
|
||||
}
|
||||
|
||||
using var actual = new FileStream(full, FileMode.Open, FileAccess.Read, FileShare.ReadWrite | FileShare.Delete);
|
||||
using Stream expected = blob.GetContentStream(new FilteringOptions(originalPath));
|
||||
byte[] expectedBuffer = new byte[81920];
|
||||
byte[] actualBuffer = new byte[81920];
|
||||
while(true)
|
||||
@@ -1147,6 +1533,62 @@ public sealed class LibGit2GitClient : IGitClient
|
||||
|
||||
|
||||
|
||||
/// <summary>
|
||||
/// True when <paramref name="actual"/> holds what <paramref name="expected"/> holds,
|
||||
/// or the beginning of it and nothing else, carriage returns aside: a file a
|
||||
/// checkout wrote, or was writing when it was cut off. Carriage returns are left
|
||||
/// out on both sides because a checkout converts line endings by the repository's
|
||||
/// attributes and the user's configuration, and a text that differs from the
|
||||
/// commit's only in its line endings is the commit's text.
|
||||
/// </summary>
|
||||
private static bool HoldsTheBeginningOf(Stream expected, Stream actual)
|
||||
{
|
||||
const byte CarriageReturn = 13;
|
||||
byte[] expectedBuffer = new byte[81920];
|
||||
byte[] actualBuffer = new byte[81920];
|
||||
int expectedCount = 0;
|
||||
int expectedAt = 0;
|
||||
while(true)
|
||||
{
|
||||
int read = actual.Read(actualBuffer, 0, actualBuffer.Length);
|
||||
if(read == 0)
|
||||
{
|
||||
return true; // the file has ended: all of it was the commit's, from its start
|
||||
}
|
||||
|
||||
for(int i = 0; i < read; i++)
|
||||
{
|
||||
if(actualBuffer[i] == CarriageReturn)
|
||||
{
|
||||
continue;
|
||||
}
|
||||
|
||||
byte next;
|
||||
do
|
||||
{
|
||||
if(expectedAt == expectedCount)
|
||||
{
|
||||
expectedCount = expected.Read(expectedBuffer, 0, expectedBuffer.Length);
|
||||
expectedAt = 0;
|
||||
if(expectedCount == 0)
|
||||
{
|
||||
return false; // the file goes on where the commit's has ended
|
||||
}
|
||||
}
|
||||
next = expectedBuffer[expectedAt++];
|
||||
}
|
||||
while(next == CarriageReturn);
|
||||
|
||||
if(next != actualBuffer[i])
|
||||
{
|
||||
return false;
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/// <summary>
|
||||
/// Overlays <paramref name="incoming"/> onto <paramref name="stored"/>. The newer
|
||||
/// choice for a path wins whatever kind it is: an incoming rename replaces a stored
|
||||
@@ -1237,45 +1679,225 @@ public sealed class LibGit2GitClient : IGitClient
|
||||
|
||||
|
||||
/// <summary>
|
||||
/// Finishes a clone whose checkout never completed and that has no stored
|
||||
/// recovery: moves to the upstream tip, re-validates its paths on Windows, and
|
||||
/// force-checks-out the whole tree (whatever partial files exist are overwritten
|
||||
/// to match). If the paths are still invalid, rethrows the same typed failure —
|
||||
/// the repo must not silently report success with an empty or partial tree.
|
||||
/// What a marked folder needs before the ordinary pull: its checkout finished, or
|
||||
/// only its marker taken off.
|
||||
/// </summary>
|
||||
/// <remarks>
|
||||
/// git writes the index when a checkout has reached its end. A marked folder
|
||||
/// without an index is therefore a tree that was never finished: it is completed,
|
||||
/// with force (<see cref="CompletePendingCheckout"/>). A marked folder that has an
|
||||
/// index and holds every file of its commit is one whose checkout did finish
|
||||
/// (gclo's own, whose marker could not be cleared any more, or one a person
|
||||
/// completed with git): its tree is git's to update, and the ordinary pull does
|
||||
/// that without replacing anything that was changed. An index that somebody's git
|
||||
/// wrote over an unfinished tree (a "git reset", a "git add") does not put the
|
||||
/// missing files there, and that folder is completed like one without an index.
|
||||
/// Either way the pull that follows brings what origin has got since.
|
||||
/// </remarks>
|
||||
private static void FinishOrReleaseAPendingCheckout(Repository repo, CancellationToken ct)
|
||||
{
|
||||
if(repo.Index.Count == 0 || !HoldsItsWholeTree(repo))
|
||||
{
|
||||
CompletePendingCheckout(repo, ct);
|
||||
}
|
||||
else
|
||||
{
|
||||
MarkFinished(repo);
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/// <summary>
|
||||
/// True when every file of the current commit is in the folder (as a file, or as a
|
||||
/// link: the repository may have a symbolic link there). What the index says is
|
||||
/// not asked: it is whoever's git wrote it.
|
||||
/// </summary>
|
||||
private static bool HoldsItsWholeTree(Repository repo)
|
||||
{
|
||||
string workdir = WorkingDirectoryOf(repo);
|
||||
foreach((string originalPath, _, _) in CollectEffectiveEntries(repo.Head.Tip.Tree, s_noRecovery))
|
||||
{
|
||||
string full = Path.Combine(workdir, originalPath.Replace('/', Path.DirectorySeparatorChar));
|
||||
if(!(File.Exists(full) || IsLink(full)))
|
||||
{
|
||||
return false;
|
||||
}
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
|
||||
|
||||
/// <summary>The folder's tree is written: the marker and what goes with it are taken off.</summary>
|
||||
private static void MarkFinished(Repository repo)
|
||||
{
|
||||
repo.Config.Set(CheckoutPendingKey, false, ConfigurationLevel.Local);
|
||||
repo.Config.Unset(WrittenBeforeKey, ConfigurationLevel.Local);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/// <summary>The trees of the commits recorded under <see cref="WrittenBeforeKey"/> that are still in the repository.</summary>
|
||||
private static List<Tree> WrittenBefore(Repository repo)
|
||||
{
|
||||
string ids = repo.Config.Get<string>(WrittenBeforeKey, ConfigurationLevel.Local)?.Value ?? "";
|
||||
var trees = new List<Tree>();
|
||||
foreach(string id in ids.Split(',', StringSplitOptions.RemoveEmptyEntries | StringSplitOptions.TrimEntries))
|
||||
{
|
||||
if(ObjectId.TryParse(id, out ObjectId parsed) && repo.Lookup<Commit>(parsed) is { } commit)
|
||||
{
|
||||
trees.Add(commit.Tree);
|
||||
}
|
||||
}
|
||||
return trees;
|
||||
}
|
||||
|
||||
|
||||
|
||||
/// <summary>
|
||||
/// Records, before the branch of an unfinished checkout is moved, the commit it was
|
||||
/// on. The latest sixteen are kept: a folder that was cut off more often than that
|
||||
/// while origin kept moving has to be moved out of the way by hand.
|
||||
/// </summary>
|
||||
private static void RememberAsWrittenBefore(Repository repo, Commit commit)
|
||||
{
|
||||
const int Kept = 16;
|
||||
List<string> ids =
|
||||
[
|
||||
.. (repo.Config.Get<string>(WrittenBeforeKey, ConfigurationLevel.Local)?.Value ?? "")
|
||||
.Split(',', StringSplitOptions.RemoveEmptyEntries | StringSplitOptions.TrimEntries),
|
||||
];
|
||||
if(!ids.Contains(commit.Sha, StringComparer.Ordinal))
|
||||
{
|
||||
ids.Add(commit.Sha);
|
||||
repo.Config.Set(WrittenBeforeKey, string.Join(',', ids.TakeLast(Kept)), ConfigurationLevel.Local);
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/// <summary>
|
||||
/// Finishes a clone whose checkout never completed and that has no stored
|
||||
/// recovery: force-checks-out the commit the branch is on, re-validating its paths
|
||||
/// on Windows. If the paths are invalid, rethrows the typed failure — the repo must
|
||||
/// not silently report success with an empty or partial tree.
|
||||
/// </summary>
|
||||
/// <remarks>
|
||||
/// <para>
|
||||
/// Where origin's branch has only got ahead, the branch is not moved first: every
|
||||
/// attempt to finish writes the same commit, so what an earlier attempt left on
|
||||
/// disk can be told from what somebody put there since
|
||||
/// (<see cref="RefuseToWriteOverWhatIsNotOurs"/>, #174). What origin has got
|
||||
/// meanwhile is brought by the ordinary pull that follows, over a tree that then
|
||||
/// has an index.
|
||||
/// </para>
|
||||
/// <para>
|
||||
/// The branch has to move first in two cases, as it always did here: a tree
|
||||
/// Windows cannot write (the commit the branch is on, or origin's; the failure
|
||||
/// then names the paths of the newest tree, which is the one the recovery that
|
||||
/// follows will write), and a history origin has rewritten (there is nothing to
|
||||
/// fast-forward from). The commit the branch leaves is recorded then
|
||||
/// (<see cref="WrittenBeforeKey"/>), because its files may be on disk and are
|
||||
/// judged against it as well.
|
||||
/// </para>
|
||||
/// <para>
|
||||
/// A file somebody added to the index with git, and that the commit does not have,
|
||||
/// would be removed from the folder by the forced checkout: the completion is
|
||||
/// refused instead (<see cref="RefuseToRemoveWhatIsStaged"/>). A file of a commit
|
||||
/// the branch left that the new commit does not have is not looked for: it stays
|
||||
/// in the folder, as a file git does not track.
|
||||
/// </para>
|
||||
/// </remarks>
|
||||
private static void CompletePendingCheckout(Repository repo, CancellationToken ct)
|
||||
{
|
||||
AdvanceHeadToTrackedTip(repo);
|
||||
|
||||
Commit? tip = repo.Head.Tip;
|
||||
if(tip is not null)
|
||||
Commit head = repo.Head.Tip;
|
||||
if(MustMoveBeforeItIsWritten(repo, head))
|
||||
{
|
||||
RememberAsWrittenBefore(repo, head);
|
||||
AdvanceHeadToTrackedTip(repo);
|
||||
if(OperatingSystem.IsWindows())
|
||||
{
|
||||
IReadOnlyList<InvalidPathInfo> invalidPaths = WindowsPathValidator.Validate(tip.Tree);
|
||||
IReadOnlyList<InvalidPathInfo> invalidPaths = WindowsPathValidator.Validate(repo.Head.Tip.Tree);
|
||||
if(invalidPaths.Count > 0)
|
||||
{
|
||||
throw new InvalidRepositoryPathsException(invalidPaths);
|
||||
}
|
||||
}
|
||||
|
||||
ct.ThrowIfCancellationRequested();
|
||||
try
|
||||
{
|
||||
Commands.Checkout(repo, repo.Head, new CheckoutOptions
|
||||
{
|
||||
CheckoutModifiers = CheckoutModifiers.Force, // nothing was ever checked out
|
||||
CheckoutNotifyFlags = CheckoutNotifyFlags.Updated,
|
||||
OnCheckoutNotify = (_, _) => !ct.IsCancellationRequested,
|
||||
});
|
||||
}
|
||||
catch(UserCancelledException) when(ct.IsCancellationRequested)
|
||||
{
|
||||
throw new OperationCanceledException(ct);
|
||||
}
|
||||
}
|
||||
|
||||
repo.Config.Set(CheckoutPendingKey, false, ConfigurationLevel.Local);
|
||||
RefuseToRemoveWhatIsStaged(repo);
|
||||
RefuseToWriteOverWhatIsNotOurs(repo, repo.Head.Tip.Tree, aCutOffWriteIsOurs: true, WrittenBefore(repo));
|
||||
|
||||
ct.ThrowIfCancellationRequested();
|
||||
try
|
||||
{
|
||||
Commands.Checkout(repo, repo.Head, new CheckoutOptions
|
||||
{
|
||||
CheckoutModifiers = CheckoutModifiers.Force, // what is in the way is the unfinished checkout's own (checked above)
|
||||
CheckoutNotifyFlags = CheckoutNotifyFlags.Updated,
|
||||
OnCheckoutNotify = (_, _) => !ct.IsCancellationRequested,
|
||||
});
|
||||
}
|
||||
catch(UserCancelledException) when(ct.IsCancellationRequested)
|
||||
{
|
||||
throw new OperationCanceledException(ct);
|
||||
}
|
||||
|
||||
MarkFinished(repo);
|
||||
}
|
||||
|
||||
|
||||
|
||||
/// <summary>
|
||||
/// Before a forced checkout into a folder that has an index: refuses, with nothing
|
||||
/// changed, when the index names a file the commit does not have. A forced checkout
|
||||
/// removes such a file from the folder, and it is there because somebody added it
|
||||
/// with git (#174).
|
||||
/// </summary>
|
||||
private static void RefuseToRemoveWhatIsStaged(Repository repo)
|
||||
{
|
||||
Tree tree = repo.Head.Tip.Tree;
|
||||
IndexEntry? staged = repo.Index.FirstOrDefault(entry => tree[entry.Path] is null);
|
||||
if(staged is not null)
|
||||
{
|
||||
throw new InvalidOperationException(
|
||||
$"'{staged.Path}' has been added to the index of this folder with git, and the commit whose checkout gclo has to finish "
|
||||
+ "here does not have it: finishing would remove the file. Take it out of the index ('git reset'), or move the folder "
|
||||
+ "away, then sync again.");
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
/// <summary>
|
||||
/// True when an unfinished checkout cannot be completed at the commit its branch
|
||||
/// is on: Windows cannot write that commit's tree or the one origin has now, or
|
||||
/// origin's branch is not a continuation of it (origin rewrote its history, and
|
||||
/// by <see cref="TargetOf"/> nobody committed here). Histories that diverged
|
||||
/// because of commits made here are not such a case: the commit here is completed,
|
||||
/// and the ordinary pull says what is wrong.
|
||||
/// </summary>
|
||||
private static bool MustMoveBeforeItIsWritten(Repository repo, Commit head)
|
||||
{
|
||||
Commit target;
|
||||
try
|
||||
{
|
||||
target = TargetOf(repo);
|
||||
}
|
||||
catch(InvalidOperationException)
|
||||
{
|
||||
target = head;
|
||||
}
|
||||
|
||||
if(OperatingSystem.IsWindows()
|
||||
&& (WindowsPathValidator.Validate(head.Tree).Count > 0 || WindowsPathValidator.Validate(target.Tree).Count > 0))
|
||||
{
|
||||
return true;
|
||||
}
|
||||
|
||||
return !string.Equals(target.Sha, head.Sha, StringComparison.Ordinal)
|
||||
&& !string.Equals(repo.ObjectDatabase.FindMergeBase(head, target)?.Sha, head.Sha, StringComparison.Ordinal);
|
||||
}
|
||||
|
||||
|
||||
|
||||
@@ -19,12 +19,26 @@ public sealed class WslCloner : IWslCloner
|
||||
/// <summary>Environment variable the inline credential helper reads the token from.</summary>
|
||||
public const string TokenVariable = "GCLO_GIT_TOKEN";
|
||||
|
||||
/// <summary>The folder under <c>~/gclo</c> in which clones are made before they are moved into place.</summary>
|
||||
internal const string TemporaryFolderName = ".gclo-tmp";
|
||||
|
||||
/// <summary>
|
||||
/// The clone script. $1 organization, $2 repository, $3 clone URL, $4 the URL's
|
||||
/// host (with a non-default port). Prints the distribution name and the destination
|
||||
/// as its last two lines on success. A folder that already is a clone of the same
|
||||
/// URL is fast-forwarded instead of failing, so the verb is safe to use twice; a
|
||||
/// folder that is a clone of anything else is refused. The credential helper is
|
||||
/// folder that is a clone of anything else is refused, and so is anything else at
|
||||
/// the destination but an empty folder. A new clone is made in a folder of gclo's
|
||||
/// own (<c>~/gclo/.gclo-tmp</c>) and moved into place when it is complete, so one
|
||||
/// that is cut off never leaves a folder that the next attempt would take for a
|
||||
/// clone and try to update. The temporary folder is removed when the script ends
|
||||
/// without having moved it, also when it is ended by a signal; what a killed
|
||||
/// script left there is removed by a later run once it is a day old (not sooner:
|
||||
/// another gclo may be cloning right now). A destination that appears while the
|
||||
/// clone runs is not moved into: the clone would end up inside it (checked before
|
||||
/// the move, and the move itself is <c>mv -T</c>, which renames onto the name and
|
||||
/// never into a folder of that name).
|
||||
/// The credential helper is
|
||||
/// scoped to $4, so git (and anything it starts, such as an LFS filter pointed at
|
||||
/// another server by the repository's own configuration) gets the token for that
|
||||
/// host only (#139).
|
||||
@@ -41,8 +55,25 @@ public sealed class WslCloner : IWslCloner
|
||||
+ " fi\n"
|
||||
+ " git -C \"$dest\" -c credential.helper= -c \"credential.https://$host.helper=$helper\" pull --ff-only --quiet\n"
|
||||
+ "else\n"
|
||||
+ " mkdir -p \"$HOME/gclo/$org\"\n"
|
||||
+ " git -c credential.helper= -c \"credential.https://$host.helper=$helper\" clone --quiet -- \"$url\" \"$dest\"\n"
|
||||
+ " if [ -e \"$dest\" ] && ! rmdir \"$dest\" 2>/dev/null; then\n"
|
||||
+ " echo \"$dest exists but is not a repository; move or delete it, then try again\" >&2\n"
|
||||
+ " exit 1\n"
|
||||
+ " fi\n"
|
||||
+ " mkdir -p \"$HOME/gclo/$org\" \"$HOME/gclo/.gclo-tmp\"\n"
|
||||
+ " find \"$HOME/gclo/.gclo-tmp\" -mindepth 1 -maxdepth 1 -mmin +1440 -exec rm -rf {} + 2>/dev/null || true\n"
|
||||
+ " tmp=\"$HOME/gclo/.gclo-tmp/$org.$repo.$$\"\n"
|
||||
+ " rm -rf \"$tmp\"\n"
|
||||
+ " trap 'rm -rf \"$tmp\"' EXIT\n"
|
||||
+ " trap 'exit 129' HUP\n"
|
||||
+ " trap 'exit 130' INT\n"
|
||||
+ " trap 'exit 143' TERM\n"
|
||||
+ " git -c credential.helper= -c \"credential.https://$host.helper=$helper\" clone --quiet -- \"$url\" \"$tmp\"\n"
|
||||
+ " if [ -e \"$dest\" ]; then\n"
|
||||
+ " echo \"$dest appeared while the repository was being cloned (another gclo?); nothing was put there\" >&2\n"
|
||||
+ " exit 1\n"
|
||||
+ " fi\n"
|
||||
+ " mv -T \"$tmp\" \"$dest\"\n"
|
||||
+ " trap - EXIT HUP INT TERM\n"
|
||||
+ "fi\n"
|
||||
+ "printf '%s\\n%s\\n' \"$WSL_DISTRO_NAME\" \"$dest\"\n";
|
||||
|
||||
@@ -123,6 +154,14 @@ public sealed class WslCloner : IWslCloner
|
||||
throw new WslCloneException($"Clone in WSL refused: '{name}' cannot be used as a folder name ({reason}).");
|
||||
}
|
||||
}
|
||||
if(string.Equals(organization, TemporaryFolderName, StringComparison.OrdinalIgnoreCase))
|
||||
{
|
||||
////////////////////////////////////////////////////////////////////////
|
||||
// ~/gclo/.gclo-tmp is where unfinished clones are made, and old ones //
|
||||
// there are removed. //
|
||||
////////////////////////////////////////////////////////////////////////
|
||||
throw new WslCloneException($"Clone in WSL refused: '{organization}' is the name of gclo's own temporary folder there.");
|
||||
}
|
||||
|
||||
string host = RemoteUrl.HttpsAuthority(url)
|
||||
?? throw new WslCloneException("Clone in WSL refused: the clone URL is not an HTTPS URL.");
|
||||
|
||||
Reference in new issue
Block a user