-
Notifications
You must be signed in to change notification settings - Fork 461
Streamline mount/unmount for dehydrating folders #1894
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Conversation
In microsoft#1890 I demonstrated that it's possible to dehydrate folders without unmounting at all. Unfortunately that requires deleting all the placeholders and hydrated files which is too slow for dehydrating anything substantial. The current implementation avoids this by moving instead of deleting (which has the additional benefit of providing a backup) but ProjFS doesn't support moving or renaming folders, so we can't do that while mounted. This pull request takes a different approach to reducing the overhead of unmounting for dehydration. Instead of unmounting, moving, and remounting from the dehydrate verb, those steps are moved into the mount process under its dehydrate message handler. The mount process only disposes and recreates the components required for virtualization, avoiding several costly steps (eg authentication with Azure DevOps, verification of the cache server, verification of ProjFS installation). For the repo I work in, dehydrating a top-level directory is reduced from 33 seconds to 11 seconds with this change. Specific changes: * Backup of non-src folders (.git, .gvfs) is added to dehydrate folders. Previously it was only done for full dehydrate. * Unmount, move/backup of folders, and mount are moved from DehydrateVerb to InProcessMount. To support this, the DehydrateFolders message has the backup folder added to its fields. * The core methods of Mount and Unmount have a parameter added to skip disposing (on unmount) and initialization (on mount) of certain components which are ok to leave alive during the temporary unmount. * Ownership of GVFSContext disposal fixed - FileSystemCallbacks was disposing it despite not owning it. * Missing disposal of a file stream in BackgroundFileSystemTaskRunner is fixed. * WindowsFileSystemVirtualizer.DehydrateFolder will now delete a tombstone file for the directory if present. This allows us to support fixing a directory that the user manually deleted while mounted (perhaps in a misguided attempt to dehydrate it), though that would typically require running 'gvfs dehydrate --no-status' to skip verifying that the working directory matches the index. * '--no-status' is now supported with '--folders'
| if (!this.TryBackupNonSrcFiles(tracer, enlistment, backupRoot)) | ||
| { | ||
| this.Output.WriteLine(); | ||
| this.WriteMessage(tracer, "ERROR: Backup failed. "); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe similar to other messages, something like "Failed to dehydrate non-sources files".
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I tested out a failure here and found these lines are redundant - TryBackupNonSrcFiles' output to console is more specific, so I've removed this error message.
ShiningMassXAcc
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not an expert on this codebase, but LGTM
In #1890 I demonstrated that it's possible to dehydrate folders without unmounting at all. Unfortunately that requires deleting all the placeholders and hydrated files which is too slow for dehydrating anything substantial. The current implementation avoids this by moving instead of deleting (which has the additional benefit of providing a backup) but ProjFS doesn't support moving or renaming folders, so we can't do that while mounted.
This pull request takes a different approach to reducing the overhead of unmounting for dehydration. Instead of unmounting, moving, and remounting from the dehydrate verb, those steps are moved into the mount process under its dehydrate message handler. The mount process only disposes and recreates the components required for virtualization, avoiding several costly steps (eg authentication with Azure DevOps, verification of the cache server, verification of ProjFS installation). For the repo I work in, dehydrating a top-level directory is reduced from 33 seconds to 11 seconds with this change.
Specific changes: