Best practice to continue mv











up vote
13
down vote

favorite
2












I used the terminal to copy files from one drive to another.



sudo mv -vi /location/to/drive1/ /location/to/drive2/


However that suddenly stopped, while some hours into it, and without an error, after creating a directory.



My own solution to that is often a mix of hashing and comparing which is mostly a time consuming mess as I now have to recover from an intermediate copy without really knowing which files are missing (written as very long one-liner for zsh — note that this script doesn't work in bash as written):



source_directory="/path/to/source_directory/";
target_directory="/path/to/target_directory/";
while read hash_and_file; do {
echo "${hash_and_file}" | read hash file;
echo "${file}" | sed "s/^/${source_directory}/g" | read copy_from;
echo "${copy_from}" | sed "s/${source_directory}/${target_directory}/g" | read copy_to;
mv -v "${copy_from}" "${copy_to}" | tee -a log;
rm -v "${copy_from}" | tee -a log; };
done <<<$(
comm -23 <( find ${source_directory} -type f -exec sha256sum "{}" ; |
sed "s: ${source_directory}: :g" | sort;
) <( find ${target_directory} -type f -exec sha256sum "{}" ; |
sed "s: ${target_directory}: :g" | sort; ) )


This is error prone if the name target directory or source_directory are part of the path, and delete files if they have not been moved because they were marked as duplicates. Also it does not source directory in the end.



Is there a best practice how to recover from interrupted mv?










share|improve this question
























  • I wrote a similar script, which uses cmp instead of hashing. It has dependencies, and the same issues with while read that Gilles mentioned. It is also slow and verbose. But it frees up disk-space sooner than the rsync method, because files are (re)moved from the source as it runs. It may serve as inspiration for the brave.
    – joeytwiddle
    Dec 1 at 14:46






  • 3




    @joeytwiddle rsync offers --delete-during receiver deletes during the transfer and also several other useful alternatives: --delete --delete-before --delete-delay --delete-after --delete-excluded. So, yes, rsync is the best alternative,
    – Isaac
    Dec 1 at 19:31










  • I must be missing something. Why doesn't just repeating the same mv command work? Perhaps with * appended to source path if the original source was a directory.
    – jpa
    Dec 2 at 11:04












  • @isaac No, I'm afraid rsync --delete* would be a disaster! It will remove things from dest which are not currently in src, so all files which were successfully moved in the previous attempt will now be deleted! You were probably thinking of rsync --remove-source-files which I agree would be a good alternative. (more1, more2)
    – joeytwiddle
    Dec 3 at 3:29












  • @joeytwiddle No, rsync --delete will only remove other files that are not part of the source. From [man rsync]()*delete extraneous files from dest dirs*. Understand what extraneous means: Not being synced. And yes, rsync also provides a way to remove source files after they have been correctly transmitted.
    – Isaac
    Dec 3 at 4:53















up vote
13
down vote

favorite
2












I used the terminal to copy files from one drive to another.



sudo mv -vi /location/to/drive1/ /location/to/drive2/


However that suddenly stopped, while some hours into it, and without an error, after creating a directory.



My own solution to that is often a mix of hashing and comparing which is mostly a time consuming mess as I now have to recover from an intermediate copy without really knowing which files are missing (written as very long one-liner for zsh — note that this script doesn't work in bash as written):



source_directory="/path/to/source_directory/";
target_directory="/path/to/target_directory/";
while read hash_and_file; do {
echo "${hash_and_file}" | read hash file;
echo "${file}" | sed "s/^/${source_directory}/g" | read copy_from;
echo "${copy_from}" | sed "s/${source_directory}/${target_directory}/g" | read copy_to;
mv -v "${copy_from}" "${copy_to}" | tee -a log;
rm -v "${copy_from}" | tee -a log; };
done <<<$(
comm -23 <( find ${source_directory} -type f -exec sha256sum "{}" ; |
sed "s: ${source_directory}: :g" | sort;
) <( find ${target_directory} -type f -exec sha256sum "{}" ; |
sed "s: ${target_directory}: :g" | sort; ) )


This is error prone if the name target directory or source_directory are part of the path, and delete files if they have not been moved because they were marked as duplicates. Also it does not source directory in the end.



Is there a best practice how to recover from interrupted mv?










share|improve this question
























  • I wrote a similar script, which uses cmp instead of hashing. It has dependencies, and the same issues with while read that Gilles mentioned. It is also slow and verbose. But it frees up disk-space sooner than the rsync method, because files are (re)moved from the source as it runs. It may serve as inspiration for the brave.
    – joeytwiddle
    Dec 1 at 14:46






  • 3




    @joeytwiddle rsync offers --delete-during receiver deletes during the transfer and also several other useful alternatives: --delete --delete-before --delete-delay --delete-after --delete-excluded. So, yes, rsync is the best alternative,
    – Isaac
    Dec 1 at 19:31










  • I must be missing something. Why doesn't just repeating the same mv command work? Perhaps with * appended to source path if the original source was a directory.
    – jpa
    Dec 2 at 11:04












  • @isaac No, I'm afraid rsync --delete* would be a disaster! It will remove things from dest which are not currently in src, so all files which were successfully moved in the previous attempt will now be deleted! You were probably thinking of rsync --remove-source-files which I agree would be a good alternative. (more1, more2)
    – joeytwiddle
    Dec 3 at 3:29












  • @joeytwiddle No, rsync --delete will only remove other files that are not part of the source. From [man rsync]()*delete extraneous files from dest dirs*. Understand what extraneous means: Not being synced. And yes, rsync also provides a way to remove source files after they have been correctly transmitted.
    – Isaac
    Dec 3 at 4:53













up vote
13
down vote

favorite
2









up vote
13
down vote

favorite
2






2





I used the terminal to copy files from one drive to another.



sudo mv -vi /location/to/drive1/ /location/to/drive2/


However that suddenly stopped, while some hours into it, and without an error, after creating a directory.



My own solution to that is often a mix of hashing and comparing which is mostly a time consuming mess as I now have to recover from an intermediate copy without really knowing which files are missing (written as very long one-liner for zsh — note that this script doesn't work in bash as written):



source_directory="/path/to/source_directory/";
target_directory="/path/to/target_directory/";
while read hash_and_file; do {
echo "${hash_and_file}" | read hash file;
echo "${file}" | sed "s/^/${source_directory}/g" | read copy_from;
echo "${copy_from}" | sed "s/${source_directory}/${target_directory}/g" | read copy_to;
mv -v "${copy_from}" "${copy_to}" | tee -a log;
rm -v "${copy_from}" | tee -a log; };
done <<<$(
comm -23 <( find ${source_directory} -type f -exec sha256sum "{}" ; |
sed "s: ${source_directory}: :g" | sort;
) <( find ${target_directory} -type f -exec sha256sum "{}" ; |
sed "s: ${target_directory}: :g" | sort; ) )


This is error prone if the name target directory or source_directory are part of the path, and delete files if they have not been moved because they were marked as duplicates. Also it does not source directory in the end.



Is there a best practice how to recover from interrupted mv?










share|improve this question















I used the terminal to copy files from one drive to another.



sudo mv -vi /location/to/drive1/ /location/to/drive2/


However that suddenly stopped, while some hours into it, and without an error, after creating a directory.



My own solution to that is often a mix of hashing and comparing which is mostly a time consuming mess as I now have to recover from an intermediate copy without really knowing which files are missing (written as very long one-liner for zsh — note that this script doesn't work in bash as written):



source_directory="/path/to/source_directory/";
target_directory="/path/to/target_directory/";
while read hash_and_file; do {
echo "${hash_and_file}" | read hash file;
echo "${file}" | sed "s/^/${source_directory}/g" | read copy_from;
echo "${copy_from}" | sed "s/${source_directory}/${target_directory}/g" | read copy_to;
mv -v "${copy_from}" "${copy_to}" | tee -a log;
rm -v "${copy_from}" | tee -a log; };
done <<<$(
comm -23 <( find ${source_directory} -type f -exec sha256sum "{}" ; |
sed "s: ${source_directory}: :g" | sort;
) <( find ${target_directory} -type f -exec sha256sum "{}" ; |
sed "s: ${target_directory}: :g" | sort; ) )


This is error prone if the name target directory or source_directory are part of the path, and delete files if they have not been moved because they were marked as duplicates. Also it does not source directory in the end.



Is there a best practice how to recover from interrupted mv?







data-recovery mv






share|improve this question















share|improve this question













share|improve this question




share|improve this question








edited Dec 1 at 9:50

























asked Dec 1 at 9:17









What

937




937












  • I wrote a similar script, which uses cmp instead of hashing. It has dependencies, and the same issues with while read that Gilles mentioned. It is also slow and verbose. But it frees up disk-space sooner than the rsync method, because files are (re)moved from the source as it runs. It may serve as inspiration for the brave.
    – joeytwiddle
    Dec 1 at 14:46






  • 3




    @joeytwiddle rsync offers --delete-during receiver deletes during the transfer and also several other useful alternatives: --delete --delete-before --delete-delay --delete-after --delete-excluded. So, yes, rsync is the best alternative,
    – Isaac
    Dec 1 at 19:31










  • I must be missing something. Why doesn't just repeating the same mv command work? Perhaps with * appended to source path if the original source was a directory.
    – jpa
    Dec 2 at 11:04












  • @isaac No, I'm afraid rsync --delete* would be a disaster! It will remove things from dest which are not currently in src, so all files which were successfully moved in the previous attempt will now be deleted! You were probably thinking of rsync --remove-source-files which I agree would be a good alternative. (more1, more2)
    – joeytwiddle
    Dec 3 at 3:29












  • @joeytwiddle No, rsync --delete will only remove other files that are not part of the source. From [man rsync]()*delete extraneous files from dest dirs*. Understand what extraneous means: Not being synced. And yes, rsync also provides a way to remove source files after they have been correctly transmitted.
    – Isaac
    Dec 3 at 4:53


















  • I wrote a similar script, which uses cmp instead of hashing. It has dependencies, and the same issues with while read that Gilles mentioned. It is also slow and verbose. But it frees up disk-space sooner than the rsync method, because files are (re)moved from the source as it runs. It may serve as inspiration for the brave.
    – joeytwiddle
    Dec 1 at 14:46






  • 3




    @joeytwiddle rsync offers --delete-during receiver deletes during the transfer and also several other useful alternatives: --delete --delete-before --delete-delay --delete-after --delete-excluded. So, yes, rsync is the best alternative,
    – Isaac
    Dec 1 at 19:31










  • I must be missing something. Why doesn't just repeating the same mv command work? Perhaps with * appended to source path if the original source was a directory.
    – jpa
    Dec 2 at 11:04












  • @isaac No, I'm afraid rsync --delete* would be a disaster! It will remove things from dest which are not currently in src, so all files which were successfully moved in the previous attempt will now be deleted! You were probably thinking of rsync --remove-source-files which I agree would be a good alternative. (more1, more2)
    – joeytwiddle
    Dec 3 at 3:29












  • @joeytwiddle No, rsync --delete will only remove other files that are not part of the source. From [man rsync]()*delete extraneous files from dest dirs*. Understand what extraneous means: Not being synced. And yes, rsync also provides a way to remove source files after they have been correctly transmitted.
    – Isaac
    Dec 3 at 4:53
















I wrote a similar script, which uses cmp instead of hashing. It has dependencies, and the same issues with while read that Gilles mentioned. It is also slow and verbose. But it frees up disk-space sooner than the rsync method, because files are (re)moved from the source as it runs. It may serve as inspiration for the brave.
– joeytwiddle
Dec 1 at 14:46




I wrote a similar script, which uses cmp instead of hashing. It has dependencies, and the same issues with while read that Gilles mentioned. It is also slow and verbose. But it frees up disk-space sooner than the rsync method, because files are (re)moved from the source as it runs. It may serve as inspiration for the brave.
– joeytwiddle
Dec 1 at 14:46




3




3




@joeytwiddle rsync offers --delete-during receiver deletes during the transfer and also several other useful alternatives: --delete --delete-before --delete-delay --delete-after --delete-excluded. So, yes, rsync is the best alternative,
– Isaac
Dec 1 at 19:31




@joeytwiddle rsync offers --delete-during receiver deletes during the transfer and also several other useful alternatives: --delete --delete-before --delete-delay --delete-after --delete-excluded. So, yes, rsync is the best alternative,
– Isaac
Dec 1 at 19:31












I must be missing something. Why doesn't just repeating the same mv command work? Perhaps with * appended to source path if the original source was a directory.
– jpa
Dec 2 at 11:04






I must be missing something. Why doesn't just repeating the same mv command work? Perhaps with * appended to source path if the original source was a directory.
– jpa
Dec 2 at 11:04














@isaac No, I'm afraid rsync --delete* would be a disaster! It will remove things from dest which are not currently in src, so all files which were successfully moved in the previous attempt will now be deleted! You were probably thinking of rsync --remove-source-files which I agree would be a good alternative. (more1, more2)
– joeytwiddle
Dec 3 at 3:29






@isaac No, I'm afraid rsync --delete* would be a disaster! It will remove things from dest which are not currently in src, so all files which were successfully moved in the previous attempt will now be deleted! You were probably thinking of rsync --remove-source-files which I agree would be a good alternative. (more1, more2)
– joeytwiddle
Dec 3 at 3:29














@joeytwiddle No, rsync --delete will only remove other files that are not part of the source. From [man rsync]()*delete extraneous files from dest dirs*. Understand what extraneous means: Not being synced. And yes, rsync also provides a way to remove source files after they have been correctly transmitted.
– Isaac
Dec 3 at 4:53




@joeytwiddle No, rsync --delete will only remove other files that are not part of the source. From [man rsync]()*delete extraneous files from dest dirs*. Understand what extraneous means: Not being synced. And yes, rsync also provides a way to remove source files after they have been correctly transmitted.
– Isaac
Dec 3 at 4:53










1 Answer
1






active

oldest

votes

















up vote
44
down vote



accepted










Forget about trying to reinvent rsync, and use rsync.



sudo rsync -av /location/to/drive1/ /location/to/drive2/


Make sure you use a trailing slash on the source, otherwise it would copy to /location/to/drive2/drive1.



Double-check that the command succeeded, then run rm -rf /location/to/drive1/.



The command above will overwrite any preexisting file from drive2. If you want to prompt the user to skip files that already existed in drive2, as with mv -i, it's more complicated, because you now need to distinguish files that have already been copied and files that haven't. You can pass the --ignore-existing option to rsync to skip files that already exist on the destination regardless of their content. Note that if the original mv was interrupted in the middle of creating a file, this file will remain in its half-copied state (whereas a bare rsync -a would properly finish copying it).



If you want to reproduce the exact behavior of mv -i, including the prompting, it could be done, but it's a lot more complicated.



Note that your one-giant-liner is very fragile. If there are file names containing backslashes or newlines, they may not be copied properly or they may even trick your script into removing arbitrary files. So do not use the code in the question unless you're sure that you can trust the file names not to contain backslashes or newlines.



For future reference, I recommend to never use mv for large cross-drive moves, precisely because it's hard to control what happens if it gets interrupted. Use rsync to do the copying, and then remove the original.






share|improve this answer





















  • What promises does rsync make that mv does not?
    – What
    Dec 1 at 9:48






  • 4




    well, for example rsync does what you're trying to do, while mv does not. Also: copying between different machines; compression for transfer; skipping files existing at the destination based on timestamp- or hash-based equality; configurable handling of ownership, permissions, links and special files; etc. linux.die.net/man/1/rsync
    – Silly Freak
    Dec 1 at 11:00








  • 1




    @SillyFreak should I conclude from that, that I should always use rsync instead of mv, not only as Gilles said for cross-drive, but any operation, as the boundary of "too large" is relatively subjective and if it comes to a problem it would have been solved by rsync anyway?
    – What
    Dec 1 at 11:07








  • 8




    well, when I'm moving files or directories inside one partition, I usually use mv (or the file manager) because it's only moving a reference to the file/directory. If I need to do actual data transfer, then I use rsync if one of the following is true: 1) I'm moving more files than I can check for correct transfer at a glance; 2) I anticipate that I'll need to keep files in sync; 3) I expect the transfer could be interrupted. My point is, for the use case you're presenting in the question, rsync is simply the right tool, and mv or cp are not.
    – Silly Freak
    Dec 1 at 11:22






  • 4




    I would advise to always run any rsync command with -v and —dry-run first to confirm exactly what it’s going to do.
    – Darren
    Dec 1 at 18:49











Your Answer








StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "106"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});

function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});


}
});














draft saved

draft discarded


















StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f485321%2fbest-practice-to-continue-mv%23new-answer', 'question_page');
}
);

Post as a guest















Required, but never shown

























1 Answer
1






active

oldest

votes








1 Answer
1






active

oldest

votes









active

oldest

votes






active

oldest

votes








up vote
44
down vote



accepted










Forget about trying to reinvent rsync, and use rsync.



sudo rsync -av /location/to/drive1/ /location/to/drive2/


Make sure you use a trailing slash on the source, otherwise it would copy to /location/to/drive2/drive1.



Double-check that the command succeeded, then run rm -rf /location/to/drive1/.



The command above will overwrite any preexisting file from drive2. If you want to prompt the user to skip files that already existed in drive2, as with mv -i, it's more complicated, because you now need to distinguish files that have already been copied and files that haven't. You can pass the --ignore-existing option to rsync to skip files that already exist on the destination regardless of their content. Note that if the original mv was interrupted in the middle of creating a file, this file will remain in its half-copied state (whereas a bare rsync -a would properly finish copying it).



If you want to reproduce the exact behavior of mv -i, including the prompting, it could be done, but it's a lot more complicated.



Note that your one-giant-liner is very fragile. If there are file names containing backslashes or newlines, they may not be copied properly or they may even trick your script into removing arbitrary files. So do not use the code in the question unless you're sure that you can trust the file names not to contain backslashes or newlines.



For future reference, I recommend to never use mv for large cross-drive moves, precisely because it's hard to control what happens if it gets interrupted. Use rsync to do the copying, and then remove the original.






share|improve this answer





















  • What promises does rsync make that mv does not?
    – What
    Dec 1 at 9:48






  • 4




    well, for example rsync does what you're trying to do, while mv does not. Also: copying between different machines; compression for transfer; skipping files existing at the destination based on timestamp- or hash-based equality; configurable handling of ownership, permissions, links and special files; etc. linux.die.net/man/1/rsync
    – Silly Freak
    Dec 1 at 11:00








  • 1




    @SillyFreak should I conclude from that, that I should always use rsync instead of mv, not only as Gilles said for cross-drive, but any operation, as the boundary of "too large" is relatively subjective and if it comes to a problem it would have been solved by rsync anyway?
    – What
    Dec 1 at 11:07








  • 8




    well, when I'm moving files or directories inside one partition, I usually use mv (or the file manager) because it's only moving a reference to the file/directory. If I need to do actual data transfer, then I use rsync if one of the following is true: 1) I'm moving more files than I can check for correct transfer at a glance; 2) I anticipate that I'll need to keep files in sync; 3) I expect the transfer could be interrupted. My point is, for the use case you're presenting in the question, rsync is simply the right tool, and mv or cp are not.
    – Silly Freak
    Dec 1 at 11:22






  • 4




    I would advise to always run any rsync command with -v and —dry-run first to confirm exactly what it’s going to do.
    – Darren
    Dec 1 at 18:49















up vote
44
down vote



accepted










Forget about trying to reinvent rsync, and use rsync.



sudo rsync -av /location/to/drive1/ /location/to/drive2/


Make sure you use a trailing slash on the source, otherwise it would copy to /location/to/drive2/drive1.



Double-check that the command succeeded, then run rm -rf /location/to/drive1/.



The command above will overwrite any preexisting file from drive2. If you want to prompt the user to skip files that already existed in drive2, as with mv -i, it's more complicated, because you now need to distinguish files that have already been copied and files that haven't. You can pass the --ignore-existing option to rsync to skip files that already exist on the destination regardless of their content. Note that if the original mv was interrupted in the middle of creating a file, this file will remain in its half-copied state (whereas a bare rsync -a would properly finish copying it).



If you want to reproduce the exact behavior of mv -i, including the prompting, it could be done, but it's a lot more complicated.



Note that your one-giant-liner is very fragile. If there are file names containing backslashes or newlines, they may not be copied properly or they may even trick your script into removing arbitrary files. So do not use the code in the question unless you're sure that you can trust the file names not to contain backslashes or newlines.



For future reference, I recommend to never use mv for large cross-drive moves, precisely because it's hard to control what happens if it gets interrupted. Use rsync to do the copying, and then remove the original.






share|improve this answer





















  • What promises does rsync make that mv does not?
    – What
    Dec 1 at 9:48






  • 4




    well, for example rsync does what you're trying to do, while mv does not. Also: copying between different machines; compression for transfer; skipping files existing at the destination based on timestamp- or hash-based equality; configurable handling of ownership, permissions, links and special files; etc. linux.die.net/man/1/rsync
    – Silly Freak
    Dec 1 at 11:00








  • 1




    @SillyFreak should I conclude from that, that I should always use rsync instead of mv, not only as Gilles said for cross-drive, but any operation, as the boundary of "too large" is relatively subjective and if it comes to a problem it would have been solved by rsync anyway?
    – What
    Dec 1 at 11:07








  • 8




    well, when I'm moving files or directories inside one partition, I usually use mv (or the file manager) because it's only moving a reference to the file/directory. If I need to do actual data transfer, then I use rsync if one of the following is true: 1) I'm moving more files than I can check for correct transfer at a glance; 2) I anticipate that I'll need to keep files in sync; 3) I expect the transfer could be interrupted. My point is, for the use case you're presenting in the question, rsync is simply the right tool, and mv or cp are not.
    – Silly Freak
    Dec 1 at 11:22






  • 4




    I would advise to always run any rsync command with -v and —dry-run first to confirm exactly what it’s going to do.
    – Darren
    Dec 1 at 18:49













up vote
44
down vote



accepted







up vote
44
down vote



accepted






Forget about trying to reinvent rsync, and use rsync.



sudo rsync -av /location/to/drive1/ /location/to/drive2/


Make sure you use a trailing slash on the source, otherwise it would copy to /location/to/drive2/drive1.



Double-check that the command succeeded, then run rm -rf /location/to/drive1/.



The command above will overwrite any preexisting file from drive2. If you want to prompt the user to skip files that already existed in drive2, as with mv -i, it's more complicated, because you now need to distinguish files that have already been copied and files that haven't. You can pass the --ignore-existing option to rsync to skip files that already exist on the destination regardless of their content. Note that if the original mv was interrupted in the middle of creating a file, this file will remain in its half-copied state (whereas a bare rsync -a would properly finish copying it).



If you want to reproduce the exact behavior of mv -i, including the prompting, it could be done, but it's a lot more complicated.



Note that your one-giant-liner is very fragile. If there are file names containing backslashes or newlines, they may not be copied properly or they may even trick your script into removing arbitrary files. So do not use the code in the question unless you're sure that you can trust the file names not to contain backslashes or newlines.



For future reference, I recommend to never use mv for large cross-drive moves, precisely because it's hard to control what happens if it gets interrupted. Use rsync to do the copying, and then remove the original.






share|improve this answer












Forget about trying to reinvent rsync, and use rsync.



sudo rsync -av /location/to/drive1/ /location/to/drive2/


Make sure you use a trailing slash on the source, otherwise it would copy to /location/to/drive2/drive1.



Double-check that the command succeeded, then run rm -rf /location/to/drive1/.



The command above will overwrite any preexisting file from drive2. If you want to prompt the user to skip files that already existed in drive2, as with mv -i, it's more complicated, because you now need to distinguish files that have already been copied and files that haven't. You can pass the --ignore-existing option to rsync to skip files that already exist on the destination regardless of their content. Note that if the original mv was interrupted in the middle of creating a file, this file will remain in its half-copied state (whereas a bare rsync -a would properly finish copying it).



If you want to reproduce the exact behavior of mv -i, including the prompting, it could be done, but it's a lot more complicated.



Note that your one-giant-liner is very fragile. If there are file names containing backslashes or newlines, they may not be copied properly or they may even trick your script into removing arbitrary files. So do not use the code in the question unless you're sure that you can trust the file names not to contain backslashes or newlines.



For future reference, I recommend to never use mv for large cross-drive moves, precisely because it's hard to control what happens if it gets interrupted. Use rsync to do the copying, and then remove the original.







share|improve this answer












share|improve this answer



share|improve this answer










answered Dec 1 at 9:38









Gilles

526k12710511578




526k12710511578












  • What promises does rsync make that mv does not?
    – What
    Dec 1 at 9:48






  • 4




    well, for example rsync does what you're trying to do, while mv does not. Also: copying between different machines; compression for transfer; skipping files existing at the destination based on timestamp- or hash-based equality; configurable handling of ownership, permissions, links and special files; etc. linux.die.net/man/1/rsync
    – Silly Freak
    Dec 1 at 11:00








  • 1




    @SillyFreak should I conclude from that, that I should always use rsync instead of mv, not only as Gilles said for cross-drive, but any operation, as the boundary of "too large" is relatively subjective and if it comes to a problem it would have been solved by rsync anyway?
    – What
    Dec 1 at 11:07








  • 8




    well, when I'm moving files or directories inside one partition, I usually use mv (or the file manager) because it's only moving a reference to the file/directory. If I need to do actual data transfer, then I use rsync if one of the following is true: 1) I'm moving more files than I can check for correct transfer at a glance; 2) I anticipate that I'll need to keep files in sync; 3) I expect the transfer could be interrupted. My point is, for the use case you're presenting in the question, rsync is simply the right tool, and mv or cp are not.
    – Silly Freak
    Dec 1 at 11:22






  • 4




    I would advise to always run any rsync command with -v and —dry-run first to confirm exactly what it’s going to do.
    – Darren
    Dec 1 at 18:49


















  • What promises does rsync make that mv does not?
    – What
    Dec 1 at 9:48






  • 4




    well, for example rsync does what you're trying to do, while mv does not. Also: copying between different machines; compression for transfer; skipping files existing at the destination based on timestamp- or hash-based equality; configurable handling of ownership, permissions, links and special files; etc. linux.die.net/man/1/rsync
    – Silly Freak
    Dec 1 at 11:00








  • 1




    @SillyFreak should I conclude from that, that I should always use rsync instead of mv, not only as Gilles said for cross-drive, but any operation, as the boundary of "too large" is relatively subjective and if it comes to a problem it would have been solved by rsync anyway?
    – What
    Dec 1 at 11:07








  • 8




    well, when I'm moving files or directories inside one partition, I usually use mv (or the file manager) because it's only moving a reference to the file/directory. If I need to do actual data transfer, then I use rsync if one of the following is true: 1) I'm moving more files than I can check for correct transfer at a glance; 2) I anticipate that I'll need to keep files in sync; 3) I expect the transfer could be interrupted. My point is, for the use case you're presenting in the question, rsync is simply the right tool, and mv or cp are not.
    – Silly Freak
    Dec 1 at 11:22






  • 4




    I would advise to always run any rsync command with -v and —dry-run first to confirm exactly what it’s going to do.
    – Darren
    Dec 1 at 18:49
















What promises does rsync make that mv does not?
– What
Dec 1 at 9:48




What promises does rsync make that mv does not?
– What
Dec 1 at 9:48




4




4




well, for example rsync does what you're trying to do, while mv does not. Also: copying between different machines; compression for transfer; skipping files existing at the destination based on timestamp- or hash-based equality; configurable handling of ownership, permissions, links and special files; etc. linux.die.net/man/1/rsync
– Silly Freak
Dec 1 at 11:00






well, for example rsync does what you're trying to do, while mv does not. Also: copying between different machines; compression for transfer; skipping files existing at the destination based on timestamp- or hash-based equality; configurable handling of ownership, permissions, links and special files; etc. linux.die.net/man/1/rsync
– Silly Freak
Dec 1 at 11:00






1




1




@SillyFreak should I conclude from that, that I should always use rsync instead of mv, not only as Gilles said for cross-drive, but any operation, as the boundary of "too large" is relatively subjective and if it comes to a problem it would have been solved by rsync anyway?
– What
Dec 1 at 11:07






@SillyFreak should I conclude from that, that I should always use rsync instead of mv, not only as Gilles said for cross-drive, but any operation, as the boundary of "too large" is relatively subjective and if it comes to a problem it would have been solved by rsync anyway?
– What
Dec 1 at 11:07






8




8




well, when I'm moving files or directories inside one partition, I usually use mv (or the file manager) because it's only moving a reference to the file/directory. If I need to do actual data transfer, then I use rsync if one of the following is true: 1) I'm moving more files than I can check for correct transfer at a glance; 2) I anticipate that I'll need to keep files in sync; 3) I expect the transfer could be interrupted. My point is, for the use case you're presenting in the question, rsync is simply the right tool, and mv or cp are not.
– Silly Freak
Dec 1 at 11:22




well, when I'm moving files or directories inside one partition, I usually use mv (or the file manager) because it's only moving a reference to the file/directory. If I need to do actual data transfer, then I use rsync if one of the following is true: 1) I'm moving more files than I can check for correct transfer at a glance; 2) I anticipate that I'll need to keep files in sync; 3) I expect the transfer could be interrupted. My point is, for the use case you're presenting in the question, rsync is simply the right tool, and mv or cp are not.
– Silly Freak
Dec 1 at 11:22




4




4




I would advise to always run any rsync command with -v and —dry-run first to confirm exactly what it’s going to do.
– Darren
Dec 1 at 18:49




I would advise to always run any rsync command with -v and —dry-run first to confirm exactly what it’s going to do.
– Darren
Dec 1 at 18:49


















draft saved

draft discarded




















































Thanks for contributing an answer to Unix & Linux Stack Exchange!


  • Please be sure to answer the question. Provide details and share your research!

But avoid



  • Asking for help, clarification, or responding to other answers.

  • Making statements based on opinion; back them up with references or personal experience.


To learn more, see our tips on writing great answers.





Some of your past answers have not been well-received, and you're in danger of being blocked from answering.


Please pay close attention to the following guidance:


  • Please be sure to answer the question. Provide details and share your research!

But avoid



  • Asking for help, clarification, or responding to other answers.

  • Making statements based on opinion; back them up with references or personal experience.


To learn more, see our tips on writing great answers.




draft saved


draft discarded














StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f485321%2fbest-practice-to-continue-mv%23new-answer', 'question_page');
}
);

Post as a guest















Required, but never shown





















































Required, but never shown














Required, but never shown












Required, but never shown







Required, but never shown

































Required, but never shown














Required, but never shown












Required, but never shown







Required, but never shown







Popular posts from this blog

Quarter-circle Tiles

build a pushdown automaton that recognizes the reverse language of a given pushdown automaton?

Mont Emei