Grep without duplicates? [closed]
up vote
1
down vote
favorite
I have multiple revisions of a text file in separate files in the same folder.
How can I grep
all files in that folder without listing any duplicate of lines with identical text?
text-processing command-line grep
closed as unclear what you're asking by Kusalananda, Jeff Schaller, muru, Isaac, don_crissti Nov 26 at 19:55
Please clarify your specific problem or add additional details to highlight exactly what you need. As it's currently written, it’s hard to tell exactly what you're asking. See the How to Ask page for help clarifying this question. If this question can be reworded to fit the rules in the help center, please edit the question.
add a comment |
up vote
1
down vote
favorite
I have multiple revisions of a text file in separate files in the same folder.
How can I grep
all files in that folder without listing any duplicate of lines with identical text?
text-processing command-line grep
closed as unclear what you're asking by Kusalananda, Jeff Schaller, muru, Isaac, don_crissti Nov 26 at 19:55
Please clarify your specific problem or add additional details to highlight exactly what you need. As it's currently written, it’s hard to tell exactly what you're asking. See the How to Ask page for help clarifying this question. If this question can be reworded to fit the rules in the help center, please edit the question.
2
What's your desired output ? Just the lines that match a certain pattern, without any filename prepended ? If so, you should be able to find the answer yourself...
– don_crissti
Nov 25 at 21:24
2
An example would clarify the question.
– Kusalananda
Nov 25 at 21:37
1
@Kusalananda - I wonder why none of the upvoters here takes the time to reply and enlighten us... This question "is clear" and "shows research effort" ?
– don_crissti
Nov 25 at 22:04
@Don_crissti Desired output? Filename: Can but does not have to.
– neverMind9
Nov 25 at 22:12
add a comment |
up vote
1
down vote
favorite
up vote
1
down vote
favorite
I have multiple revisions of a text file in separate files in the same folder.
How can I grep
all files in that folder without listing any duplicate of lines with identical text?
text-processing command-line grep
I have multiple revisions of a text file in separate files in the same folder.
How can I grep
all files in that folder without listing any duplicate of lines with identical text?
text-processing command-line grep
text-processing command-line grep
edited Nov 25 at 21:53
Jeff Schaller
37.1k1052121
37.1k1052121
asked Nov 25 at 20:40
neverMind9
499113
499113
closed as unclear what you're asking by Kusalananda, Jeff Schaller, muru, Isaac, don_crissti Nov 26 at 19:55
Please clarify your specific problem or add additional details to highlight exactly what you need. As it's currently written, it’s hard to tell exactly what you're asking. See the How to Ask page for help clarifying this question. If this question can be reworded to fit the rules in the help center, please edit the question.
closed as unclear what you're asking by Kusalananda, Jeff Schaller, muru, Isaac, don_crissti Nov 26 at 19:55
Please clarify your specific problem or add additional details to highlight exactly what you need. As it's currently written, it’s hard to tell exactly what you're asking. See the How to Ask page for help clarifying this question. If this question can be reworded to fit the rules in the help center, please edit the question.
2
What's your desired output ? Just the lines that match a certain pattern, without any filename prepended ? If so, you should be able to find the answer yourself...
– don_crissti
Nov 25 at 21:24
2
An example would clarify the question.
– Kusalananda
Nov 25 at 21:37
1
@Kusalananda - I wonder why none of the upvoters here takes the time to reply and enlighten us... This question "is clear" and "shows research effort" ?
– don_crissti
Nov 25 at 22:04
@Don_crissti Desired output? Filename: Can but does not have to.
– neverMind9
Nov 25 at 22:12
add a comment |
2
What's your desired output ? Just the lines that match a certain pattern, without any filename prepended ? If so, you should be able to find the answer yourself...
– don_crissti
Nov 25 at 21:24
2
An example would clarify the question.
– Kusalananda
Nov 25 at 21:37
1
@Kusalananda - I wonder why none of the upvoters here takes the time to reply and enlighten us... This question "is clear" and "shows research effort" ?
– don_crissti
Nov 25 at 22:04
@Don_crissti Desired output? Filename: Can but does not have to.
– neverMind9
Nov 25 at 22:12
2
2
What's your desired output ? Just the lines that match a certain pattern, without any filename prepended ? If so, you should be able to find the answer yourself...
– don_crissti
Nov 25 at 21:24
What's your desired output ? Just the lines that match a certain pattern, without any filename prepended ? If so, you should be able to find the answer yourself...
– don_crissti
Nov 25 at 21:24
2
2
An example would clarify the question.
– Kusalananda
Nov 25 at 21:37
An example would clarify the question.
– Kusalananda
Nov 25 at 21:37
1
1
@Kusalananda - I wonder why none of the upvoters here takes the time to reply and enlighten us... This question "is clear" and "shows research effort" ?
– don_crissti
Nov 25 at 22:04
@Kusalananda - I wonder why none of the upvoters here takes the time to reply and enlighten us... This question "is clear" and "shows research effort" ?
– don_crissti
Nov 25 at 22:04
@Don_crissti Desired output? Filename: Can but does not have to.
– neverMind9
Nov 25 at 22:12
@Don_crissti Desired output? Filename: Can but does not have to.
– neverMind9
Nov 25 at 22:12
add a comment |
6 Answers
6
active
oldest
votes
up vote
2
down vote
accepted
How about
cat * | grep exampletext | sort -u
4
This works butcat
isn't necessary. All you need is:grep -H /path/to/files/* | sort -u
.
– Nasir Riley
Nov 26 at 3:01
2
@ Nasir Riley: accepted - but it would be-h
toSuppress the prefixing of file names on output
.
– RudiC
Nov 26 at 9:35
add a comment |
up vote
1
down vote
I use:
grep -h test files* | puniq
puniq
is: perl -ne '$seen{$_}++ or print;'
It is similar to sort -u
but it does not sort the input and it gives output while running.
If you want the file name and avoid duplicate lines in each file:
parallel --tag --lb 'grep string {} | puniq' ::: files*
If you want the file name and do not want duplicate lines from any of the files (File names must not contain TAB (t)):
parallel --tag --lb grep string {} ::: files* |
perl -ne '/^[^t]+(.*)/ and $seen{$1}++ or print;'
add a comment |
up vote
1
down vote
Maybe something like this could be close to what you imagine (works with gnu awk):
cat file1
1
2
3
22
cat file11
1
2
3
8
9
cat file111
1
2
3
5
6
awk '{seen[$0]++;fname[$0]=FILENAME};END{for (k in seen) {if (seen[k]==1) print fname[k],":",k}}' file1*
file111 : 5
file111 : 6
file11 : 8
file11 : 9
file1 : 22
add a comment |
up vote
1
down vote
Pipe the result into sort to filter duplicates.
grep -re pattern files and dirs ... | sort -ut: -k2
The -t:
and -k2
options of sort will cause it to ignore the file name when doing the sorting and merging.
Or, if you don't want the filenames, simply:
grep -hre pattern files and dirs ... | sort -u
2
You are missing the files that you are going togrep
.
– Nasir Riley
Nov 25 at 21:18
5
and then, once you supply multiple filenames, the filenames themselves would destroy the purpose ofsort -u
...
– Jeff Schaller
Nov 25 at 21:52
1
come on, bothsort
anduniq
have the ability to consider only some fields; this answer could be improved:grep -re pattern files and dirs | sort -ut: -k2
– mosvy
Nov 25 at 23:47
Caveat: Will give wrong output if file names contain:
.
– Ole Tange
Nov 26 at 0:00
@OleTange it would only give false positives in that case (it would write more than once a line that it should print only once). That's an acceptable compromise. If you omit the-t:
it will have the same problem with filenames that contain spaces -- I think that filenames with spaces are more common than files with colons.
– mosvy
Nov 26 at 0:05
|
show 3 more comments
up vote
-1
down vote
If what you need is to find which file(s) match some text, use:
$ grep -rl 'text to find' ./dir
If you need only the first match of each file:
$ for file in ./*; do sed -n '/text to match/{p,q}' "$file"; done
which will not print the name of the files matching, but will be fast.
Or:
$ find ../* -type f -exec sh -c '
a=$(sed -n "/echo/{p;q}" "$1");
[ "$a" ] && printf "%sn" "$1 : $a"
' findsh {} ;
If you need the filename also (separated with :
).
add a comment |
up vote
-2
down vote
I would use something like:
#!/bin/bash
echo "What is the files base name? e.g. testfile"
read filename
for i in *$filename*; do
cat $i >> tempfile
cat tempfile | sort -u -o tempfile
done
or
cat *filename* | sort -u
The placement of the *'s is based on how you name your itterations.
- When done read tempfile.
- Rename to whatever.
Use with caution in a test directory with test files first
- Tailor to your needs.
My script turns:
testfile1 testfile2 testfile3
This is line 1 This is line 5 This is line 5
This is line 2 This is line 6 This is line 2
This is line 3 This is line 7 This is line 7
This is line 4 This is line 8 This is line 3
Into :
This is line 1
This is line 2
This is line 3
This is line 4
This is line 5
This is line 6
This is line 7
This is line 8
add a comment |
6 Answers
6
active
oldest
votes
6 Answers
6
active
oldest
votes
active
oldest
votes
active
oldest
votes
up vote
2
down vote
accepted
How about
cat * | grep exampletext | sort -u
4
This works butcat
isn't necessary. All you need is:grep -H /path/to/files/* | sort -u
.
– Nasir Riley
Nov 26 at 3:01
2
@ Nasir Riley: accepted - but it would be-h
toSuppress the prefixing of file names on output
.
– RudiC
Nov 26 at 9:35
add a comment |
up vote
2
down vote
accepted
How about
cat * | grep exampletext | sort -u
4
This works butcat
isn't necessary. All you need is:grep -H /path/to/files/* | sort -u
.
– Nasir Riley
Nov 26 at 3:01
2
@ Nasir Riley: accepted - but it would be-h
toSuppress the prefixing of file names on output
.
– RudiC
Nov 26 at 9:35
add a comment |
up vote
2
down vote
accepted
up vote
2
down vote
accepted
How about
cat * | grep exampletext | sort -u
How about
cat * | grep exampletext | sort -u
answered Nov 25 at 23:23
RudiC
3,6721312
3,6721312
4
This works butcat
isn't necessary. All you need is:grep -H /path/to/files/* | sort -u
.
– Nasir Riley
Nov 26 at 3:01
2
@ Nasir Riley: accepted - but it would be-h
toSuppress the prefixing of file names on output
.
– RudiC
Nov 26 at 9:35
add a comment |
4
This works butcat
isn't necessary. All you need is:grep -H /path/to/files/* | sort -u
.
– Nasir Riley
Nov 26 at 3:01
2
@ Nasir Riley: accepted - but it would be-h
toSuppress the prefixing of file names on output
.
– RudiC
Nov 26 at 9:35
4
4
This works but
cat
isn't necessary. All you need is: grep -H /path/to/files/* | sort -u
.– Nasir Riley
Nov 26 at 3:01
This works but
cat
isn't necessary. All you need is: grep -H /path/to/files/* | sort -u
.– Nasir Riley
Nov 26 at 3:01
2
2
@ Nasir Riley: accepted - but it would be
-h
to Suppress the prefixing of file names on output
.– RudiC
Nov 26 at 9:35
@ Nasir Riley: accepted - but it would be
-h
to Suppress the prefixing of file names on output
.– RudiC
Nov 26 at 9:35
add a comment |
up vote
1
down vote
I use:
grep -h test files* | puniq
puniq
is: perl -ne '$seen{$_}++ or print;'
It is similar to sort -u
but it does not sort the input and it gives output while running.
If you want the file name and avoid duplicate lines in each file:
parallel --tag --lb 'grep string {} | puniq' ::: files*
If you want the file name and do not want duplicate lines from any of the files (File names must not contain TAB (t)):
parallel --tag --lb grep string {} ::: files* |
perl -ne '/^[^t]+(.*)/ and $seen{$1}++ or print;'
add a comment |
up vote
1
down vote
I use:
grep -h test files* | puniq
puniq
is: perl -ne '$seen{$_}++ or print;'
It is similar to sort -u
but it does not sort the input and it gives output while running.
If you want the file name and avoid duplicate lines in each file:
parallel --tag --lb 'grep string {} | puniq' ::: files*
If you want the file name and do not want duplicate lines from any of the files (File names must not contain TAB (t)):
parallel --tag --lb grep string {} ::: files* |
perl -ne '/^[^t]+(.*)/ and $seen{$1}++ or print;'
add a comment |
up vote
1
down vote
up vote
1
down vote
I use:
grep -h test files* | puniq
puniq
is: perl -ne '$seen{$_}++ or print;'
It is similar to sort -u
but it does not sort the input and it gives output while running.
If you want the file name and avoid duplicate lines in each file:
parallel --tag --lb 'grep string {} | puniq' ::: files*
If you want the file name and do not want duplicate lines from any of the files (File names must not contain TAB (t)):
parallel --tag --lb grep string {} ::: files* |
perl -ne '/^[^t]+(.*)/ and $seen{$1}++ or print;'
I use:
grep -h test files* | puniq
puniq
is: perl -ne '$seen{$_}++ or print;'
It is similar to sort -u
but it does not sort the input and it gives output while running.
If you want the file name and avoid duplicate lines in each file:
parallel --tag --lb 'grep string {} | puniq' ::: files*
If you want the file name and do not want duplicate lines from any of the files (File names must not contain TAB (t)):
parallel --tag --lb grep string {} ::: files* |
perl -ne '/^[^t]+(.*)/ and $seen{$1}++ or print;'
edited Nov 25 at 23:52
answered Nov 25 at 21:45
Ole Tange
11.9k1451105
11.9k1451105
add a comment |
add a comment |
up vote
1
down vote
Maybe something like this could be close to what you imagine (works with gnu awk):
cat file1
1
2
3
22
cat file11
1
2
3
8
9
cat file111
1
2
3
5
6
awk '{seen[$0]++;fname[$0]=FILENAME};END{for (k in seen) {if (seen[k]==1) print fname[k],":",k}}' file1*
file111 : 5
file111 : 6
file11 : 8
file11 : 9
file1 : 22
add a comment |
up vote
1
down vote
Maybe something like this could be close to what you imagine (works with gnu awk):
cat file1
1
2
3
22
cat file11
1
2
3
8
9
cat file111
1
2
3
5
6
awk '{seen[$0]++;fname[$0]=FILENAME};END{for (k in seen) {if (seen[k]==1) print fname[k],":",k}}' file1*
file111 : 5
file111 : 6
file11 : 8
file11 : 9
file1 : 22
add a comment |
up vote
1
down vote
up vote
1
down vote
Maybe something like this could be close to what you imagine (works with gnu awk):
cat file1
1
2
3
22
cat file11
1
2
3
8
9
cat file111
1
2
3
5
6
awk '{seen[$0]++;fname[$0]=FILENAME};END{for (k in seen) {if (seen[k]==1) print fname[k],":",k}}' file1*
file111 : 5
file111 : 6
file11 : 8
file11 : 9
file1 : 22
Maybe something like this could be close to what you imagine (works with gnu awk):
cat file1
1
2
3
22
cat file11
1
2
3
8
9
cat file111
1
2
3
5
6
awk '{seen[$0]++;fname[$0]=FILENAME};END{for (k in seen) {if (seen[k]==1) print fname[k],":",k}}' file1*
file111 : 5
file111 : 6
file11 : 8
file11 : 9
file1 : 22
answered Nov 26 at 2:20
George Vasiliou
5,57531028
5,57531028
add a comment |
add a comment |
up vote
1
down vote
Pipe the result into sort to filter duplicates.
grep -re pattern files and dirs ... | sort -ut: -k2
The -t:
and -k2
options of sort will cause it to ignore the file name when doing the sorting and merging.
Or, if you don't want the filenames, simply:
grep -hre pattern files and dirs ... | sort -u
2
You are missing the files that you are going togrep
.
– Nasir Riley
Nov 25 at 21:18
5
and then, once you supply multiple filenames, the filenames themselves would destroy the purpose ofsort -u
...
– Jeff Schaller
Nov 25 at 21:52
1
come on, bothsort
anduniq
have the ability to consider only some fields; this answer could be improved:grep -re pattern files and dirs | sort -ut: -k2
– mosvy
Nov 25 at 23:47
Caveat: Will give wrong output if file names contain:
.
– Ole Tange
Nov 26 at 0:00
@OleTange it would only give false positives in that case (it would write more than once a line that it should print only once). That's an acceptable compromise. If you omit the-t:
it will have the same problem with filenames that contain spaces -- I think that filenames with spaces are more common than files with colons.
– mosvy
Nov 26 at 0:05
|
show 3 more comments
up vote
1
down vote
Pipe the result into sort to filter duplicates.
grep -re pattern files and dirs ... | sort -ut: -k2
The -t:
and -k2
options of sort will cause it to ignore the file name when doing the sorting and merging.
Or, if you don't want the filenames, simply:
grep -hre pattern files and dirs ... | sort -u
2
You are missing the files that you are going togrep
.
– Nasir Riley
Nov 25 at 21:18
5
and then, once you supply multiple filenames, the filenames themselves would destroy the purpose ofsort -u
...
– Jeff Schaller
Nov 25 at 21:52
1
come on, bothsort
anduniq
have the ability to consider only some fields; this answer could be improved:grep -re pattern files and dirs | sort -ut: -k2
– mosvy
Nov 25 at 23:47
Caveat: Will give wrong output if file names contain:
.
– Ole Tange
Nov 26 at 0:00
@OleTange it would only give false positives in that case (it would write more than once a line that it should print only once). That's an acceptable compromise. If you omit the-t:
it will have the same problem with filenames that contain spaces -- I think that filenames with spaces are more common than files with colons.
– mosvy
Nov 26 at 0:05
|
show 3 more comments
up vote
1
down vote
up vote
1
down vote
Pipe the result into sort to filter duplicates.
grep -re pattern files and dirs ... | sort -ut: -k2
The -t:
and -k2
options of sort will cause it to ignore the file name when doing the sorting and merging.
Or, if you don't want the filenames, simply:
grep -hre pattern files and dirs ... | sort -u
Pipe the result into sort to filter duplicates.
grep -re pattern files and dirs ... | sort -ut: -k2
The -t:
and -k2
options of sort will cause it to ignore the file name when doing the sorting and merging.
Or, if you don't want the filenames, simply:
grep -hre pattern files and dirs ... | sort -u
edited Nov 26 at 3:20
mosvy
5,166323
5,166323
answered Nov 25 at 21:04
Jon Reinhold
846614
846614
2
You are missing the files that you are going togrep
.
– Nasir Riley
Nov 25 at 21:18
5
and then, once you supply multiple filenames, the filenames themselves would destroy the purpose ofsort -u
...
– Jeff Schaller
Nov 25 at 21:52
1
come on, bothsort
anduniq
have the ability to consider only some fields; this answer could be improved:grep -re pattern files and dirs | sort -ut: -k2
– mosvy
Nov 25 at 23:47
Caveat: Will give wrong output if file names contain:
.
– Ole Tange
Nov 26 at 0:00
@OleTange it would only give false positives in that case (it would write more than once a line that it should print only once). That's an acceptable compromise. If you omit the-t:
it will have the same problem with filenames that contain spaces -- I think that filenames with spaces are more common than files with colons.
– mosvy
Nov 26 at 0:05
|
show 3 more comments
2
You are missing the files that you are going togrep
.
– Nasir Riley
Nov 25 at 21:18
5
and then, once you supply multiple filenames, the filenames themselves would destroy the purpose ofsort -u
...
– Jeff Schaller
Nov 25 at 21:52
1
come on, bothsort
anduniq
have the ability to consider only some fields; this answer could be improved:grep -re pattern files and dirs | sort -ut: -k2
– mosvy
Nov 25 at 23:47
Caveat: Will give wrong output if file names contain:
.
– Ole Tange
Nov 26 at 0:00
@OleTange it would only give false positives in that case (it would write more than once a line that it should print only once). That's an acceptable compromise. If you omit the-t:
it will have the same problem with filenames that contain spaces -- I think that filenames with spaces are more common than files with colons.
– mosvy
Nov 26 at 0:05
2
2
You are missing the files that you are going to
grep
.– Nasir Riley
Nov 25 at 21:18
You are missing the files that you are going to
grep
.– Nasir Riley
Nov 25 at 21:18
5
5
and then, once you supply multiple filenames, the filenames themselves would destroy the purpose of
sort -u
...– Jeff Schaller
Nov 25 at 21:52
and then, once you supply multiple filenames, the filenames themselves would destroy the purpose of
sort -u
...– Jeff Schaller
Nov 25 at 21:52
1
1
come on, both
sort
and uniq
have the ability to consider only some fields; this answer could be improved: grep -re pattern files and dirs | sort -ut: -k2
– mosvy
Nov 25 at 23:47
come on, both
sort
and uniq
have the ability to consider only some fields; this answer could be improved: grep -re pattern files and dirs | sort -ut: -k2
– mosvy
Nov 25 at 23:47
Caveat: Will give wrong output if file names contain
:
.– Ole Tange
Nov 26 at 0:00
Caveat: Will give wrong output if file names contain
:
.– Ole Tange
Nov 26 at 0:00
@OleTange it would only give false positives in that case (it would write more than once a line that it should print only once). That's an acceptable compromise. If you omit the
-t:
it will have the same problem with filenames that contain spaces -- I think that filenames with spaces are more common than files with colons.– mosvy
Nov 26 at 0:05
@OleTange it would only give false positives in that case (it would write more than once a line that it should print only once). That's an acceptable compromise. If you omit the
-t:
it will have the same problem with filenames that contain spaces -- I think that filenames with spaces are more common than files with colons.– mosvy
Nov 26 at 0:05
|
show 3 more comments
up vote
-1
down vote
If what you need is to find which file(s) match some text, use:
$ grep -rl 'text to find' ./dir
If you need only the first match of each file:
$ for file in ./*; do sed -n '/text to match/{p,q}' "$file"; done
which will not print the name of the files matching, but will be fast.
Or:
$ find ../* -type f -exec sh -c '
a=$(sed -n "/echo/{p;q}" "$1");
[ "$a" ] && printf "%sn" "$1 : $a"
' findsh {} ;
If you need the filename also (separated with :
).
add a comment |
up vote
-1
down vote
If what you need is to find which file(s) match some text, use:
$ grep -rl 'text to find' ./dir
If you need only the first match of each file:
$ for file in ./*; do sed -n '/text to match/{p,q}' "$file"; done
which will not print the name of the files matching, but will be fast.
Or:
$ find ../* -type f -exec sh -c '
a=$(sed -n "/echo/{p;q}" "$1");
[ "$a" ] && printf "%sn" "$1 : $a"
' findsh {} ;
If you need the filename also (separated with :
).
add a comment |
up vote
-1
down vote
up vote
-1
down vote
If what you need is to find which file(s) match some text, use:
$ grep -rl 'text to find' ./dir
If you need only the first match of each file:
$ for file in ./*; do sed -n '/text to match/{p,q}' "$file"; done
which will not print the name of the files matching, but will be fast.
Or:
$ find ../* -type f -exec sh -c '
a=$(sed -n "/echo/{p;q}" "$1");
[ "$a" ] && printf "%sn" "$1 : $a"
' findsh {} ;
If you need the filename also (separated with :
).
If what you need is to find which file(s) match some text, use:
$ grep -rl 'text to find' ./dir
If you need only the first match of each file:
$ for file in ./*; do sed -n '/text to match/{p,q}' "$file"; done
which will not print the name of the files matching, but will be fast.
Or:
$ find ../* -type f -exec sh -c '
a=$(sed -n "/echo/{p;q}" "$1");
[ "$a" ] && printf "%sn" "$1 : $a"
' findsh {} ;
If you need the filename also (separated with :
).
answered Nov 25 at 21:42
Isaac
10.3k11446
10.3k11446
add a comment |
add a comment |
up vote
-2
down vote
I would use something like:
#!/bin/bash
echo "What is the files base name? e.g. testfile"
read filename
for i in *$filename*; do
cat $i >> tempfile
cat tempfile | sort -u -o tempfile
done
or
cat *filename* | sort -u
The placement of the *'s is based on how you name your itterations.
- When done read tempfile.
- Rename to whatever.
Use with caution in a test directory with test files first
- Tailor to your needs.
My script turns:
testfile1 testfile2 testfile3
This is line 1 This is line 5 This is line 5
This is line 2 This is line 6 This is line 2
This is line 3 This is line 7 This is line 7
This is line 4 This is line 8 This is line 3
Into :
This is line 1
This is line 2
This is line 3
This is line 4
This is line 5
This is line 6
This is line 7
This is line 8
add a comment |
up vote
-2
down vote
I would use something like:
#!/bin/bash
echo "What is the files base name? e.g. testfile"
read filename
for i in *$filename*; do
cat $i >> tempfile
cat tempfile | sort -u -o tempfile
done
or
cat *filename* | sort -u
The placement of the *'s is based on how you name your itterations.
- When done read tempfile.
- Rename to whatever.
Use with caution in a test directory with test files first
- Tailor to your needs.
My script turns:
testfile1 testfile2 testfile3
This is line 1 This is line 5 This is line 5
This is line 2 This is line 6 This is line 2
This is line 3 This is line 7 This is line 7
This is line 4 This is line 8 This is line 3
Into :
This is line 1
This is line 2
This is line 3
This is line 4
This is line 5
This is line 6
This is line 7
This is line 8
add a comment |
up vote
-2
down vote
up vote
-2
down vote
I would use something like:
#!/bin/bash
echo "What is the files base name? e.g. testfile"
read filename
for i in *$filename*; do
cat $i >> tempfile
cat tempfile | sort -u -o tempfile
done
or
cat *filename* | sort -u
The placement of the *'s is based on how you name your itterations.
- When done read tempfile.
- Rename to whatever.
Use with caution in a test directory with test files first
- Tailor to your needs.
My script turns:
testfile1 testfile2 testfile3
This is line 1 This is line 5 This is line 5
This is line 2 This is line 6 This is line 2
This is line 3 This is line 7 This is line 7
This is line 4 This is line 8 This is line 3
Into :
This is line 1
This is line 2
This is line 3
This is line 4
This is line 5
This is line 6
This is line 7
This is line 8
I would use something like:
#!/bin/bash
echo "What is the files base name? e.g. testfile"
read filename
for i in *$filename*; do
cat $i >> tempfile
cat tempfile | sort -u -o tempfile
done
or
cat *filename* | sort -u
The placement of the *'s is based on how you name your itterations.
- When done read tempfile.
- Rename to whatever.
Use with caution in a test directory with test files first
- Tailor to your needs.
My script turns:
testfile1 testfile2 testfile3
This is line 1 This is line 5 This is line 5
This is line 2 This is line 6 This is line 2
This is line 3 This is line 7 This is line 7
This is line 4 This is line 8 This is line 3
Into :
This is line 1
This is line 2
This is line 3
This is line 4
This is line 5
This is line 6
This is line 7
This is line 8
edited Nov 26 at 2:50
answered Nov 25 at 20:55
Michael Prokopec
74616
74616
add a comment |
add a comment |
2
What's your desired output ? Just the lines that match a certain pattern, without any filename prepended ? If so, you should be able to find the answer yourself...
– don_crissti
Nov 25 at 21:24
2
An example would clarify the question.
– Kusalananda
Nov 25 at 21:37
1
@Kusalananda - I wonder why none of the upvoters here takes the time to reply and enlighten us... This question "is clear" and "shows research effort" ?
– don_crissti
Nov 25 at 22:04
@Don_crissti Desired output? Filename: Can but does not have to.
– neverMind9
Nov 25 at 22:12