Is there a more efficient way to use grep?Stop Search param in directories by grep immediately after param matchCommand line tool to search phrases in large number of pdf filesHow to search for files and directories using a single find commandEfficient way to search string within file find and grepVim: How do you use external grep on windows to search for quotation marks?How do I grep the first 50 lines of each file in a directory recursively?Anything faster than grep?Pipe arguments into “open” commandTrying to exclude a specific directory from a grep commandAny way to search inside dmg file?
Is a switch from R to Python worth it?
Vectorised way to calculate mean of left and right neighbours in a vector
…down the primrose path
What is the reason behind water not falling from a bucket at the top of loop?
Is it okay to use different fingers every time while playing a song on keyboard? Is it considered a bad practice?
Upper Bound for a Sum
How to design an effective polearm-bow hybrid?
How to call made-up data?
Make lens aperture in Tikz
If someone else uploads my GPL'd code to Github without my permission, is that a copyright violation?
split inside flalign
How does Rust's 128-bit integer `i128` work on a 64-bit system?
Plotting Autoregressive Functions / Linear Difference Equations
Getting an entry level IT position later in life
Variable doesn't parse as string
Why are there yellow dot stickers on the front doors of businesses in Russia?
What are the limitations of the Hendersson-Hasselbalch equation?
What is it exactly about flying a Flyboard across the English channel that made Zapata's thighs burn?
On the consistency of different well-polished astronomy software
Write The Shortest Program To Check If A Binary Tree Is Balanced
Conditional probability of dependent random variables
Can a Hogwarts student refuse the Sorting Hat's decision?
Why did the US Airways Flight 1549 passengers stay on the wings?
How easy is it to get a gun illegally in the United States?
Is there a more efficient way to use grep?
Stop Search param in directories by grep immediately after param matchCommand line tool to search phrases in large number of pdf filesHow to search for files and directories using a single find commandEfficient way to search string within file find and grepVim: How do you use external grep on windows to search for quotation marks?How do I grep the first 50 lines of each file in a directory recursively?Anything faster than grep?Pipe arguments into “open” commandTrying to exclude a specific directory from a grep commandAny way to search inside dmg file?
.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty margin-bottom:0;
I have been using grep to search databases. I've been using the following command when searching multiple folders with multiple text files.
grep 'text_search' */*
is there a better command that will speed up the process of searching through large amounts of data? I am open to using other tools if needed.
linux grep
asked Jul 25 at 16:35
dsec35dsec35
111 bronze badge
add a comment |
I have been using grep to search databases. I've been using the following command when searching multiple folders with multiple text files.
grep 'text_search' */*
is there a better command that will speed up the process of searching through large amounts of data? I am open to using other tools if needed.
linux grep
asked Jul 25 at 16:35
dsec35dsec35
111 bronze badge
add a comment |
I have been using grep to search databases. I've been using the following command when searching multiple folders with multiple text files.
grep 'text_search' */*
is there a better command that will speed up the process of searching through large amounts of data? I am open to using other tools if needed.
linux grep
asked Jul 25 at 16:35
dsec35dsec35
111 bronze badge
I have been using grep to search databases. I've been using the following command when searching multiple folders with multiple text files.
grep 'text_search' */*
is there a better command that will speed up the process of searching through large amounts of data? I am open to using other tools if needed.
linux grep
linux grep
asked Jul 25 at 16:35
dsec35dsec35
111 bronze badge
asked Jul 25 at 16:35
dsec35dsec35
111 bronze badge
asked Jul 25 at 16:35
dsec35dsec35
111 bronze badge
asked Jul 25 at 16:35
dsec35dsec35
111 bronze badge
asked Jul 25 at 16:35
dsec35dsec35
111 bronze badge
111 bronze badge
add a comment |
add a comment |
1 Answer
1
active
oldest
votes
I'm a fan of ripgrep
If you prefer to stick with grep, grep -F matches strings not patterns (which may or may not be faster; I'm not sure if modern greps simplify a simple pattern to a string search).
Running grep in parallel is also an option. I use GNU parallel for this.
find . -type f | parallel --jobs #jobs -n 500 -k -m grep -H search-pattern
(jobs and search-pattern aren't surrounded in braces; they indicate a variable you need to enter)
edited Jul 25 at 18:35
Toto
5,68412 gold badges14 silver badges29 bronze badges
answered Jul 25 at 16:50
meangrapemeangrape
411 bronze badge
Theres also ack and ag, off the top of my head, and fzf depending on what youre doing.
– D. Ben Knoble
Jul 26 at 1:08
I'd avoid parallelism if you have a rotational hard disk, however fast it may be. Or at least benchmark it with a cold cache (echo 3 > /proc/sys/vm/drop_caches) before settling on it as a real benefit. The only time I have benefited from any parallelism is when the data can fit in RAM and I have to make multiple greps, one after another. In that case, the first one is slow (it reads from disk), but subsequent ones are very fast.
– sitaram
Jul 29 at 1:10
add a comment |
Your Answer
StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "3"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);
else
createEditor();
);
function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);
);
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fsuperuser.com%2fquestions%2f1464059%2fis-there-a-more-efficient-way-to-use-grep%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
I'm a fan of ripgrep
If you prefer to stick with grep, grep -F matches strings not patterns (which may or may not be faster; I'm not sure if modern greps simplify a simple pattern to a string search).
Running grep in parallel is also an option. I use GNU parallel for this.
find . -type f | parallel --jobs #jobs -n 500 -k -m grep -H search-pattern
(jobs and search-pattern aren't surrounded in braces; they indicate a variable you need to enter)
edited Jul 25 at 18:35
Toto
5,68412 gold badges14 silver badges29 bronze badges
answered Jul 25 at 16:50
meangrapemeangrape
411 bronze badge
Theres also ack and ag, off the top of my head, and fzf depending on what youre doing.
– D. Ben Knoble
Jul 26 at 1:08
I'd avoid parallelism if you have a rotational hard disk, however fast it may be. Or at least benchmark it with a cold cache (echo 3 > /proc/sys/vm/drop_caches) before settling on it as a real benefit. The only time I have benefited from any parallelism is when the data can fit in RAM and I have to make multiple greps, one after another. In that case, the first one is slow (it reads from disk), but subsequent ones are very fast.
– sitaram
Jul 29 at 1:10
add a comment |
I'm a fan of ripgrep
If you prefer to stick with grep, grep -F matches strings not patterns (which may or may not be faster; I'm not sure if modern greps simplify a simple pattern to a string search).
Running grep in parallel is also an option. I use GNU parallel for this.
find . -type f | parallel --jobs #jobs -n 500 -k -m grep -H search-pattern
(jobs and search-pattern aren't surrounded in braces; they indicate a variable you need to enter)
edited Jul 25 at 18:35
Toto
5,68412 gold badges14 silver badges29 bronze badges
answered Jul 25 at 16:50
meangrapemeangrape
411 bronze badge
Theres also ack and ag, off the top of my head, and fzf depending on what youre doing.
– D. Ben Knoble
Jul 26 at 1:08
I'd avoid parallelism if you have a rotational hard disk, however fast it may be. Or at least benchmark it with a cold cache (echo 3 > /proc/sys/vm/drop_caches) before settling on it as a real benefit. The only time I have benefited from any parallelism is when the data can fit in RAM and I have to make multiple greps, one after another. In that case, the first one is slow (it reads from disk), but subsequent ones are very fast.
– sitaram
Jul 29 at 1:10
add a comment |
I'm a fan of ripgrep
If you prefer to stick with grep, grep -F matches strings not patterns (which may or may not be faster; I'm not sure if modern greps simplify a simple pattern to a string search).
Running grep in parallel is also an option. I use GNU parallel for this.
find . -type f | parallel --jobs #jobs -n 500 -k -m grep -H search-pattern
(jobs and search-pattern aren't surrounded in braces; they indicate a variable you need to enter)
edited Jul 25 at 18:35
Toto
5,68412 gold badges14 silver badges29 bronze badges
answered Jul 25 at 16:50
meangrapemeangrape
411 bronze badge
I'm a fan of ripgrep
If you prefer to stick with grep, grep -F matches strings not patterns (which may or may not be faster; I'm not sure if modern greps simplify a simple pattern to a string search).
Running grep in parallel is also an option. I use GNU parallel for this.
find . -type f | parallel --jobs #jobs -n 500 -k -m grep -H search-pattern
(jobs and search-pattern aren't surrounded in braces; they indicate a variable you need to enter)
edited Jul 25 at 18:35
Toto
5,68412 gold badges14 silver badges29 bronze badges
answered Jul 25 at 16:50
meangrapemeangrape
411 bronze badge
edited Jul 25 at 18:35
Toto
5,68412 gold badges14 silver badges29 bronze badges
edited Jul 25 at 18:35
Toto
5,68412 gold badges14 silver badges29 bronze badges
edited Jul 25 at 18:35
Toto
5,68412 gold badges14 silver badges29 bronze badges
5,68412 gold badges14 silver badges29 bronze badges
answered Jul 25 at 16:50
meangrapemeangrape
411 bronze badge
answered Jul 25 at 16:50
meangrapemeangrape
411 bronze badge
answered Jul 25 at 16:50
meangrapemeangrape
411 bronze badge
411 bronze badge
Theres also ack and ag, off the top of my head, and fzf depending on what youre doing.
– D. Ben Knoble
Jul 26 at 1:08
I'd avoid parallelism if you have a rotational hard disk, however fast it may be. Or at least benchmark it with a cold cache (echo 3 > /proc/sys/vm/drop_caches) before settling on it as a real benefit. The only time I have benefited from any parallelism is when the data can fit in RAM and I have to make multiple greps, one after another. In that case, the first one is slow (it reads from disk), but subsequent ones are very fast.
– sitaram
Jul 29 at 1:10
add a comment |
Theres also ack and ag, off the top of my head, and fzf depending on what youre doing.
– D. Ben Knoble
Jul 26 at 1:08
I'd avoid parallelism if you have a rotational hard disk, however fast it may be. Or at least benchmark it with a cold cache (echo 3 > /proc/sys/vm/drop_caches) before settling on it as a real benefit. The only time I have benefited from any parallelism is when the data can fit in RAM and I have to make multiple greps, one after another. In that case, the first one is slow (it reads from disk), but subsequent ones are very fast.
– sitaram
Jul 29 at 1:10
Theres also ack and ag, off the top of my head, and fzf depending on what youre doing.
– D. Ben Knoble
Jul 26 at 1:08
Theres also ack and ag, off the top of my head, and fzf depending on what youre doing.
– D. Ben Knoble
Jul 26 at 1:08
I'd avoid parallelism if you have a rotational hard disk, however fast it may be. Or at least benchmark it with a cold cache (
echo 3 > /proc/sys/vm/drop_caches) before settling on it as a real benefit. The only time I have benefited from any parallelism is when the data can fit in RAM and I have to make multiple greps, one after another. In that case, the first one is slow (it reads from disk), but subsequent ones are very fast.– sitaram
Jul 29 at 1:10
I'd avoid parallelism if you have a rotational hard disk, however fast it may be. Or at least benchmark it with a cold cache (
echo 3 > /proc/sys/vm/drop_caches) before settling on it as a real benefit. The only time I have benefited from any parallelism is when the data can fit in RAM and I have to make multiple greps, one after another. In that case, the first one is slow (it reads from disk), but subsequent ones are very fast.– sitaram
Jul 29 at 1:10
add a comment |
Thanks for contributing an answer to Super User!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fsuperuser.com%2fquestions%2f1464059%2fis-there-a-more-efficient-way-to-use-grep%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
