Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Below are the results I got from _test2.

    + echo .{150}(https:)|(http:).{10}
    + n=0
    + test 0 -le 3
    + busybox time rg -o .{150}(https:)|(http:).{10} test.json
    real 1m 33.37s
    user 0m 1.25s
    sys  0m 2.97s
    + sleep 2
    + echo
    + echo Now try this with a pipe to wc -l...
    + echo
    + sleep 2
    + busybox time+  rg -o .{150}(https:)|(http:).{10} test.json
    wc -l
    real 0m 0.49s
    user 0m 0.45s
    sys  0m 0.02s
    + echo
    + sleep 5
    + echo
    + n=1
    + test 1 -le 3
    + busybox time rg -o .{150}(https:)|(http:).{10} test.json
    real 1m 34.23s
    user 0m 1.75s
    sys  0m 4.22s
    + sleep 2
    + echo
    + echo Now try this with a pipe to wc -l...
    + echo
    + sleep 2
    + busybox time rg -o .{150}(https:)|(http:).{10} test.json
    wc -l
    real 0m 0.40s
    user 0m 0.37s
    sys  0m 0.02s
    + echo
    + sleep 5
    + echo
    + n=2
    + test 2 -le 3
    + busybox time rg -o .{150}(https:)|(http:).{10} test.json
    real 1m 33.59s
    user 0m 1.05s
    sys  0m 1.76s
    + sleep 2
    + echo
    + echo Now try this with a pipe to wc -l...
    + echo
    + sleep 2
    + busybox time rg -o .{150}(https:)|(http:).{10}+  test.json
    wc -l
    real 0m 0.45s
    user 0m 0.37s
    sys  0m 0.04s
    + echo
    + sleep 5
    + echo
    + n=3
    + test 3 -le 3
    + busybox time rg -o .{150}(https:)|(http:).{10} test.json
    real 1m 33.99s
    user 0m 1.93s
    sys  0m 4.82s
    + sleep 2
    + echo
    + echo Now try this with a pipe to wc -l...
    + echo
    + sleep 2
    + busybox time rg -o .{150}(https:)|(http:).{10} test.json
    wc -l
    real 0m 0.40s
    user 0m 0.37s
    sys  0m 0.02s
    + echo
    + sleep 5
    + echo
    + n=4
    + test 4 -le 3
    + exit
No, I am not going to stare at the screen for a minute and half as thousands of matches are displayed. (In fact I am unlikely to even be examining a file of this size. It's more likely to be under 6M.) With a file this size what I would do is examine a sample of the matches, let's say for example the first 20.

Look at the speed of a shell script with 9 pipes, using ired 3x to examine the first 20 matches.

    busybox time sh -c "echo -n https:
    |od -tx1 -An \
    |tr -d '\40' \
    |sed 's>^>/>' \
    |ired -n test.json \
    |sed  '1,21s/.*/s&@s-150@b166@X/;21q' \
    |tr @ '\12' \
    |ired -n test.json \
    |sed 's/.*/w&0a/' \
    |ired -n /dev/stdout"

    real 0m 0.01s
    user 0m 0.00s
    sys 0m 0.00s
That speed is just right.

Now with the same ripgrep.

    busybox time rg -o .{150}https:.{10} test.json|sed 20q 

    real 0m 0.40s
    user 0m 0.37s
    sys 0m 0.02s
Being used to the speed of ired, this is slow.

Even more, ired is fraction of the size of ripgrep.

For me, the choice of what to use is easy, until I discover something better than ired.



    $ busybox time rg -o '.{150}(https:)|(http:).{10}' test.json
    real    0m 0.02s
    user    0m 0.01s
    sys     0m 0.00s
I don't know how you're getting over a minute for that. Perhaps you aren't doing a good enough job at providing a reproduction.

> That speed is just right.

... unlike its results. "Wrong and fast, just the way I like it!" Lmao.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: