Rectangle 27 0

wget -m -O file.txt http://google.com 2> /dev/null & sleep 1 && tail -f -n1 file.txt | grep pattern

This will print out all lines that contain pattern to stdout, and wget itself will produce no output visible from the terminal. The sleep is included because otherwise file.txt would not be created by the time the tail command executed.

As a note, this command will miss any results that wget downloads within the first second.

great!..+1 for finding your solution

linux - Grep files in between wget recursive downloads - Stack Overflo...

linux bash unix grep wget
Rectangle 27 0

Update, a grep-style version using sed:

wget ... 2>&1 | sed -n '$,$s/.*(\(.*\)).*/\1/p'

I thought, it's easier to divide the file size by the download time after the download. ;-)

(/usr/bin/time -p wget ... 2>&1 >/dev/null; ls -l newfile) | \
awk '
   NR==1 {t=$2};
   NR==4 {printf("rate=%f bytes/second\n", $5/t)}
'

The first awk line stores the elapsed real time of "real xx.xx" in variabe t. The second awk line divides the file size (column 5 of ls -l) by the time and outputs this as the rate.

Not easier, it seems to me. I have to determine both file size and the download time, divide, and what if time equals zero? Please provide the example how to do it e.g. with bash

Reported time zero needs special attention. It seems grepping is perhaps easier. Somehow I had the dynamical and progressive output of wget in mind which I thought hard to grep.

linux - How to grep download speed from wget output? - Stack Overflow

linux command-line grep wget
Rectangle 27 0

I started using sed to get the speed from wget, but I found it irritating so I switched to grep.

wget ... 2>&1 | grep -o "[0-9.]\+ [KM]*B/s"

The -o option means it only returns that part. It matches 1 or more of the 10 digits then a space. Then optionally K or M before the B/s

423 KB/s
grep -o "[KM]*B/s"
grep -o "[0123456789]\+

I would remove the star from the second character class, you only need exactly one K or M. Also note that there are several occurances of speeds in the output, not all of which are of interest to the OP; a complete solution needs to take context into account, which afaik cannot be done by a single grep call.

That only gave 1 output for me... :/ He's got multiple ones but he said that he wouldn't have 2 files...? I was only using 1 file.

linux - How to grep download speed from wget output? - Stack Overflow

linux command-line grep wget
Rectangle 27 0

As c4f4t0r pointed out

wget -m -O - <wesbites>|grep --color 'pattern'

using grep's color function to highlight the patterns may seem helpful especially when dealing with bulky data output to terminal.

Below is a command line you can use. it creates a file called file and save the output messages from wget.Afterwards it tails the message file.

Using awk to find any lines with "saved" and extract filename, then use grep to pattern from filename.

wget -m websites  &> file &  tail -f -n1 file|awk -F "\'|\`"  '/saved/{system( ("grep  --colour pattern ") $2)}'

-mO - doesn't work because wget can't find the next link to recursively download.

@RogueBaneling hmmm..interesting...answer edited showing a command line you can use

I played around with that a bit and was able to get it working with this: wget -m -O file.txt http://google.com 2> /dev/null & sleep 1 && tail -f -n1 file.txt | grep pattern. Originally the tail command was not working and I believe it is because file.txt was not created by the time tail was executing, so that is why I added in the sleep.

linux - Grep files in between wget recursive downloads - Stack Overflo...

linux bash unix grep wget
Rectangle 27 0

perl -ne "/^Downloaded.*?\((.*?)\)/; print $1"

The line with "Downloaded" might not be there, if only one file was fetched.

linux - How to grep download speed from wget output? - Stack Overflow

linux command-line grep wget
Rectangle 27 0

wget -O /dev/null <resource>
\([0-9.]\+ [KM]B/s\)
stderr
stdout
923 KB/s
1.4 MB/s

grep just finds matches. To get the value(s) you can use sed instead:

wget -O /dev/null http://example.com/index.html 2>&1 |
    sed -e 's|^.*(\([0-9.]\+ [KM]B/s\)).*$|\1|'

For me, it returns the whole line that contains the speed. But how do I get only what's inside the brackets? e.g. 923 KB/s or 1.4 MB/s?

Now, the example with sed does work - it extracts the speed (923 KB/s), but before that it prints out all the other output as well.

wget abc | grep ghi | sed xyz

linux - How to grep download speed from wget output? - Stack Overflow

linux command-line grep wget
Rectangle 27 0

here's suggestion. You can make use of wget's --limit-rate=amount option. For example,

--limit-rate=400k will limit the retrieval rate to 400KB/s. Then its easier for you to calculate the total speed. Saves you time and mental anguish trying to regex it.

linux - How to grep download speed from wget output? - Stack Overflow

linux command-line grep wget