On 04/15/2012 01:52 PM, Amadeus W.M. wrote:
The real code is like this:
#!/bin/bash
for url in $(cat myURLs) do curl -s $url & done
I pipe the combined curl outputs to a program that parses the html and keeps track of something (I do pipe afterall). I could do that serially (without &), but parallel is better. I'm only spewing out some 20 network requests simultaneously and so far no warnings from verizon. I'm guessing if I do 1000, say, I might set off some alarms. But that's another problem.
I don't know what tracking your other program is doing...but couldn't you simply do....
curl -s $url | program &