Re: [CentOS] parallel bash scripts
by Mark LaPierre
I solved a similar problem by installing gnu parallel on my system.
It did everything that I wanted, and better than I would have coded.
Ali
On Tue, Mar 27, 2012 at 6:37 PM, Mark LaPierre <marklapier(a)aol.com> wrote:
> Check out the redirection at the end of each command. 1>&2 redirects
> the standard out of your child command to the standard error which then
> appears in the parent shell. At the end the last & launches your
> command into a background shell and then moves on to launch the next
> command. The redirections don't care if the command ever terminates.
>
> The result is that both commands are launched and the parent shell
> terminates leaving the standard error attached to the terminal that the
> parent was launched in.
>
> On 03/27/2012 09:08 PM, bruce wrote:
> > marklapier(a)aol.com
> >
> > hey mark....
> >
> > what you have, appears to be pretty close to what i had... except my
> > tests never ended... the loops are infinite...
> >
> > can i do a fpaste and have you take a look at what i have?
> >
> > -btuce
> >
> >
> > On Tue, Mar 27, 2012 at 9:00 PM, Mark LaPierre<marklapier(a)aol.com>
> wrote:
> >> On 03/27/2012 05:25 PM, bruce wrote:
> >>>
> >>> hi.
> >>>
> >>> got a couple of test bash scripts.
> >>>
> >>> dog.sh, cat.sh
> >>> each script runs the underlying php in an endless loop.
> >>>
> >>> I'm trying to figure out how to run the scripts in parallel, from the
> >>> same parent shell script. something like:
> >>>
> >>> test.sh
> >>>
> >>>
> >>> where dog.sh would be :
> >>> --------------------------------
> >>> while true
> >>> do
> >>> pgrep dog
> >>> if [ $? -ne 0 ]
> >>> then
> >>> /dog.php
> >>> fi
> >>> sleep 5
> >>> done
> >>>
> >>> my current tests, run dog.sh, which runs the dog.php ... but the test
> >>> never gets to run cat.sh
> >>>
> >>> thoughts/comments...
> >>>
> >>> thanks
> >>
> >> Hey Bruce,
> >>
> >> Do you mean to run these subprograms in parallel or in series?
> >>
> >> cat.sh
> >> #! /bin/bash
> >>
> >> CAT=0
> >> until [ $CAT -eq 10 ]
> >> do
> >> echo "Inside a dog it's too dark to read. $CAT"
> >> CAT=$[$CAT + 1]
> >> sleep 2
> >> done
> >>
> >>
> >> dog.sh
> >> #! /bin/bash
> >>
> >> DOG=0
> >> until [ $DOG -eq 10 ]
> >> do
> >> echo "Next to a dog a book is man's best friend. $DOG"
> >> DOG=$[$DOG + 1]
> >> sleep 2
> >> done
> >>
> >>
> >> test.sh
> >> #! /bin/sh
> >>
> >> /home/mlapier/test/dog.sh 1>&2&
> >> /home/mlapier/test/cat.sh 1>&2&
> >>
> >>
> >> [mlapier@mushroom test]$ ./test.sh
> >> [mlapier@mushroom test]$ Next to a dog a book is man's best friend. 0
> >> Inside a dog it's too dark to read. 0
> >> Next to a dog a book is man's best friend. 1
> >> Inside a dog it's too dark to read. 1
> >> Next to a dog a book is man's best friend. 2
> >> Inside a dog it's too dark to read. 2
> >> Next to a dog a book is man's best friend. 3
> >> Inside a dog it's too dark to read. 3
> >> Next to a dog a book is man's best friend. 4
> >> Inside a dog it's too dark to read. 4
> >> Next to a dog a book is man's best friend. 5
> >> Inside a dog it's too dark to read. 5
> >> Next to a dog a book is man's best friend. 6
> >> Inside a dog it's too dark to read. 6
> >> Next to a dog a book is man's best friend. 7
> >> Inside a dog it's too dark to read. 7
> >> Next to a dog a book is man's best friend. 8
> >> Inside a dog it's too dark to read. 8
> >> Next to a dog a book is man's best friend. 9
> >> Inside a dog it's too dark to read. 9
> >>
> >>
> >>
> >> --
> >> users mailing list
> >> users(a)lists.fedoraproject.org
> >> To unsubscribe or change subscription options:
> >> https://admin.fedoraproject.org/mailman/listinfo/users
> >> Guidelines: http://fedoraproject.org/wiki/Mailing_list_guidelines
> >> Have a question? Ask away: http://ask.fedoraproject.org
>
> _______________________________________________
> CentOS mailing list
> CentOS(a)centos.org
> http://lists.centos.org/mailman/listinfo/centos
>
_______________________________________________
CentOS mailing list
CentOS(a)centos.org
http://lists.centos.org/mailman/listinfo/centos
12 years, 1 month
Mouse sensitivity suddenly changed
by Bill Davidsen
This is a really odd one, I have a keyboard and PS/2 mouse on a KVM
switch, accessing four Linux system. All of a sudden the mouse
sensitivity changed to "dead slow" on one machine. I checked the
settings in preferences, and tried (a) another account and later (b)
KDE, mouse is slow in all cases. Sensitivity is high on the other three
machines.
FC13, GNOME2 (hasn't changed in ages, it's a host for five VMs
(i7-950+24GB RAM). Really annoying, can anyone suggest a cause?
--
Bill Davidsen <davidsen(a)tmr.com>
"We have more to fear from the bungling of the incompetent than from
the machinations of the wicked." - from Slashdot
12 years, 1 month
floppy
by Patrick Dupre
Hello,
man I make a mount /dev/fd0 /mnt/floppy -t msdos
every thing seems OK (no complain), but it is just not mounted:
df does not show it and I can redo the command again and again!
mformat A: works fine!
Thank for your help.
--
---
==========================================================================
Patrick DUPRÉ | |
Department of Chemistry | | Phone: (44)-(0)-1904-434384
The University of York | | Fax: (44)-(0)-1904-432516
Heslington | |
York YO10 5DD United Kingdom | | email: patrick.dupre(a)york.ac.uk
==========================================================================
12 years, 1 month
Re: parallel bash scripts
by Mark LaPierre
On 03/27/2012 10:35 PM, bruce wrote:
> Hi mark.
>
> I have a series of php apps that are async, and modify different tbls
> within a mysql db. The php apps also perform other tasks as well.
>
> The process needs to be able to periodically run the php apps. The
> shell scripts are used to run the individual php apps. During testing,
> I wanted to be able to run the apps, in a background manner, as well
> as display the output for debugging.
>
> The system has ~5-10 php apps that will be running.
>
> The overall goal for all of this is part of a targeted web crawler,
> where the php apps perform backend processing to manage the data from
> the crawling process as it gets stored in the database.
>
> So the process should fire up all of the processes, and they should
> then run every X secs... At least thats the way I ultimately want it
> to work. But I'm unable to figure out how to do this, and at the same
> time, display the output for the debugging process...
>
> Hopefully I've done a better job of explaining some of what this is for..
>
>
>
> On Tue, Mar 27, 2012 at 10:23 PM, Mark LaPierre<marklapier(a)aol.com> wrote:
>> I don't understand what you are trying to do.
>>
>> Do you want to run these commands in series? Run one, wait for it to
>> complete, then start the other?
>>
>> If that's where you want to go then leave off the trailing ampersand but
>> keep the redirection. The output of the first program will appear in the
>> terminal due to the redirection. When the first program ends the second
>> program will start. That one needs redirection of standard out to standard
>> error too so that it's output will appear in the terminal as well.
>>
>> Solution with trailing ampersand is appropriate for endless loops.
>> Solution without trailing ampersand is appropriate for serial operation.
>>
>> I put the loop counter in the demo program just to stop the thing without
>> having to us ps and kill. Without the loop counter they will continue to
>> print inter-twined messages forever.
>>
>> Does the program webservice_setup.sh start programs itself that you are
>> trying to read the output from? Are they correctly redirected to send their
>> standard output to standard error?
>>
>>
>> On 03/27/2012 10:01 PM, bruce wrote:
>>>
>>> Hi mark.
>>>
>>> the following is part of what i have tried... and what isn't working.
>>> i changed the test shell scripts to no longer have infinite loops.
>>> they simply execute the php app one time.
>>>
>>> however, it still doesn't work.
>>>
>>> in the initial shell script, it calls a php app that does some mysql
>>> processes. the php app has a number of echo/print msgs to track what's
>>> going on for testing.. when running as a background process the shell
>>> script/php app isn't working.
>>>
>>> the cmdline sits.. when i select the "return" key.. the process is
>>> diplayed as being stopped...
>>>
>>> if i remove the background "&" the shell script runs...
>>>
>>> #### -- not working
>>> /apps/parseapp2/webservice_setup.sh 1>&2&
>>>
>>> #### -- works
>>> /apps/parseapp2/webservice_setup.sh
>>>
>>> /apps/parseapp2/webservice_setup.sh
>>> -------------------------------
>>> echo "starting the osu webservice test"
>>> mysql -uroot1 -pfoo< /apps/parseapp2/setup_tblDb.sql
>>> mysql -uroot1 -pfoo< /apps/parseapp2/webservice/webservice_tbls.sql
>>> /apps/parseapp2/parse_tbl_setup.php
>>> echo "started the osu webservice test"
>>>
>>>
>>> thoughts???
>>>
>>> thanks
>>>
>>>
>>>
>>>
>>>
>>> On Tue, Mar 27, 2012 at 9:37 PM, Mark LaPierre<marklapier(a)aol.com> wrote:
>>>>
>>>> Check out the redirection at the end of each command. 1>&2 redirects the
>>>> standard out of your child command to the standard error which then
>>>> appears
>>>> in the parent shell. At the end the last& launches your command into a
>>>>
>>>> background shell and then moves on to launch the next command. The
>>>> redirections don't care if the command ever terminates.
>>>>
>>>> The result is that both commands are launched and the parent shell
>>>> terminates leaving the standard error attached to the terminal that the
>>>> parent was launched in.
>>>>
>>>>
>>>> On 03/27/2012 09:08 PM, bruce wrote:
>>>>>
>>>>>
>>>>> marklapier(a)aol.com
>>>>>
>>>>> hey mark....
>>>>>
>>>>> what you have, appears to be pretty close to what i had... except my
>>>>> tests never ended... the loops are infinite...
>>>>>
>>>>> can i do a fpaste and have you take a look at what i have?
>>>>>
>>>>> -btuce
>>>>>
>>>>>
>>>>> On Tue, Mar 27, 2012 at 9:00 PM, Mark LaPierre<marklapier(a)aol.com>
>>>>> wrote:
>>>>>>
>>>>>>
>>>>>> On 03/27/2012 05:25 PM, bruce wrote:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> hi.
>>>>>>>
>>>>>>> got a couple of test bash scripts.
>>>>>>>
>>>>>>> dog.sh, cat.sh
>>>>>>> each script runs the underlying php in an endless loop.
>>>>>>>
>>>>>>> I'm trying to figure out how to run the scripts in parallel, from the
>>>>>>> same parent shell script. something like:
>>>>>>>
>>>>>>> test.sh
>>>>>>>
>>>>>>>
>>>>>>> where dog.sh would be :
>>>>>>> --------------------------------
>>>>>>> while true
>>>>>>> do
>>>>>>> pgrep dog
>>>>>>> if [ $? -ne 0 ]
>>>>>>> then
>>>>>>> /dog.php
>>>>>>> fi
>>>>>>> sleep 5
>>>>>>> done
>>>>>>>
>>>>>>> my current tests, run dog.sh, which runs the dog.php ... but the test
>>>>>>> never gets to run cat.sh
>>>>>>>
>>>>>>> thoughts/comments...
>>>>>>>
>>>>>>> thanks
>>>>>>
>>>>>>
>>>>>>
>>>>>> Hey Bruce,
>>>>>>
>>>>>> Do you mean to run these subprograms in parallel or in series?
>>>>>>
>>>>>> cat.sh
>>>>>> #! /bin/bash
>>>>>>
>>>>>> CAT=0
>>>>>> until [ $CAT -eq 10 ]
>>>>>> do
>>>>>> echo "Inside a dog it's too dark to read. $CAT"
>>>>>> CAT=$[$CAT + 1]
>>>>>> sleep 2
>>>>>> done
>>>>>>
>>>>>>
>>>>>> dog.sh
>>>>>> #! /bin/bash
>>>>>>
>>>>>> DOG=0
>>>>>> until [ $DOG -eq 10 ]
>>>>>> do
>>>>>> echo "Next to a dog a book is man's best friend. $DOG"
>>>>>> DOG=$[$DOG + 1]
>>>>>> sleep 2
>>>>>> done
>>>>>>
>>>>>>
>>>>>> test.sh
>>>>>> #! /bin/sh
>>>>>>
>>>>>> /home/mlapier/test/dog.sh 1>&2&
>>>>>> /home/mlapier/test/cat.sh 1>&2&
>>>>>>
>>>>>>
>>>>>> [mlapier@mushroom test]$ ./test.sh
>>>>>> [mlapier@mushroom test]$ Next to a dog a book is man's best friend. 0
>>>>>> Inside a dog it's too dark to read. 0
>>>>>> Next to a dog a book is man's best friend. 1
>>>>>> Inside a dog it's too dark to read. 1
>>>>>> Next to a dog a book is man's best friend. 2
>>>>>> Inside a dog it's too dark to read. 2
>>>>>> Next to a dog a book is man's best friend. 3
>>>>>> Inside a dog it's too dark to read. 3
>>>>>> Next to a dog a book is man's best friend. 4
>>>>>> Inside a dog it's too dark to read. 4
>>>>>> Next to a dog a book is man's best friend. 5
>>>>>> Inside a dog it's too dark to read. 5
>>>>>> Next to a dog a book is man's best friend. 6
>>>>>> Inside a dog it's too dark to read. 6
>>>>>> Next to a dog a book is man's best friend. 7
>>>>>> Inside a dog it's too dark to read. 7
>>>>>> Next to a dog a book is man's best friend. 8
>>>>>> Inside a dog it's too dark to read. 8
>>>>>> Next to a dog a book is man's best friend. 9
>>>>>> Inside a dog it's too dark to read. 9
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> users mailing list
>>>>>> users(a)lists.fedoraproject.org
>>>>>> To unsubscribe or change subscription options:
>>>>>> https://admin.fedoraproject.org/mailman/listinfo/users
>>>>>> Guidelines: http://fedoraproject.org/wiki/Mailing_list_guidelines
>>>>>> Have a question? Ask away: http://ask.fedoraproject.org
>>>>
>>>>
>>>>
>>
12 years, 1 month
firefox 11 -- keyboard popup
by Geoffrey Leach
Firefox 11 appears to default to an on-screen keyboard that pops up
whenever I attempt to type in a box. I'm unable to dismiss it.
Any suggestions on how to get rid of this very annoying feature?
12 years, 1 month
[Fwd: Re: Processor time consumption]
by Howard Howell
-------- Forwarded Message --------
> From: Joe Zeff <joe(a)zeff.us>
> To: hlhowell(a)pacbell.net
> Subject: Re: Processor time consumption
> Date: Wed, 28 Mar 2012 10:26:16 -0700
>
> On 03/28/2012 09:50 AM, les wrote:
> > I don't think it is RAM space being consumed. System Monitor shows very
> > little memory in use, and very little network processing.
>
> OK, that's one blind alley eliminated. However, this should have gone
> to the list so that everybody knew about it.
I mistakenly sent this only to Joe, but it applies to the thread.
Regards,
Les H
12 years, 1 month
Fwd: Re: parallel bash scripts
by Mark LaPierre
Hi mark.
I have a series of php apps that are async, and modify different tbls
within a mysql db. The php apps also perform other tasks as well.
The process needs to be able to periodically run the php apps. The
shell scripts are used to run the individual php apps. During testing,
I wanted to be able to run the apps, in a background manner, as well
as display the output for debugging.
The system has ~5-10 php apps that will be running.
The overall goal for all of this is part of a targeted web crawler,
where the php apps perform backend processing to manage the data from
the crawling process as it gets stored in the database.
So the process should fire up all of the processes, and they should
then run every X secs... At least thats the way I ultimately want it
to work. But I'm unable to figure out how to do this, and at the same
time, display the output for the debugging process...
Hopefully I've done a better job of explaining some of what this is for..
On Tue, Mar 27, 2012 at 10:23 PM, Mark LaPierre <marklapier(a)aol.com> wrote:
> I don't understand what you are trying to do.
>
> Do you want to run these commands in series? Run one, wait for it to
> complete, then start the other?
>
> If that's where you want to go then leave off the trailing ampersand but
> keep the redirection. The output of the first program will appear in the
> terminal due to the redirection. When the first program ends the second
> program will start. That one needs redirection of standard out to standard
> error too so that it's output will appear in the terminal as well.
>
> Solution with trailing ampersand is appropriate for endless loops.
> Solution without trailing ampersand is appropriate for serial operation.
>
> I put the loop counter in the demo program just to stop the thing without
> having to us ps and kill. Without the loop counter they will continue to
> print inter-twined messages forever.
>
> Does the program webservice_setup.sh start programs itself that you are
> trying to read the output from? Are they correctly redirected to send their
> standard output to standard error?
>
>
> On 03/27/2012 10:01 PM, bruce wrote:
>>
>> Hi mark.
>>
>> the following is part of what i have tried... and what isn't working.
>> i changed the test shell scripts to no longer have infinite loops.
>> they simply execute the php app one time.
>>
>> however, it still doesn't work.
>>
>> in the initial shell script, it calls a php app that does some mysql
>> processes. the php app has a number of echo/print msgs to track what's
>> going on for testing.. when running as a background process the shell
>> script/php app isn't working.
>>
>> the cmdline sits.. when i select the "return" key.. the process is
>> diplayed as being stopped...
>>
>> if i remove the background "&" the shell script runs...
>>
>> #### -- not working
>> /apps/parseapp2/webservice_setup.sh 1>&2&
>>
>> #### -- works
>> /apps/parseapp2/webservice_setup.sh
>>
>> /apps/parseapp2/webservice_setup.sh
>> -------------------------------
>> echo "starting the osu webservice test"
>> mysql -uroot1 -pfoo< /apps/parseapp2/setup_tblDb.sql
>> mysql -uroot1 -pfoo< /apps/parseapp2/webservice/webservice_tbls.sql
>> /apps/parseapp2/parse_tbl_setup.php
>> echo "started the osu webservice test"
>>
>>
>> thoughts???
>>
>> thanks
>>
>>
>>
>>
>>
>> On Tue, Mar 27, 2012 at 9:37 PM, Mark LaPierre<marklapier(a)aol.com> wrote:
>>>
>>> Check out the redirection at the end of each command. 1>&2 redirects the
>>> standard out of your child command to the standard error which then
>>> appears
>>> in the parent shell. At the end the last& launches your command into a
>>>
>>> background shell and then moves on to launch the next command. The
>>> redirections don't care if the command ever terminates.
>>>
>>> The result is that both commands are launched and the parent shell
>>> terminates leaving the standard error attached to the terminal that the
>>> parent was launched in.
>>>
>>>
>>> On 03/27/2012 09:08 PM, bruce wrote:
>>>>
>>>>
>>>> marklapier(a)aol.com
>>>>
>>>> hey mark....
>>>>
>>>> what you have, appears to be pretty close to what i had... except my
>>>> tests never ended... the loops are infinite...
>>>>
>>>> can i do a fpaste and have you take a look at what i have?
>>>>
>>>> -btuce
>>>>
>>>>
>>>> On Tue, Mar 27, 2012 at 9:00 PM, Mark LaPierre<marklapier(a)aol.com>
>>>> wrote:
>>>>>
>>>>>
>>>>> On 03/27/2012 05:25 PM, bruce wrote:
>>>>>>
>>>>>>
>>>>>>
>>>>>> hi.
>>>>>>
>>>>>> got a couple of test bash scripts.
>>>>>>
>>>>>> dog.sh, cat.sh
>>>>>> each script runs the underlying php in an endless loop.
>>>>>>
>>>>>> I'm trying to figure out how to run the scripts in parallel, from the
>>>>>> same parent shell script. something like:
>>>>>>
>>>>>> test.sh
>>>>>>
>>>>>>
>>>>>> where dog.sh would be :
>>>>>> --------------------------------
>>>>>> while true
>>>>>> do
>>>>>> pgrep dog
>>>>>> if [ $? -ne 0 ]
>>>>>> then
>>>>>> /dog.php
>>>>>> fi
>>>>>> sleep 5
>>>>>> done
>>>>>>
>>>>>> my current tests, run dog.sh, which runs the dog.php ... but the test
>>>>>> never gets to run cat.sh
>>>>>>
>>>>>> thoughts/comments...
>>>>>>
>>>>>> thanks
>>>>>
>>>>>
>>>>>
>>>>> Hey Bruce,
>>>>>
>>>>> Do you mean to run these subprograms in parallel or in series?
>>>>>
>>>>> cat.sh
>>>>> #! /bin/bash
>>>>>
>>>>> CAT=0
>>>>> until [ $CAT -eq 10 ]
>>>>> do
>>>>> echo "Inside a dog it's too dark to read. $CAT"
>>>>> CAT=$[$CAT + 1]
>>>>> sleep 2
>>>>> done
>>>>>
>>>>>
>>>>> dog.sh
>>>>> #! /bin/bash
>>>>>
>>>>> DOG=0
>>>>> until [ $DOG -eq 10 ]
>>>>> do
>>>>> echo "Next to a dog a book is man's best friend. $DOG"
>>>>> DOG=$[$DOG + 1]
>>>>> sleep 2
>>>>> done
>>>>>
>>>>>
>>>>> test.sh
>>>>> #! /bin/sh
>>>>>
>>>>> /home/mlapier/test/dog.sh 1>&2&
>>>>> /home/mlapier/test/cat.sh 1>&2&
>>>>>
>>>>>
>>>>> [mlapier@mushroom test]$ ./test.sh
>>>>> [mlapier@mushroom test]$ Next to a dog a book is man's best friend. 0
>>>>> Inside a dog it's too dark to read. 0
>>>>> Next to a dog a book is man's best friend. 1
>>>>> Inside a dog it's too dark to read. 1
>>>>> Next to a dog a book is man's best friend. 2
>>>>> Inside a dog it's too dark to read. 2
>>>>> Next to a dog a book is man's best friend. 3
>>>>> Inside a dog it's too dark to read. 3
>>>>> Next to a dog a book is man's best friend. 4
>>>>> Inside a dog it's too dark to read. 4
>>>>> Next to a dog a book is man's best friend. 5
>>>>> Inside a dog it's too dark to read. 5
>>>>> Next to a dog a book is man's best friend. 6
>>>>> Inside a dog it's too dark to read. 6
>>>>> Next to a dog a book is man's best friend. 7
>>>>> Inside a dog it's too dark to read. 7
>>>>> Next to a dog a book is man's best friend. 8
>>>>> Inside a dog it's too dark to read. 8
>>>>> Next to a dog a book is man's best friend. 9
>>>>> Inside a dog it's too dark to read. 9
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> users mailing list
>>>>> users(a)lists.fedoraproject.org
>>>>> To unsubscribe or change subscription options:
>>>>> https://admin.fedoraproject.org/mailman/listinfo/users
>>>>> Guidelines: http://fedoraproject.org/wiki/Mailing_list_guidelines
>>>>> Have a question? Ask away: http://ask.fedoraproject.org
>>>
>>>
>>>
>
12 years, 1 month
