Re: devops/clusterSSH - (offtopic)

Yo Rick!!! You are da mann!!! Owe you a six pac of OJ one of these days!

I completely forgot about the changing IP on the remote client, means that while the ssh key is valid, the request for confirmation will occur for the "known_host" (iirc). So yeah I'll have to mod either the local or remote sshd/ssh config file for that..

As to the initial IP of the cloned droplets, I populate predata with that via the API that's used when generating the set of remote instances. I'll take a look at the tools you mentioned as well.. Thanks much for all, for being another set of "eyes"!! Really do appreciate the comments.!!

On Tue, Nov 15, 2016 at 12:36 PM, Rick Stevens <ricks(a)alldigital.com&gt; wrote: > On 11/15/2016 07:39 AM, bruce wrote: >> Hey! >> >> --This is offtopic from fed -- >> >> Thanks to all who've commented on the SSH/screen issue! Think I got >> that part resolved. >> >> Now, in order to use the ipAddresses that have been generated for the >> droplets, need to get opinions/thoughts on apps to use to handle >> connecting to the different numbers of droplets/instances for the >> project. >> >> The following items are a kind of rough overview of what we're >> thinking needs to be done. >> >> We're looking at clusterSSH (even though it appears to need to be >> built from source!). Other thoughts/opinions on apps that might >> suffice would be good. > > ClusterSSH is available from the standard repos, no need to build it: > > [root@prophead ~]# dnf list clusterssh* > Last metadata expiration check: 2:18:20 ago on Tue Nov 15 06:48:02 2016. > Installed Packages > clusterssh.noarch 4.08-1.fc24 @updates > >> Any tool we use, will need to initially handle ~50 droplets/instances, >> and scale to ~500-1000 >> >> >> The tool should: >> - be able to fire up group(s) of servers based on ipAddress (type of >> droplet -fetch/parse/etc..) >> -be able to generate cmd to single/multiple >> -be able to display either single/group of terms based on ipAddress >> (single/group) >> -be able to group the term(ipAddress) -- single term/multiple term in >> the group(s) >> -be able to switch between the groups, which in turn display the terms >> for the group > > cssh can do all of that, but once launched in interactive mode, you > can't "switch between the groups". If you use the /etc/clusters > mechanism, e.g. "cssh group-from-cluster-file", then the terminals for > all machines in that cluster will be displayed on your workspace. You > could have multiple workspaces and run a cssh for a different cluster on > each one and switch between workspaces. We do that a lot. > >> The use case... >> -the crawler spins up a number of droplets/instances >> -the crawler generates the required ipAddresses and "groups" the >> ipAddresses, based on fetch/parse use > > Uhm, if the droplets get different IPs each time, one of the first > things I'd have them do is figure out what their IP is and report it > to a database or something so it's known by your app. > >> -All the "instances' are generated/cloned to have the required apps on >> the server in order to be fetch or parse - there's no need to >> upload/scp files to the remote instances > > This is outside the spec of a terminal manager. > >> Tool Requirements: >> -Nice if the termManagerApp is able to use config/xternal files to >> handle the ipAddresses to create groups as required >> -Nice if the termManagerApp is able to display terms in a given group >> -App has to handle external pub/priv key, all terms (cloned) have the same key > > That's a function of the access method. If the droplets regenerate ssh > keys and such each time they spin up, you'll have issues since the > ssh client will see a new host key each time you spin up a droplet. > >> The termManagerApp needs to be able to display terms from the selected group >> >> TermManagerApp needs to be able to send same command to all terms in >> the displayed group >> TermManagerApp needs to be able to select a given term, and insert >> cmds to that term only >> Terms being displayed, should display "realtime" window update > > cssh can do that. > >> Nice if termManagerApp can display 20-50 terms simultaneously > > With cssh, this is a function of your screen and font sizes. You'll > need a big screen and relatively small fonts. Too small for these old > eyes of mine. > >> Basically, the tool/app will be used to allow the project to manage >> the multiple instances/VMs that are being run for the crawl. >> >> Project use functions requires: >> -Running commands on remote servers >> -Checking on the progress of the remote processes via the screen function >> -Starting up/Shutting Down remote servers as needed >> -TBD.. > > You may also want to look at pdsh/dshbak (available from the repos) to > send commands to remote machines. pdsh doesn't open windows as cssh > does, but sends commands and receives the responses back. The responses > are fed to dshbak to sort them into host order so they're not all > jumbled together. Other tools to look at are ansible and puppet (both > available from the repos) and there are others I haven't mentioned. All > of them have their uses and strengths and weaknesses and there really > isn't any one, single "best" solution. If there were, there wouldn't be > this plethora of tools. I think you'll find you use several, depending > on exactly what you're trying to accomplish at that particular time. > > Note that the issues with ssh keys and such will rear their heads > regardless of what tool(s) you use. If you want password-less access, > each droplet needs to use the same ssh hostkey each time (and ideally > the same IP) or you'll need to set the /etc/ssh/ssh_config or > ~/.ssh/config file's "StrictHostKeyChecking" to "no" or "ask" (not > necessarily a good thing--it depends on your level of paranoia). > ---------------------------------------------------------------------- > - Rick Stevens, Systems Engineer, AllDigital ricks(a)alldigital.com - > - AIM/Skype: therps2 ICQ: 226437340 Yahoo: origrps2 - > - - > - Heisenberg _may_ have slept here - > ---------------------------------------------------------------------- > _______________________________________________ > users mailing list -- users(a)lists.fedoraproject.org > To unsubscribe send an email to users-leave(a)lists.fedoraproject.org _______________________________________________ users mailing list -- users(a)lists.fedoraproject.org To unsubscribe send an email to users-leave(a)lists.fedoraproject.org