On Thu, Aug 30, 2012 at 11:32:02AM +0800, Xu He Jie wrote: > Hi, > > I submited a patch for text-based console > http://gerrit.ovirt.org/#/c/7165/ > > the issue I want to discussing as below: > 1. fix port VS dynamic port > > Use fix port for all VM's console. connect console with 'ssh > vmUUID@ip -p port'. > Distinguishing VM by vmUUID. > > > The current implement was vdsm will allocated port for console > dynamically and spawn sub-process when VM creating. > In sub-process the main thread responsible for accept new connection > and dispatch output of console to each connection. > When new connection is coming, main processing create new thread for > each new connection. Dynamic port will allocated > port for each VM and use range port. It isn't good for firewall rules. > > > so I got a suggestion that use fix port. and connect console with > 'ssh vmuuid@hostip -p fixport'. this is simple for user. > We need one process for accept new connection from fix port and when > new connection is coming, spawn sub-process for each vm. > But because the console only can open by one process, main process > need responsible for dispatching console's output of all vms and all > connection. > So the code will be a little complex then dynamic port. > > So this is dynamic port VS fix port and simple code VS complex code. >From a usability point of view, I think the fixed port suggestion is nicer. This means that a system administrator needs only to open one port to enable remote console access. If your initial implementation limits console access to one connection per VM would that simplify the code?

On Thu, Aug 30, 2012 at 04:26:31PM -0500, Adam Litke wrote: > On Thu, Aug 30, 2012 at 11:32:02AM +0800, Xu He Jie wrote: >> Hi, >> >> I submited a patch for text-based console >> http://gerrit.ovirt.org/#/c/7165/ >> >> the issue I want to discussing as below: >> 1. fix port VS dynamic port >> >> Use fix port for all VM's console. connect console with 'ssh >> vmUUID@ip -p port'. >> Distinguishing VM by vmUUID. >> >> >> The current implement was vdsm will allocated port for console >> dynamically and spawn sub-process when VM creating. >> In sub-process the main thread responsible for accept new connection >> and dispatch output of console to each connection. >> When new connection is coming, main processing create new thread for >> each new connection. Dynamic port will allocated >> port for each VM and use range port. It isn't good for firewall rules. >> >> >> so I got a suggestion that use fix port. and connect console with >> 'ssh vmuuid@hostip -p fixport'. this is simple for user. >> We need one process for accept new connection from fix port and when >> new connection is coming, spawn sub-process for each vm. >> But because the console only can open by one process, main process >> need responsible for dispatching console's output of all vms and all >> connection. >> So the code will be a little complex then dynamic port. >> >> So this is dynamic port VS fix port and simple code VS complex code. > >From a usability point of view, I think the fixed port suggestion is nicer. > This means that a system administrator needs only to open one port to enable > remote console access. If your initial implementation limits console access to > one connection per VM would that simplify the code? Yes, using a fixed port for all consoles of all VMs seems like a cooler idea. Besides the firewall issue, there's user experience: instead of calling getVmStats to tell the vm port, and then use ssh, only one ssh call is needed. (Taking this one step further - it would make sense to add another layer on top, directing console clients to the specific host currently running the Vm.) I did not take a close look at your implementation, and did not research this myself, but have you considered using sshd for this? I suppose you can configure sshd to collect the list of known "users" from `getAllVmStats`, and force it to run a command that redirects VM's console to the ssh client. It has a potential of being a more robust implementation.

We may want to start thinking about migration. It would be great if we could have a smart console client that connects to the source and destination consoles, and moves to the destination on-line, without loosing a character.

Regards, Dan.

----- Original Message ----- > From: "Xu He Jie" <xuhj(a)linux.vnet.ibm.com&gt; > To: "Dan Kenigsberg" <danken(a)redhat.com&gt; > Cc: "VDSM Project Development" <vdsm-devel(a)lists.fedorahosted.org&gt; > Sent: Tuesday, 4 September, 2012 10:05:37 AM > Subject: Re: [vdsm] [RFC]about the implement of text-based console > > On 09/03/2012 10:33 PM, Dan Kenigsberg wrote: >> On Thu, Aug 30, 2012 at 04:26:31PM -0500, Adam Litke wrote: >>> On Thu, Aug 30, 2012 at 11:32:02AM +0800, Xu He Jie wrote: >>>> Hi, >>>> >>>> I submited a patch for text-based console >>>> http://gerrit.ovirt.org/#/c/7165/ >>>> >>>> the issue I want to discussing as below: >>>> 1. fix port VS dynamic port >>>> >>>> Use fix port for all VM's console. connect console with 'ssh >>>> vmUUID@ip -p port'. >>>> Distinguishing VM by vmUUID. >>>> >>>> >>>> The current implement was vdsm will allocated port for console >>>> dynamically and spawn sub-process when VM creating. >>>> In sub-process the main thread responsible for accept new >>>> connection >>>> and dispatch output of console to each connection. >>>> When new connection is coming, main processing create new thread >>>> for >>>> each new connection. Dynamic port will allocated >>>> port for each VM and use range port. It isn't good for firewall >>>> rules. >>>> >>>> >>>> so I got a suggestion that use fix port. and connect console >>>> with >>>> 'ssh vmuuid@hostip -p fixport'. this is simple for user. >>>> We need one process for accept new connection from fix port and >>>> when >>>> new connection is coming, spawn sub-process for each vm. >>>> But because the console only can open by one process, main >>>> process >>>> need responsible for dispatching console's output of all vms and >>>> all >>>> connection. >>>> So the code will be a little complex then dynamic port. >>>> >>>> So this is dynamic port VS fix port and simple code VS complex >>>> code. >>> >From a usability point of view, I think the fixed port suggestion >>>> is nicer. >>> This means that a system administrator needs only to open one port >>> to enable >>> remote console access. If your initial implementation limits >>> console access to >>> one connection per VM would that simplify the code? >> Yes, using a fixed port for all consoles of all VMs seems like a >> cooler >> idea. Besides the firewall issue, there's user experience: instead >> of >> calling getVmStats to tell the vm port, and then use ssh, only one >> ssh >> call is needed. (Taking this one step further - it would make sense >> to >> add another layer on top, directing console clients to the specific >> host >> currently running the Vm.) >> >> I did not take a close look at your implementation, and did not >> research >> this myself, but have you considered using sshd for this? I suppose >> you >> can configure sshd to collect the list of known "users" from >> `getAllVmStats`, and force it to run a command that redirects VM's >> console to the ssh client. It has a potential of being a more >> robust >> implementation. > I have considered using sshd and ssh tunnel. They > can't implement fixed port and share console. Current implement > we can do anything that what we want. > >> >> We may want to start thinking about migration. It would be great if >> we >> could have a smart console client that connects to the source and >> destination consoles, and moves to the destination on-line, without >> loosing a character. > This is interesting. My first thinking is it's easy implement at > client > side. I think we will implement ssh client in webbrowser. Engine will > know the vm was migrated. Engine can tell client reconnect console to > another host. I will try to think about is there any better idea. If we implement this in a web client, we lose the use case of people without a GUI, who really have to use the serial text consoles. If we really need a separate console for every VM, how about we keep a console server as a VM in the system, and that console server will be running sshd, with an open session to every VM. And in order to connect to a VMs serial console, we will actually ssh to this console server VM as a certain console user. The way I see this is once a VM gets started, the console server will create a user/passwd for that VM, and once someone opens an ssh session to the console server as this user, it will automatically connect the ssh session to the console on whatever host the target VM is running on. When the VM stops, the user can be removed.

On 09/04/2012 10:14 AM, Dan Yasny wrote: > > > ----- Original Message ----- >> From: "Xu He Jie" <xuhj(a)linux.vnet.ibm.com&gt; >> To: "Dan Kenigsberg" <danken(a)redhat.com&gt; >> Cc: "VDSM Project Development" <vdsm-devel(a)lists.fedorahosted.org&gt; >> Sent: Tuesday, 4 September, 2012 10:05:37 AM >> Subject: Re: [vdsm] [RFC]about the implement of text-based console >> >> On 09/03/2012 10:33 PM, Dan Kenigsberg wrote: >>> On Thu, Aug 30, 2012 at 04:26:31PM -0500, Adam Litke wrote: >>>> On Thu, Aug 30, 2012 at 11:32:02AM +0800, Xu He Jie wrote: >>>>> Hi, >>>>> >>>>> I submited a patch for text-based console >>>>> http://gerrit.ovirt.org/#/c/7165/ >>>>> >>>>> the issue I want to discussing as below: >>>>> 1. fix port VS dynamic port >>>>> >>>>> Use fix port for all VM's console. connect console with 'ssh >>>>> vmUUID@ip -p port'. >>>>> Distinguishing VM by vmUUID. >>>>> >>>>> >>>>> The current implement was vdsm will allocated port for console >>>>> dynamically and spawn sub-process when VM creating. >>>>> In sub-process the main thread responsible for accept new >>>>> connection >>>>> and dispatch output of console to each connection. >>>>> When new connection is coming, main processing create new thread >>>>> for >>>>> each new connection. Dynamic port will allocated >>>>> port for each VM and use range port. It isn't good for firewall >>>>> rules. >>>>> >>>>> >>>>> so I got a suggestion that use fix port. and connect console >>>>> with >>>>> 'ssh vmuuid@hostip -p fixport'. this is simple for user. >>>>> We need one process for accept new connection from fix port and >>>>> when >>>>> new connection is coming, spawn sub-process for each vm. >>>>> But because the console only can open by one process, main >>>>> process >>>>> need responsible for dispatching console's output of all vms and >>>>> all >>>>> connection. >>>>> So the code will be a little complex then dynamic port. >>>>> >>>>> So this is dynamic port VS fix port and simple code VS complex >>>>> code. >>>> >From a usability point of view, I think the fixed port suggestion >>>>> is nicer. >>>> This means that a system administrator needs only to open one port >>>> to enable >>>> remote console access. If your initial implementation limits >>>> console access to >>>> one connection per VM would that simplify the code? >>> Yes, using a fixed port for all consoles of all VMs seems like a >>> cooler >>> idea. Besides the firewall issue, there's user experience: instead >>> of >>> calling getVmStats to tell the vm port, and then use ssh, only one >>> ssh >>> call is needed. (Taking this one step further - it would make sense >>> to >>> add another layer on top, directing console clients to the specific >>> host >>> currently running the Vm.) >>> >>> I did not take a close look at your implementation, and did not >>> research >>> this myself, but have you considered using sshd for this? I suppose >>> you >>> can configure sshd to collect the list of known "users" from >>> `getAllVmStats`, and force it to run a command that redirects VM's >>> console to the ssh client. It has a potential of being a more >>> robust >>> implementation. >> I have considered using sshd and ssh tunnel. They >> can't implement fixed port and share console. Current implement >> we can do anything that what we want. >> >>> >>> We may want to start thinking about migration. It would be great if >>> we >>> could have a smart console client that connects to the source and >>> destination consoles, and moves to the destination on-line, without >>> loosing a character. >> This is interesting. My first thinking is it's easy implement at >> client >> side. I think we will implement ssh client in webbrowser. Engine will >> know the vm was migrated. Engine can tell client reconnect console to >> another host. I will try to think about is there any better idea. > > If we implement this in a web client, we lose the use case of people > without a GUI, who really have to use the serial text consoles. > > If we really need a separate console for every VM, how about we keep > a console server as a VM in the system, and that console server will > be running sshd, with an open session to every VM. And in order to > connect to a VMs serial console, we will actually ssh to this console > server VM as a certain console user. > > The way I see this is once a VM gets started, the console server will > create a user/passwd for that VM, and once someone opens an ssh > session to the console server as this user, it will automatically > connect the ssh session to the console on whatever host the target VM > is running on. When the VM stops, the user can be removed. i'm less concerned by single or multiple ports, since we already do multiple ports for vnc/spice. the main question to me is can we easily frontend the serial of the VM by an ssh server authenticating based on the SetVmTicket flow (which qemu doesn't support). (well, unless qemu adds builtin ssh support to the serial port)

From: "Xu He Jie" <xuhj(a)linux.vnet.ibm.com&gt; To: "Dan Yasny" <dyasny(a)redhat.com&gt; Cc: "VDSM Project Development" <vdsm-devel(a)lists.fedorahosted.org&gt;, "Dan Kenigsberg" <danken(a)redhat.com&gt; Sent: Tuesday, 4 September, 2012 11:42:04 AM Subject: Re: [vdsm] [RFC]about the implement of text-based console On 09/04/2012 03:14 PM, Dan Yasny wrote: > > ----- Original Message ----- >> From: "Xu He Jie" <xuhj(a)linux.vnet.ibm.com&gt; >> To: "Dan Kenigsberg" <danken(a)redhat.com&gt; >> Cc: "VDSM Project Development" <vdsm-devel(a)lists.fedorahosted.org&gt; >> Sent: Tuesday, 4 September, 2012 10:05:37 AM >> Subject: Re: [vdsm] [RFC]about the implement of text-based console >> >> On 09/03/2012 10:33 PM, Dan Kenigsberg wrote: >>> On Thu, Aug 30, 2012 at 04:26:31PM -0500, Adam Litke wrote: >>>> On Thu, Aug 30, 2012 at 11:32:02AM +0800, Xu He Jie wrote: >>>>> Hi, >>>>> >>>>> I submited a patch for text-based console >>>>> http://gerrit.ovirt.org/#/c/7165/ >>>>> >>>>> the issue I want to discussing as below: >>>>> 1. fix port VS dynamic port >>>>> >>>>> Use fix port for all VM's console. connect console with 'ssh >>>>> vmUUID@ip -p port'. >>>>> Distinguishing VM by vmUUID. >>>>> >>>>> >>>>> The current implement was vdsm will allocated port for >>>>> console >>>>> dynamically and spawn sub-process when VM creating. >>>>> In sub-process the main thread responsible for accept new >>>>> connection >>>>> and dispatch output of console to each connection. >>>>> When new connection is coming, main processing create new >>>>> thread >>>>> for >>>>> each new connection. Dynamic port will allocated >>>>> port for each VM and use range port. It isn't good for firewall >>>>> rules. >>>>> >>>>> >>>>> so I got a suggestion that use fix port. and connect >>>>> console >>>>> with >>>>> 'ssh vmuuid@hostip -p fixport'. this is simple for user. >>>>> We need one process for accept new connection from fix port and >>>>> when >>>>> new connection is coming, spawn sub-process for each vm. >>>>> But because the console only can open by one process, main >>>>> process >>>>> need responsible for dispatching console's output of all vms >>>>> and >>>>> all >>>>> connection. >>>>> So the code will be a little complex then dynamic port. >>>>> >>>>> So this is dynamic port VS fix port and simple code VS >>>>> complex >>>>> code. >>>> >From a usability point of view, I think the fixed port >>>> >suggestion >>>>> is nicer. >>>> This means that a system administrator needs only to open one >>>> port >>>> to enable >>>> remote console access. If your initial implementation limits >>>> console access to >>>> one connection per VM would that simplify the code? >>> Yes, using a fixed port for all consoles of all VMs seems like a >>> cooler >>> idea. Besides the firewall issue, there's user experience: >>> instead >>> of >>> calling getVmStats to tell the vm port, and then use ssh, only >>> one >>> ssh >>> call is needed. (Taking this one step further - it would make >>> sense >>> to >>> add another layer on top, directing console clients to the >>> specific >>> host >>> currently running the Vm.) >>> >>> I did not take a close look at your implementation, and did not >>> research >>> this myself, but have you considered using sshd for this? I >>> suppose >>> you >>> can configure sshd to collect the list of known "users" from >>> `getAllVmStats`, and force it to run a command that redirects >>> VM's >>> console to the ssh client. It has a potential of being a more >>> robust >>> implementation. >> I have considered using sshd and ssh tunnel. They >> can't implement fixed port and share console. Current implement >> we can do anything that what we want. >> >>> We may want to start thinking about migration. It would be great >>> if >>> we >>> could have a smart console client that connects to the source and >>> destination consoles, and moves to the destination on-line, >>> without >>> loosing a character. >> This is interesting. My first thinking is it's easy implement at >> client >> side. I think we will implement ssh client in webbrowser. Engine >> will >> know the vm was migrated. Engine can tell client reconnect console >> to >> another host. I will try to think about is there any better idea. > If we implement this in a web client, we lose the use case of > people without a GUI, who really have to use the serial text > consoles. If we implement it at client, we have engine command line tools for user, and we also can implement it in that tools. > > If we really need a separate console for every VM, how about we > keep a console server as a VM in the system, and that console > server will be running sshd, with an open session to every VM. And > in order to connect to a VMs serial console, we will actually ssh > to this console server VM as a certain console user. Thanks for your idea. But I think it's too heavy. The console server VM will be another centralize management server for managing vm's console. If we use this, we need think about when engine setTicket for vm, how vdsm tell the console server VM the ticket, so there need another communicate method between vm and all vdsm host. And we need think about the guest os that running at vm, it's a customize linux, or fully fedora. And we can't use the text-based console when vdsm running as standalone.

> > The way I see this is once a VM gets started, the console server > will create a user/passwd for that VM, and once someone opens an > ssh session to the console server as this user, it will > automatically connect the ssh session to the console on whatever > host the target VM is running on. When the VM stops, the user can > be removed. > >>> Regards, >>> Dan. >>> >> _______________________________________________ >> vdsm-devel mailing list >> vdsm-devel(a)lists.fedorahosted.org >> https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel >>

On 09/03/2012 10:33 PM, Dan Kenigsberg wrote: >On Thu, Aug 30, 2012 at 04:26:31PM -0500, Adam Litke wrote: >>On Thu, Aug 30, 2012 at 11:32:02AM +0800, Xu He Jie wrote: >>>Hi, >>> >>> I submited a patch for text-based console >>>http://gerrit.ovirt.org/#/c/7165/ >>> >>>the issue I want to discussing as below: >>>1. fix port VS dynamic port >>> >>>Use fix port for all VM's console. connect console with 'ssh >>>vmUUID@ip -p port'. >>>Distinguishing VM by vmUUID. >>> >>> >>> The current implement was vdsm will allocated port for console >>>dynamically and spawn sub-process when VM creating. >>>In sub-process the main thread responsible for accept new connection >>>and dispatch output of console to each connection. >>>When new connection is coming, main processing create new thread for >>>each new connection. Dynamic port will allocated >>>port for each VM and use range port. It isn't good for firewall rules. >>> >>> >>> so I got a suggestion that use fix port. and connect console with >>>'ssh vmuuid@hostip -p fixport'. this is simple for user. >>>We need one process for accept new connection from fix port and when >>>new connection is coming, spawn sub-process for each vm. >>>But because the console only can open by one process, main process >>>need responsible for dispatching console's output of all vms and all >>>connection. >>>So the code will be a little complex then dynamic port. >>> >>> So this is dynamic port VS fix port and simple code VS complex code. >>>From a usability point of view, I think the fixed port suggestion is nicer. >>This means that a system administrator needs only to open one port to enable >>remote console access. If your initial implementation limits console access to >>one connection per VM would that simplify the code? >Yes, using a fixed port for all consoles of all VMs seems like a cooler >idea. Besides the firewall issue, there's user experience: instead of >calling getVmStats to tell the vm port, and then use ssh, only one ssh >call is needed. (Taking this one step further - it would make sense to >add another layer on top, directing console clients to the specific host >currently running the Vm.) > >I did not take a close look at your implementation, and did not research >this myself, but have you considered using sshd for this? I suppose you >can configure sshd to collect the list of known "users" from >`getAllVmStats`, and force it to run a command that redirects VM's >console to the ssh client. It has a potential of being a more robust >implementation. I have considered using sshd and ssh tunnel. They can't implement fixed port and share console.

Current implement we can do anything that what we want.

On Tue, Sep 04, 2012 at 03:05:37PM +0800, Xu He Jie wrote: > On 09/03/2012 10:33 PM, Dan Kenigsberg wrote: > >On Thu, Aug 30, 2012 at 04:26:31PM -0500, Adam Litke wrote: > >>On Thu, Aug 30, 2012 at 11:32:02AM +0800, Xu He Jie wrote: > >>>Hi, > >>> > >>> I submited a patch for text-based console > >>>http://gerrit.ovirt.org/#/c/7165/ > >>> > >>>the issue I want to discussing as below: > >>>1. fix port VS dynamic port > >>> > >>>Use fix port for all VM's console. connect console with 'ssh > >>>vmUUID@ip -p port'. > >>>Distinguishing VM by vmUUID. > >>> > >>> > >>> The current implement was vdsm will allocated port for console > >>>dynamically and spawn sub-process when VM creating. > >>>In sub-process the main thread responsible for accept new connection > >>>and dispatch output of console to each connection. > >>>When new connection is coming, main processing create new thread for > >>>each new connection. Dynamic port will allocated > >>>port for each VM and use range port. It isn't good for firewall rules. > >>> > >>> > >>> so I got a suggestion that use fix port. and connect console with > >>>'ssh vmuuid@hostip -p fixport'. this is simple for user. > >>>We need one process for accept new connection from fix port and when > >>>new connection is coming, spawn sub-process for each vm. > >>>But because the console only can open by one process, main process > >>>need responsible for dispatching console's output of all vms and all > >>>connection. > >>>So the code will be a little complex then dynamic port. > >>> > >>> So this is dynamic port VS fix port and simple code VS complex code. > >>>From a usability point of view, I think the fixed port suggestion is nicer. > >>This means that a system administrator needs only to open one port to enable > >>remote console access. If your initial implementation limits console access to > >>one connection per VM would that simplify the code? > >Yes, using a fixed port for all consoles of all VMs seems like a cooler > >idea. Besides the firewall issue, there's user experience: instead of > >calling getVmStats to tell the vm port, and then use ssh, only one ssh > >call is needed. (Taking this one step further - it would make sense to > >add another layer on top, directing console clients to the specific host > >currently running the Vm.) > > > >I did not take a close look at your implementation, and did not research > >this myself, but have you considered using sshd for this? I suppose you > >can configure sshd to collect the list of known "users" from > >`getAllVmStats`, and force it to run a command that redirects VM's > >console to the ssh client. It has a potential of being a more robust > >implementation. > I have considered using sshd and ssh tunnel. They > can't implement fixed port and share console. Would you elaborate on that? Usually sshd listens to a fixed port 22, and allows multiple users to have independet shells. What do you mean by "share console"? > Current implement > we can do anything that what we want. Yes, it is completely under our control, but there are down sides, too: we have to maintain another process, and another entry point, instead of configuring a universally-used, well maintained and debugged application.

Dan.

on 09/04/2012 22:19, Ryan Harper wrote: >* Dan Kenigsberg <danken(a)redhat.com&gt; [2012-09-04 05:53]: >>On Tue, Sep 04, 2012 at 03:05:37PM +0800, Xu He Jie wrote: >>>On 09/03/2012 10:33 PM, Dan Kenigsberg wrote: >>>>On Thu, Aug 30, 2012 at 04:26:31PM -0500, Adam Litke wrote: >>>>>On Thu, Aug 30, 2012 at 11:32:02AM +0800, Xu He Jie wrote: >>>>>>Hi, >>>>>> >>>>>> I submited a patch for text-based console >>>>>>http://gerrit.ovirt.org/#/c/7165/ >>>>>> >>>>>>the issue I want to discussing as below: >>>>>>1. fix port VS dynamic port >>>>>> >>>>>>Use fix port for all VM's console. connect console with 'ssh >>>>>>vmUUID@ip -p port'. >>>>>>Distinguishing VM by vmUUID. >>>>>> >>>>>> >>>>>> The current implement was vdsm will allocated port for console >>>>>>dynamically and spawn sub-process when VM creating. >>>>>>In sub-process the main thread responsible for accept new connection >>>>>>and dispatch output of console to each connection. >>>>>>When new connection is coming, main processing create new thread for >>>>>>each new connection. Dynamic port will allocated >>>>>>port for each VM and use range port. It isn't good for firewall rules. >>>>>> >>>>>> >>>>>> so I got a suggestion that use fix port. and connect console with >>>>>>'ssh vmuuid@hostip -p fixport'. this is simple for user. >>>>>>We need one process for accept new connection from fix port and when >>>>>>new connection is coming, spawn sub-process for each vm. >>>>>>But because the console only can open by one process, main process >>>>>>need responsible for dispatching console's output of all vms and all >>>>>>connection. >>>>>>So the code will be a little complex then dynamic port. >>>>>> >>>>>> So this is dynamic port VS fix port and simple code VS complex code. >>>>>>From a usability point of view, I think the fixed port suggestion is nicer. >>>>>This means that a system administrator needs only to open one port to enable >>>>>remote console access. If your initial implementation limits console access to >>>>>one connection per VM would that simplify the code? >>>>Yes, using a fixed port for all consoles of all VMs seems like a cooler >>>>idea. Besides the firewall issue, there's user experience: instead of >>>>calling getVmStats to tell the vm port, and then use ssh, only one ssh >>>>call is needed. (Taking this one step further - it would make sense to >>>>add another layer on top, directing console clients to the specific host >>>>currently running the Vm.) >>>> >>>>I did not take a close look at your implementation, and did not research >>>>this myself, but have you considered using sshd for this? I suppose you >>>>can configure sshd to collect the list of known "users" from >>>>`getAllVmStats`, and force it to run a command that redirects VM's >>>>console to the ssh client. It has a potential of being a more robust >>>>implementation. >>>I have considered using sshd and ssh tunnel. They >>>can't implement fixed port and share console. >>Would you elaborate on that? Usually sshd listens to a fixed port 22, >>and allows multiple users to have independet shells. What do you mean by >>"share console"? >> >>>Current implement >>>we can do anything that what we want. >>Yes, it is completely under our control, but there are down sides, too: >>we have to maintain another process, and another entry point, instead of >>configuring a universally-used, well maintained and debugged >>application. >Think of the security implications of having another remote shell >access point to a host. I'd much rather trust sshd if we can make it >work. > > >>Dan. At first glance, the standard sshd on the host is stronger and more robust than a custom ssh server, but the risk using the host sshd is high. If we implement this feature via host ssd, when a hacker attacks the sshd successfully, he will get access to the host shell. After all, the custom ssh server is not for accessing host shell, but just for forwarding the data from the guest console (a host /dev/pts/X device). If we just use a custom ssh server, the code in this server only does 1. auth, 2. data forwarding, when the hacker attacks, he just gets access to that virtual machine. Notice that there is no code written about login to the host in the custom ssh server, and the custom ssh server can be protected under selinux, only allowing it to access /dev/pts/X. In fact using a custom VNC server in qemu is as risky as a custom ssh server in vdsm. If we accepts the former one, then I can accepts the latter one. The consideration is how robust of the custom ssh server, and the difficulty to maintain it. In He Jie's current patch, the ssh auth and transport library is an open-source third-party project, unless the project is well maintained and well proven, using it can be risky. So my opinion is using neither the host sshd, nor a custom ssh server. Maybe we can apply the suggestion from Dan Yasny, running a standard sshd in a very small VM in every host, and forward data from this VM to other guest consoles. The ssh part is in the VM, then our work is just forwarding data from the VM via virto serial channels, to the guest via the pty.

* Adam Litke <agl(a)us.ibm.com&gt; [2012-10-12 08:13]: > On Fri, Oct 12, 2012 at 04:55:20PM +0800, Zhou Zheng Sheng wrote: > > > > on 09/04/2012 22:19, Ryan Harper wrote: > > >* Dan Kenigsberg <danken(a)redhat.com&gt; [2012-09-04 05:53]: > > >>On Tue, Sep 04, 2012 at 03:05:37PM +0800, Xu He Jie wrote: > > >>>On 09/03/2012 10:33 PM, Dan Kenigsberg wrote: > > >>>>On Thu, Aug 30, 2012 at 04:26:31PM -0500, Adam Litke wrote: > > >>>>>On Thu, Aug 30, 2012 at 11:32:02AM +0800, Xu He Jie wrote: > > >>>>>>Hi, > > >>>>>> > > >>>>>> I submited a patch for text-based console > > >>>>>>http://gerrit.ovirt.org/#/c/7165/ > > >>>>>> > > >>>>>>the issue I want to discussing as below: > > >>>>>>1. fix port VS dynamic port > > >>>>>> > > >>>>>>Use fix port for all VM's console. connect console with 'ssh > > >>>>>>vmUUID@ip -p port'. > > >>>>>>Distinguishing VM by vmUUID. > > >>>>>> > > >>>>>> > > >>>>>> The current implement was vdsm will allocated port for console > > >>>>>>dynamically and spawn sub-process when VM creating. > > >>>>>>In sub-process the main thread responsible for accept new connection > > >>>>>>and dispatch output of console to each connection. > > >>>>>>When new connection is coming, main processing create new thread for > > >>>>>>each new connection. Dynamic port will allocated > > >>>>>>port for each VM and use range port. It isn't good for firewall rules. > > >>>>>> > > >>>>>> > > >>>>>> so I got a suggestion that use fix port. and connect console with > > >>>>>>'ssh vmuuid@hostip -p fixport'. this is simple for user. > > >>>>>>We need one process for accept new connection from fix port and when > > >>>>>>new connection is coming, spawn sub-process for each vm. > > >>>>>>But because the console only can open by one process, main process > > >>>>>>need responsible for dispatching console's output of all vms and all > > >>>>>>connection. > > >>>>>>So the code will be a little complex then dynamic port. > > >>>>>> > > >>>>>> So this is dynamic port VS fix port and simple code VS complex code. > > >>>>>>From a usability point of view, I think the fixed port suggestion is nicer. > > >>>>>This means that a system administrator needs only to open one port to enable > > >>>>>remote console access. If your initial implementation limits console access to > > >>>>>one connection per VM would that simplify the code? > > >>>>Yes, using a fixed port for all consoles of all VMs seems like a cooler > > >>>>idea. Besides the firewall issue, there's user experience: instead of > > >>>>calling getVmStats to tell the vm port, and then use ssh, only one ssh > > >>>>call is needed. (Taking this one step further - it would make sense to > > >>>>add another layer on top, directing console clients to the specific host > > >>>>currently running the Vm.) > > >>>> > > >>>>I did not take a close look at your implementation, and did not research > > >>>>this myself, but have you considered using sshd for this? I suppose you > > >>>>can configure sshd to collect the list of known "users" from > > >>>>`getAllVmStats`, and force it to run a command that redirects VM's > > >>>>console to the ssh client. It has a potential of being a more robust > > >>>>implementation. > > >>>I have considered using sshd and ssh tunnel. They > > >>>can't implement fixed port and share console. > > >>Would you elaborate on that? Usually sshd listens to a fixed port 22, > > >>and allows multiple users to have independet shells. What do you mean by > > >>"share console"? > > >> > > >>>Current implement > > >>>we can do anything that what we want. > > >>Yes, it is completely under our control, but there are down sides, too: > > >>we have to maintain another process, and another entry point, instead of > > >>configuring a universally-used, well maintained and debugged > > >>application. > > >Think of the security implications of having another remote shell > > >access point to a host. I'd much rather trust sshd if we can make it > > >work. > > > > > > > > >>Dan. > > > > At first glance, the standard sshd on the host is stronger and more > > robust than a custom ssh server, but the risk using the host sshd is > > high. If we implement this feature via host ssd, when a hacker > > attacks the sshd successfully, he will get access to the host shell. > > After all, the custom ssh server is not for accessing host shell, > > but just for forwarding the data from the guest console (a host > > /dev/pts/X device). If we just use a custom ssh server, the code in > > this server only does 1. auth, 2. data forwarding, when the hacker > > attacks, he just gets access to that virtual machine. Notice that > > there is no code written about login to the host in the custom ssh > > server, and the custom ssh server can be protected under selinux, > > only allowing it to access /dev/pts/X. > > > > In fact using a custom VNC server in qemu is as risky as a custom > > ssh server in vdsm. If we accepts the former one, then I can accepts > > the latter one. The consideration is how robust of the custom ssh > > server, and the difficulty to maintain it. In He Jie's current > > patch, the ssh auth and transport library is an open-source > > third-party project, unless the project is well maintained and well > > proven, using it can be risky. > > > > So my opinion is using neither the host sshd, nor a custom ssh > > server. Maybe we can apply the suggestion from Dan Yasny, running a > > standard sshd in a very small VM in every host, and forward data > > from this VM to other guest consoles. The ssh part is in the VM, > > then our work is just forwarding data from the VM via virto serial > > channels, to the guest via the pty. > > I really dislike the idea of a service VM for something as fundamental as a VM > console. The logistics of maintaining such a VM are a nightmare: provisioning, > deployment, software upgrades, HA, etc. > > Maybe we can start simple and provide console access locally only. What sort of > functionality would the vdsm api need to provide to enable only local access to > the console? Presumably, it would set up a connection and provide the user with > a port/pty to use to connect locally. For now it would be "BYOSSH - bring your > own SSH" as clients would need to access the hosts with something like: > > ssh -t <host> "<connect command>" > > The above command could be wrapped in a vdsm-tool command. I think we have a patch that does local-only here but using virsh http://gerrit.ovirt.org/#/c/8041/ I'd prefer to let the user tunnel that output over ssh if they need to.

> In the future, we can take a look at extending this feature via some sort of > remote streaming API. Keep in mind that in order for this feature to be truly > useful to ovirt-engine consumers, the console connection must survive a VM > migration. To me, this means that vdsm will need to implement a generic > streaming API like libvirt has. Indeed. > > -- > Adam Litke <agl(a)us.ibm.com&gt; > IBM Linux Technology Center > > _______________________________________________ > vdsm-devel mailing list > vdsm-devel(a)lists.fedorahosted.org > https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel -- Ryan Harper Software Engineer; Linux Technology Center IBM Corp., Austin, Tx ryanh(a)us.ibm.com _______________________________________________ vdsm-devel mailing list vdsm-devel(a)lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel

On 10/15/2012 07:41 PM, Dave Allan wrote: >On Fri, Oct 12, 2012 at 11:25:47AM -0500, Ryan Harper wrote: >>* Adam Litke <agl(a)us.ibm.com&gt; [2012-10-12 08:13]: >>>On Fri, Oct 12, 2012 at 04:55:20PM +0800, Zhou Zheng Sheng wrote: >>>> >>>>on 09/04/2012 22:19, Ryan Harper wrote: >>>>>* Dan Kenigsberg <danken(a)redhat.com&gt; [2012-09-04 05:53]: >>>>>>On Tue, Sep 04, 2012 at 03:05:37PM +0800, Xu He Jie wrote: >>>>>>>On 09/03/2012 10:33 PM, Dan Kenigsberg wrote: >>>>>>>>On Thu, Aug 30, 2012 at 04:26:31PM -0500, Adam Litke wrote: >>>>>>>>>On Thu, Aug 30, 2012 at 11:32:02AM +0800, Xu He Jie wrote: >>>>>>>>>>Hi, >>>>>>>>>> >>>>>>>>>> I submited a patch for text-based console >>>>>>>>>>http://gerrit.ovirt.org/#/c/7165/ >>>>>>>>>> >>>>>>>>>>the issue I want to discussing as below: >>>>>>>>>>1. fix port VS dynamic port >>>>>>>>>> >>>>>>>>>>Use fix port for all VM's console. connect console with 'ssh >>>>>>>>>>vmUUID@ip -p port'. >>>>>>>>>>Distinguishing VM by vmUUID. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> The current implement was vdsm will allocated port for console >>>>>>>>>>dynamically and spawn sub-process when VM creating. >>>>>>>>>>In sub-process the main thread responsible for accept new connection >>>>>>>>>>and dispatch output of console to each connection. >>>>>>>>>>When new connection is coming, main processing create new thread for >>>>>>>>>>each new connection. Dynamic port will allocated >>>>>>>>>>port for each VM and use range port. It isn't good for firewall rules. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> so I got a suggestion that use fix port. and connect console with >>>>>>>>>>'ssh vmuuid@hostip -p fixport'. this is simple for user. >>>>>>>>>>We need one process for accept new connection from fix port and when >>>>>>>>>>new connection is coming, spawn sub-process for each vm. >>>>>>>>>>But because the console only can open by one process, main process >>>>>>>>>>need responsible for dispatching console's output of all vms and all >>>>>>>>>>connection. >>>>>>>>>>So the code will be a little complex then dynamic port. >>>>>>>>>> >>>>>>>>>> So this is dynamic port VS fix port and simple code VS complex code. >>>>>>>>>>From a usability point of view, I think the fixed port suggestion is nicer. >>>>>>>>>This means that a system administrator needs only to open one port to enable >>>>>>>>>remote console access. If your initial implementation limits console access to >>>>>>>>>one connection per VM would that simplify the code? >>>>>>>>Yes, using a fixed port for all consoles of all VMs seems like a cooler >>>>>>>>idea. Besides the firewall issue, there's user experience: instead of >>>>>>>>calling getVmStats to tell the vm port, and then use ssh, only one ssh >>>>>>>>call is needed. (Taking this one step further - it would make sense to >>>>>>>>add another layer on top, directing console clients to the specific host >>>>>>>>currently running the Vm.) >>>>>>>> >>>>>>>>I did not take a close look at your implementation, and did not research >>>>>>>>this myself, but have you considered using sshd for this? I suppose you >>>>>>>>can configure sshd to collect the list of known "users" from >>>>>>>>`getAllVmStats`, and force it to run a command that redirects VM's >>>>>>>>console to the ssh client. It has a potential of being a more robust >>>>>>>>implementation. >>>>>>>I have considered using sshd and ssh tunnel. They >>>>>>>can't implement fixed port and share console. >>>>>>Would you elaborate on that? Usually sshd listens to a fixed port 22, >>>>>>and allows multiple users to have independet shells. What do you mean by >>>>>>"share console"? >>>>>> >>>>>>>Current implement >>>>>>>we can do anything that what we want. >>>>>>Yes, it is completely under our control, but there are down sides, too: >>>>>>we have to maintain another process, and another entry point, instead of >>>>>>configuring a universally-used, well maintained and debugged >>>>>>application. >>>>>Think of the security implications of having another remote shell >>>>>access point to a host. I'd much rather trust sshd if we can make it >>>>>work. >>>>> >>>>> >>>>>>Dan. >>>> >>>>At first glance, the standard sshd on the host is stronger and more >>>>robust than a custom ssh server, but the risk using the host sshd is >>>>high. If we implement this feature via host ssd, when a hacker >>>>attacks the sshd successfully, he will get access to the host shell. >>>>After all, the custom ssh server is not for accessing host shell, >>>>but just for forwarding the data from the guest console (a host >>>>/dev/pts/X device). If we just use a custom ssh server, the code in >>>>this server only does 1. auth, 2. data forwarding, when the hacker >>>>attacks, he just gets access to that virtual machine. Notice that >>>>there is no code written about login to the host in the custom ssh >>>>server, and the custom ssh server can be protected under selinux, >>>>only allowing it to access /dev/pts/X. >>>> >>>>In fact using a custom VNC server in qemu is as risky as a custom >>>>ssh server in vdsm. If we accepts the former one, then I can accepts >>>>the latter one. The consideration is how robust of the custom ssh >>>>server, and the difficulty to maintain it. In He Jie's current >>>>patch, the ssh auth and transport library is an open-source >>>>third-party project, unless the project is well maintained and well >>>>proven, using it can be risky. >>>> >>>>So my opinion is using neither the host sshd, nor a custom ssh >>>>server. Maybe we can apply the suggestion from Dan Yasny, running a >>>>standard sshd in a very small VM in every host, and forward data >>>>from this VM to other guest consoles. The ssh part is in the VM, >>>>then our work is just forwarding data from the VM via virto serial >>>>channels, to the guest via the pty. >>> >>>I really dislike the idea of a service VM for something as fundamental as a VM >>>console. The logistics of maintaining such a VM are a nightmare: provisioning, >>>deployment, software upgrades, HA, etc. >>> >>>Maybe we can start simple and provide console access locally only. What sort of >>>functionality would the vdsm api need to provide to enable only local access to >>>the console? Presumably, it would set up a connection and provide the user with >>>a port/pty to use to connect locally. For now it would be "BYOSSH - bring your >>>own SSH" as clients would need to access the hosts with something like: >>> >>>ssh -t <host> "<connect command>" >>> >>>The above command could be wrapped in a vdsm-tool command. >> >>I think we have a patch that does local-only here but using virsh >> >>http://gerrit.ovirt.org/#/c/8041/ >> >>I'd prefer to let the user tunnel that output over ssh if they need to. > >I haven't been following this thread all that closely, so perhaps this >idea has already been mentioned, but would the libvirt ssh transport >help with this situation, ie, > >virsh -c qemu+ssh://root@host/system console some_vm i hope this would work/be simple. some questions: 1. can clients script with/over this method like they can with ssh?

2. can we make it support tickets like we have for spice/vnc?

thanks, Itamar

On 10/18/2012 09:13 PM, Dave Allan wrote: >On Thu, Oct 18, 2012 at 02:21:44AM +0200, Itamar Heim wrote: >>On 10/15/2012 07:41 PM, Dave Allan wrote: >>>On Fri, Oct 12, 2012 at 11:25:47AM -0500, Ryan Harper wrote: >>>>* Adam Litke <agl(a)us.ibm.com&gt; [2012-10-12 08:13]: >>>>>On Fri, Oct 12, 2012 at 04:55:20PM +0800, Zhou Zheng Sheng wrote: >>>>>> >>>>>>on 09/04/2012 22:19, Ryan Harper wrote: >>>>>>>* Dan Kenigsberg <danken(a)redhat.com&gt; [2012-09-04 05:53]: >>>>>>>>On Tue, Sep 04, 2012 at 03:05:37PM +0800, Xu He Jie wrote: >>>>>>>>>On 09/03/2012 10:33 PM, Dan Kenigsberg wrote: >>>>>>>>>>On Thu, Aug 30, 2012 at 04:26:31PM -0500, Adam Litke wrote: >>>>>>>>>>>On Thu, Aug 30, 2012 at 11:32:02AM +0800, Xu He Jie wrote: >>>>>>>>>>>>Hi, >>>>>>>>>>>> >>>>>>>>>>>> I submited a patch for text-based console >>>>>>>>>>>>http://gerrit.ovirt.org/#/c/7165/ >>>>>>>>>>>> >>>>>>>>>>>>the issue I want to discussing as below: >>>>>>>>>>>>1. fix port VS dynamic port >>>>>>>>>>>> >>>>>>>>>>>>Use fix port for all VM's console. connect console with 'ssh >>>>>>>>>>>>vmUUID@ip -p port'. >>>>>>>>>>>>Distinguishing VM by vmUUID. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> The current implement was vdsm will allocated port for console >>>>>>>>>>>>dynamically and spawn sub-process when VM creating. >>>>>>>>>>>>In sub-process the main thread responsible for accept new connection >>>>>>>>>>>>and dispatch output of console to each connection. >>>>>>>>>>>>When new connection is coming, main processing create new thread for >>>>>>>>>>>>each new connection. Dynamic port will allocated >>>>>>>>>>>>port for each VM and use range port. It isn't good for firewall rules. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> so I got a suggestion that use fix port. and connect console with >>>>>>>>>>>>'ssh vmuuid@hostip -p fixport'. this is simple for user. >>>>>>>>>>>>We need one process for accept new connection from fix port and when >>>>>>>>>>>>new connection is coming, spawn sub-process for each vm. >>>>>>>>>>>>But because the console only can open by one process, main process >>>>>>>>>>>>need responsible for dispatching console's output of all vms and all >>>>>>>>>>>>connection. >>>>>>>>>>>>So the code will be a little complex then dynamic port. >>>>>>>>>>>> >>>>>>>>>>>> So this is dynamic port VS fix port and simple code VS complex code. >>>>>>>>>>>>From a usability point of view, I think the fixed port suggestion is nicer. >>>>>>>>>>>This means that a system administrator needs only to open one port to enable >>>>>>>>>>>remote console access. If your initial implementation limits console access to >>>>>>>>>>>one connection per VM would that simplify the code? >>>>>>>>>>Yes, using a fixed port for all consoles of all VMs seems like a cooler >>>>>>>>>>idea. Besides the firewall issue, there's user experience: instead of >>>>>>>>>>calling getVmStats to tell the vm port, and then use ssh, only one ssh >>>>>>>>>>call is needed. (Taking this one step further - it would make sense to >>>>>>>>>>add another layer on top, directing console clients to the specific host >>>>>>>>>>currently running the Vm.) >>>>>>>>>> >>>>>>>>>>I did not take a close look at your implementation, and did not research >>>>>>>>>>this myself, but have you considered using sshd for this? I suppose you >>>>>>>>>>can configure sshd to collect the list of known "users" from >>>>>>>>>>`getAllVmStats`, and force it to run a command that redirects VM's >>>>>>>>>>console to the ssh client. It has a potential of being a more robust >>>>>>>>>>implementation. >>>>>>>>>I have considered using sshd and ssh tunnel. They >>>>>>>>>can't implement fixed port and share console. >>>>>>>>Would you elaborate on that? Usually sshd listens to a fixed port 22, >>>>>>>>and allows multiple users to have independet shells. What do you mean by >>>>>>>>"share console"? >>>>>>>> >>>>>>>>>Current implement >>>>>>>>>we can do anything that what we want. >>>>>>>>Yes, it is completely under our control, but there are down sides, too: >>>>>>>>we have to maintain another process, and another entry point, instead of >>>>>>>>configuring a universally-used, well maintained and debugged >>>>>>>>application. >>>>>>>Think of the security implications of having another remote shell >>>>>>>access point to a host. I'd much rather trust sshd if we can make it >>>>>>>work. >>>>>>> >>>>>>> >>>>>>>>Dan. >>>>>> >>>>>>At first glance, the standard sshd on the host is stronger and more >>>>>>robust than a custom ssh server, but the risk using the host sshd is >>>>>>high. If we implement this feature via host ssd, when a hacker >>>>>>attacks the sshd successfully, he will get access to the host shell. >>>>>>After all, the custom ssh server is not for accessing host shell, >>>>>>but just for forwarding the data from the guest console (a host >>>>>>/dev/pts/X device). If we just use a custom ssh server, the code in >>>>>>this server only does 1. auth, 2. data forwarding, when the hacker >>>>>>attacks, he just gets access to that virtual machine. Notice that >>>>>>there is no code written about login to the host in the custom ssh >>>>>>server, and the custom ssh server can be protected under selinux, >>>>>>only allowing it to access /dev/pts/X. >>>>>> >>>>>>In fact using a custom VNC server in qemu is as risky as a custom >>>>>>ssh server in vdsm. If we accepts the former one, then I can accepts >>>>>>the latter one. The consideration is how robust of the custom ssh >>>>>>server, and the difficulty to maintain it. In He Jie's current >>>>>>patch, the ssh auth and transport library is an open-source >>>>>>third-party project, unless the project is well maintained and well >>>>>>proven, using it can be risky. >>>>>> >>>>>>So my opinion is using neither the host sshd, nor a custom ssh >>>>>>server. Maybe we can apply the suggestion from Dan Yasny, running a >>>>>>standard sshd in a very small VM in every host, and forward data >>>>>>from this VM to other guest consoles. The ssh part is in the VM, >>>>>>then our work is just forwarding data from the VM via virto serial >>>>>>channels, to the guest via the pty. >>>>> >>>>>I really dislike the idea of a service VM for something as fundamental as a VM >>>>>console. The logistics of maintaining such a VM are a nightmare: provisioning, >>>>>deployment, software upgrades, HA, etc. >>>>> >>>>>Maybe we can start simple and provide console access locally only. What sort of >>>>>functionality would the vdsm api need to provide to enable only local access to >>>>>the console? Presumably, it would set up a connection and provide the user with >>>>>a port/pty to use to connect locally. For now it would be "BYOSSH - bring your >>>>>own SSH" as clients would need to access the hosts with something like: >>>>> >>>>>ssh -t <host> "<connect command>" >>>>> >>>>>The above command could be wrapped in a vdsm-tool command. >>>> >>>>I think we have a patch that does local-only here but using virsh >>>> >>>>http://gerrit.ovirt.org/#/c/8041/ >>>> >>>>I'd prefer to let the user tunnel that output over ssh if they need to. >>> >>>I haven't been following this thread all that closely, so perhaps this >>>idea has already been mentioned, but would the libvirt ssh transport >>>help with this situation, ie, >>> >>>virsh -c qemu+ssh://root@host/system console some_vm >> >>i hope this would work/be simple. some questions: >>1. can clients script with/over this method like they can with ssh? > >The serial console API does not do things like > >$ ssh root@bar 'hostname' >bar > >since it's just a stream containing the console i/o, but I'm not sure >that answers your question. Can you give me an example of what kind >of script you're thinking of? > >>2. can we make it support tickets like we have for spice/vnc? > >The console API will use any authentication that the libvirtd on the >target will accept, so it could authenticate the same credentials as >vdsm. console does require a read-write connection to libvirt. > >There's example python code in the libvirt git tree: > >http://libvirt.org/git/?p=libvirt.git;a=blob;f=examples/python/consolecallback.py I don't think giving end-users direct read/write access to libvirt is the way to go (even with libvirt having a permission model). we don't manage users/credentials at that level, only tickets.

the thing i like about the spice approach is it goes to qemu process, via channels we already have with clients (spice).

From: "Itamar Heim" <iheim(a)redhat.com&gt; To: "Ryan Harper" <ryanh(a)us.ibm.com&gt; Cc: "VDSM Project Development" <vdsm-devel(a)lists.fedorahosted.org&gt; Sent: Sunday, 21 October, 2012 11:17:24 AM Subject: Re: [vdsm] [RFC]about the implement of text-based console On 10/19/2012 04:31 PM, Ryan Harper wrote: > * Itamar Heim <iheim(a)redhat.com&gt; [2012-10-18 14:17]: >> On 10/18/2012 09:13 PM, Dave Allan wrote: >>> On Thu, Oct 18, 2012 at 02:21:44AM +0200, Itamar Heim wrote: >>>> On 10/15/2012 07:41 PM, Dave Allan wrote: >>>>> On Fri, Oct 12, 2012 at 11:25:47AM -0500, Ryan Harper wrote: >>>>>> * Adam Litke <agl(a)us.ibm.com&gt; [2012-10-12 08:13]: >>>>>>> On Fri, Oct 12, 2012 at 04:55:20PM +0800, Zhou Zheng Sheng >>>>>>> wrote: >>>>>>>> >>>>>>>> on 09/04/2012 22:19, Ryan Harper wrote: >>>>>>>>> * Dan Kenigsberg <danken(a)redhat.com&gt; [2012-09-04 05:53]: >>>>>>>>>> On Tue, Sep 04, 2012 at 03:05:37PM +0800, Xu He Jie wrote: >>>>>>>>>>> On 09/03/2012 10:33 PM, Dan Kenigsberg wrote: >>>>>>>>>>>> On Thu, Aug 30, 2012 at 04:26:31PM -0500, Adam Litke >>>>>>>>>>>> wrote: >>>>>>>>>>>>> On Thu, Aug 30, 2012 at 11:32:02AM +0800, Xu He Jie >>>>>>>>>>>>> wrote: >>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>> >>>>>>>>>>>>>> I submited a patch for text-based console >>>>>>>>>>>>>> http://gerrit.ovirt.org/#/c/7165/ >>>>>>>>>>>>>> >>>>>>>>>>>>>> the issue I want to discussing as below: >>>>>>>>>>>>>> 1. fix port VS dynamic port >>>>>>>>>>>>>> >>>>>>>>>>>>>> Use fix port for all VM's console. connect console >>>>>>>>>>>>>> with 'ssh >>>>>>>>>>>>>> vmUUID@ip -p port'. >>>>>>>>>>>>>> Distinguishing VM by vmUUID. >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> The current implement was vdsm will allocated port >>>>>>>>>>>>>> for console >>>>>>>>>>>>>> dynamically and spawn sub-process when VM creating. >>>>>>>>>>>>>> In sub-process the main thread responsible for accept >>>>>>>>>>>>>> new connection >>>>>>>>>>>>>> and dispatch output of console to each connection. >>>>>>>>>>>>>> When new connection is coming, main processing create >>>>>>>>>>>>>> new thread for >>>>>>>>>>>>>> each new connection. Dynamic port will allocated >>>>>>>>>>>>>> port for each VM and use range port. It isn't good for >>>>>>>>>>>>>> firewall rules. >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> so I got a suggestion that use fix port. and >>>>>>>>>>>>>> connect console with >>>>>>>>>>>>>> 'ssh vmuuid@hostip -p fixport'. this is simple for >>>>>>>>>>>>>> user. >>>>>>>>>>>>>> We need one process for accept new connection from fix >>>>>>>>>>>>>> port and when >>>>>>>>>>>>>> new connection is coming, spawn sub-process for each >>>>>>>>>>>>>> vm. >>>>>>>>>>>>>> But because the console only can open by one process, >>>>>>>>>>>>>> main process >>>>>>>>>>>>>> need responsible for dispatching console's output of >>>>>>>>>>>>>> all vms and all >>>>>>>>>>>>>> connection. >>>>>>>>>>>>>> So the code will be a little complex then dynamic >>>>>>>>>>>>>> port. >>>>>>>>>>>>>> >>>>>>>>>>>>>> So this is dynamic port VS fix port and simple code >>>>>>>>>>>>>> VS complex code. >>>>>>>>>>>>> >From a usability point of view, I think the fixed port >>>>>>>>>>>>> >suggestion is nicer. >>>>>>>>>>>>> This means that a system administrator needs only to >>>>>>>>>>>>> open one port to enable >>>>>>>>>>>>> remote console access. If your initial implementation >>>>>>>>>>>>> limits console access to >>>>>>>>>>>>> one connection per VM would that simplify the code? >>>>>>>>>>>> Yes, using a fixed port for all consoles of all VMs >>>>>>>>>>>> seems like a cooler >>>>>>>>>>>> idea. Besides the firewall issue, there's user >>>>>>>>>>>> experience: instead of >>>>>>>>>>>> calling getVmStats to tell the vm port, and then use >>>>>>>>>>>> ssh, only one ssh >>>>>>>>>>>> call is needed. (Taking this one step further - it would >>>>>>>>>>>> make sense to >>>>>>>>>>>> add another layer on top, directing console clients to >>>>>>>>>>>> the specific host >>>>>>>>>>>> currently running the Vm.) >>>>>>>>>>>> >>>>>>>>>>>> I did not take a close look at your implementation, and >>>>>>>>>>>> did not research >>>>>>>>>>>> this myself, but have you considered using sshd for >>>>>>>>>>>> this? I suppose you >>>>>>>>>>>> can configure sshd to collect the list of known "users" >>>>>>>>>>>> from >>>>>>>>>>>> `getAllVmStats`, and force it to run a command that >>>>>>>>>>>> redirects VM's >>>>>>>>>>>> console to the ssh client. It has a potential of being a >>>>>>>>>>>> more robust >>>>>>>>>>>> implementation. >>>>>>>>>>> I have considered using sshd and ssh tunnel. They >>>>>>>>>>> can't implement fixed port and share console. >>>>>>>>>> Would you elaborate on that? Usually sshd listens to a >>>>>>>>>> fixed port 22, >>>>>>>>>> and allows multiple users to have independet shells. What >>>>>>>>>> do you mean by >>>>>>>>>> "share console"? >>>>>>>>>> >>>>>>>>>>> Current implement >>>>>>>>>>> we can do anything that what we want. >>>>>>>>>> Yes, it is completely under our control, but there are >>>>>>>>>> down sides, too: >>>>>>>>>> we have to maintain another process, and another entry >>>>>>>>>> point, instead of >>>>>>>>>> configuring a universally-used, well maintained and >>>>>>>>>> debugged >>>>>>>>>> application. >>>>>>>>> Think of the security implications of having another remote >>>>>>>>> shell >>>>>>>>> access point to a host. I'd much rather trust sshd if we >>>>>>>>> can make it >>>>>>>>> work. >>>>>>>>> >>>>>>>>> >>>>>>>>>> Dan. >>>>>>>> >>>>>>>> At first glance, the standard sshd on the host is stronger >>>>>>>> and more >>>>>>>> robust than a custom ssh server, but the risk using the host >>>>>>>> sshd is >>>>>>>> high. If we implement this feature via host ssd, when a >>>>>>>> hacker >>>>>>>> attacks the sshd successfully, he will get access to the >>>>>>>> host shell. >>>>>>>> After all, the custom ssh server is not for accessing host >>>>>>>> shell, >>>>>>>> but just for forwarding the data from the guest console (a >>>>>>>> host >>>>>>>> /dev/pts/X device). If we just use a custom ssh server, the >>>>>>>> code in >>>>>>>> this server only does 1. auth, 2. data forwarding, when the >>>>>>>> hacker >>>>>>>> attacks, he just gets access to that virtual machine. Notice >>>>>>>> that >>>>>>>> there is no code written about login to the host in the >>>>>>>> custom ssh >>>>>>>> server, and the custom ssh server can be protected under >>>>>>>> selinux, >>>>>>>> only allowing it to access /dev/pts/X. >>>>>>>> >>>>>>>> In fact using a custom VNC server in qemu is as risky as a >>>>>>>> custom >>>>>>>> ssh server in vdsm. If we accepts the former one, then I can >>>>>>>> accepts >>>>>>>> the latter one. The consideration is how robust of the >>>>>>>> custom ssh >>>>>>>> server, and the difficulty to maintain it. In He Jie's >>>>>>>> current >>>>>>>> patch, the ssh auth and transport library is an open-source >>>>>>>> third-party project, unless the project is well maintained >>>>>>>> and well >>>>>>>> proven, using it can be risky. >>>>>>>> >>>>>>>> So my opinion is using neither the host sshd, nor a custom >>>>>>>> ssh >>>>>>>> server. Maybe we can apply the suggestion from Dan Yasny, >>>>>>>> running a >>>>>>>> standard sshd in a very small VM in every host, and forward >>>>>>>> data >>>>>>> >from this VM to other guest consoles. The ssh part is in the >>>>>>> >VM, >>>>>>>> then our work is just forwarding data from the VM via virto >>>>>>>> serial >>>>>>>> channels, to the guest via the pty. >>>>>>> >>>>>>> I really dislike the idea of a service VM for something as >>>>>>> fundamental as a VM >>>>>>> console. The logistics of maintaining such a VM are a >>>>>>> nightmare: provisioning, >>>>>>> deployment, software upgrades, HA, etc. >>>>>>> >>>>>>> Maybe we can start simple and provide console access locally >>>>>>> only. What sort of >>>>>>> functionality would the vdsm api need to provide to enable >>>>>>> only local access to >>>>>>> the console? Presumably, it would set up a connection and >>>>>>> provide the user with >>>>>>> a port/pty to use to connect locally. For now it would be >>>>>>> "BYOSSH - bring your >>>>>>> own SSH" as clients would need to access the hosts with >>>>>>> something like: >>>>>>> >>>>>>> ssh -t <host> "<connect command>" >>>>>>> >>>>>>> The above command could be wrapped in a vdsm-tool command. >>>>>> >>>>>> I think we have a patch that does local-only here but using >>>>>> virsh >>>>>> >>>>>> http://gerrit.ovirt.org/#/c/8041/ >>>>>> >>>>>> I'd prefer to let the user tunnel that output over ssh if they >>>>>> need to. >>>>> >>>>> I haven't been following this thread all that closely, so >>>>> perhaps this >>>>> idea has already been mentioned, but would the libvirt ssh >>>>> transport >>>>> help with this situation, ie, >>>>> >>>>> virsh -c qemu+ssh://root@host/system console some_vm >>>> >>>> i hope this would work/be simple. some questions: >>>> 1. can clients script with/over this method like they can with >>>> ssh? >>> >>> The serial console API does not do things like >>> >>> $ ssh root@bar 'hostname' >>> bar >>> >>> since it's just a stream containing the console i/o, but I'm not >>> sure >>> that answers your question. Can you give me an example of what >>> kind >>> of script you're thinking of? >>> >>>> 2. can we make it support tickets like we have for spice/vnc? >>> >>> The console API will use any authentication that the libvirtd on >>> the >>> target will accept, so it could authenticate the same credentials >>> as >>> vdsm. console does require a read-write connection to libvirt. >>> >>> There's example python code in the libvirt git tree: >>> >>> http://libvirt.org/git/?p=libvirt.git;a=blob;f=examples/python/consolecal... >> >> I don't think giving end-users direct read/write access to libvirt >> is the way to go (even with libvirt having a permission model). >> we don't manage users/credentials at that level, only tickets. > > I agree with the additional creditials, I'd prefer not to have to > manage > this. > >> the thing i like about the spice approach is it goes to qemu >> process, via channels we already have with clients (spice). > > We already have a device that works without additional clients or > channels, the pty on the host the VM runs. > > The only question left is how to provide remote access to this. In > the > short term, we don't need remoting to make use of it. Having vdsm > invoke virsh console for a local vdsClient is sufficent to start > with. > > For remoting, something we've been discussing is a > console_read/console_write API call which would be something that a > management application could consume and re-use some of those AJAX > web-console widgets. I thought the interesting use case for serial console was ability to script to it remotely, rather than a UI (which vnc/spice would give you anyway)?

> > w.r.t permissions for console_read/console_write we could use the > setTicket mechanism to enable access to those verbs. read/write > API > should also work well with VM migration. > > _______________________________________________ vdsm-devel mailing list vdsm-devel(a)lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel

From: "Adam Litke" <agl(a)us.ibm.com&gt; To: "Zhou Zheng Sheng" <zhshzhou(a)linux.vnet.ibm.com&gt; Cc: "VDSM Project Development" <vdsm-devel(a)lists.fedorahosted.org&gt; Sent: Friday, 12 October, 2012 3:10:57 PM Subject: Re: [vdsm] [RFC]about the implement of text-based console On Fri, Oct 12, 2012 at 04:55:20PM +0800, Zhou Zheng Sheng wrote: > > on 09/04/2012 22:19, Ryan Harper wrote: > >* Dan Kenigsberg <danken(a)redhat.com&gt; [2012-09-04 05:53]: > >>On Tue, Sep 04, 2012 at 03:05:37PM +0800, Xu He Jie wrote: > >>>On 09/03/2012 10:33 PM, Dan Kenigsberg wrote: > >>>>On Thu, Aug 30, 2012 at 04:26:31PM -0500, Adam Litke wrote: > >>>>>On Thu, Aug 30, 2012 at 11:32:02AM +0800, Xu He Jie wrote: > >>>>>>Hi, > >>>>>> > >>>>>> I submited a patch for text-based console > >>>>>>http://gerrit.ovirt.org/#/c/7165/ > >>>>>> > >>>>>>the issue I want to discussing as below: > >>>>>>1. fix port VS dynamic port > >>>>>> > >>>>>>Use fix port for all VM's console. connect console with 'ssh > >>>>>>vmUUID@ip -p port'. > >>>>>>Distinguishing VM by vmUUID. > >>>>>> > >>>>>> > >>>>>> The current implement was vdsm will allocated port for > >>>>>> console > >>>>>>dynamically and spawn sub-process when VM creating. > >>>>>>In sub-process the main thread responsible for accept new > >>>>>>connection > >>>>>>and dispatch output of console to each connection. > >>>>>>When new connection is coming, main processing create new > >>>>>>thread for > >>>>>>each new connection. Dynamic port will allocated > >>>>>>port for each VM and use range port. It isn't good for > >>>>>>firewall rules. > >>>>>> > >>>>>> > >>>>>> so I got a suggestion that use fix port. and connect > >>>>>> console with > >>>>>>'ssh vmuuid@hostip -p fixport'. this is simple for user. > >>>>>>We need one process for accept new connection from fix port > >>>>>>and when > >>>>>>new connection is coming, spawn sub-process for each vm. > >>>>>>But because the console only can open by one process, main > >>>>>>process > >>>>>>need responsible for dispatching console's output of all vms > >>>>>>and all > >>>>>>connection. > >>>>>>So the code will be a little complex then dynamic port. > >>>>>> > >>>>>> So this is dynamic port VS fix port and simple code VS > >>>>>> complex code. > >>>>>>From a usability point of view, I think the fixed port > >>>>>>suggestion is nicer. > >>>>>This means that a system administrator needs only to open one > >>>>>port to enable > >>>>>remote console access. If your initial implementation limits > >>>>>console access to > >>>>>one connection per VM would that simplify the code? > >>>>Yes, using a fixed port for all consoles of all VMs seems like > >>>>a cooler > >>>>idea. Besides the firewall issue, there's user experience: > >>>>instead of > >>>>calling getVmStats to tell the vm port, and then use ssh, only > >>>>one ssh > >>>>call is needed. (Taking this one step further - it would make > >>>>sense to > >>>>add another layer on top, directing console clients to the > >>>>specific host > >>>>currently running the Vm.) > >>>> > >>>>I did not take a close look at your implementation, and did not > >>>>research > >>>>this myself, but have you considered using sshd for this? I > >>>>suppose you > >>>>can configure sshd to collect the list of known "users" from > >>>>`getAllVmStats`, and force it to run a command that redirects > >>>>VM's > >>>>console to the ssh client. It has a potential of being a more > >>>>robust > >>>>implementation. > >>>I have considered using sshd and ssh tunnel. They > >>>can't implement fixed port and share console. > >>Would you elaborate on that? Usually sshd listens to a fixed port > >>22, > >>and allows multiple users to have independet shells. What do you > >>mean by > >>"share console"? > >> > >>>Current implement > >>>we can do anything that what we want. > >>Yes, it is completely under our control, but there are down > >>sides, too: > >>we have to maintain another process, and another entry point, > >>instead of > >>configuring a universally-used, well maintained and debugged > >>application. > >Think of the security implications of having another remote shell > >access point to a host. I'd much rather trust sshd if we can make > >it > >work. > > > > > >>Dan. > > At first glance, the standard sshd on the host is stronger and more > robust than a custom ssh server, but the risk using the host sshd > is > high. If we implement this feature via host ssd, when a hacker > attacks the sshd successfully, he will get access to the host > shell. > After all, the custom ssh server is not for accessing host shell, > but just for forwarding the data from the guest console (a host > /dev/pts/X device). If we just use a custom ssh server, the code in > this server only does 1. auth, 2. data forwarding, when the hacker > attacks, he just gets access to that virtual machine. Notice that > there is no code written about login to the host in the custom ssh > server, and the custom ssh server can be protected under selinux, > only allowing it to access /dev/pts/X. > > In fact using a custom VNC server in qemu is as risky as a custom > ssh server in vdsm. If we accepts the former one, then I can > accepts > the latter one. The consideration is how robust of the custom ssh > server, and the difficulty to maintain it. In He Jie's current > patch, the ssh auth and transport library is an open-source > third-party project, unless the project is well maintained and well > proven, using it can be risky. > > So my opinion is using neither the host sshd, nor a custom ssh > server. Maybe we can apply the suggestion from Dan Yasny, running a > standard sshd in a very small VM in every host, and forward data > from this VM to other guest consoles. The ssh part is in the VM, > then our work is just forwarding data from the VM via virto serial > channels, to the guest via the pty. I really dislike the idea of a service VM for something as fundamental as a VM console. The logistics of maintaining such a VM are a nightmare: provisioning, deployment, software upgrades, HA, etc.

Maybe we can start simple and provide console access locally only. What sort of functionality would the vdsm api need to provide to enable only local access to the console? Presumably, it would set up a connection and provide the user with a port/pty to use to connect locally. For now it would be "BYOSSH - bring your own SSH" as clients would need to access the hosts with something like: ssh -t <host> "<connect command>" The above command could be wrapped in a vdsm-tool command. In the future, we can take a look at extending this feature via some sort of remote streaming API. Keep in mind that in order for this feature to be truly useful to ovirt-engine consumers, the console connection must survive a VM migration. To me, this means that vdsm will need to implement a generic streaming API like libvirt has. -- Adam Litke <agl(a)us.ibm.com&gt; IBM Linux Technology Center _______________________________________________ vdsm-devel mailing list vdsm-devel(a)lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel

On Mon, Oct 15, 2012 at 04:40:00AM -0400, Dan Yasny wrote: > > > ----- Original Message ----- > > From: "Adam Litke" <agl(a)us.ibm.com&gt; > > To: "Zhou Zheng Sheng" <zhshzhou(a)linux.vnet.ibm.com&gt; > > Cc: "VDSM Project Development" <vdsm-devel(a)lists.fedorahosted.org&gt; > > Sent: Friday, 12 October, 2012 3:10:57 PM > > Subject: Re: [vdsm] [RFC]about the implement of text-based console > > > > On Fri, Oct 12, 2012 at 04:55:20PM +0800, Zhou Zheng Sheng wrote: > > > > > > on 09/04/2012 22:19, Ryan Harper wrote: > > > >* Dan Kenigsberg <danken(a)redhat.com&gt; [2012-09-04 05:53]: > > > >>On Tue, Sep 04, 2012 at 03:05:37PM +0800, Xu He Jie wrote: > > > >>>On 09/03/2012 10:33 PM, Dan Kenigsberg wrote: > > > >>>>On Thu, Aug 30, 2012 at 04:26:31PM -0500, Adam Litke wrote: > > > >>>>>On Thu, Aug 30, 2012 at 11:32:02AM +0800, Xu He Jie wrote: > > > >>>>>>Hi, > > > >>>>>> > > > >>>>>> I submited a patch for text-based console > > > >>>>>>http://gerrit.ovirt.org/#/c/7165/ > > > >>>>>> > > > >>>>>>the issue I want to discussing as below: > > > >>>>>>1. fix port VS dynamic port > > > >>>>>> > > > >>>>>>Use fix port for all VM's console. connect console with 'ssh > > > >>>>>>vmUUID@ip -p port'. > > > >>>>>>Distinguishing VM by vmUUID. > > > >>>>>> > > > >>>>>> > > > >>>>>> The current implement was vdsm will allocated port for > > > >>>>>> console > > > >>>>>>dynamically and spawn sub-process when VM creating. > > > >>>>>>In sub-process the main thread responsible for accept new > > > >>>>>>connection > > > >>>>>>and dispatch output of console to each connection. > > > >>>>>>When new connection is coming, main processing create new > > > >>>>>>thread for > > > >>>>>>each new connection. Dynamic port will allocated > > > >>>>>>port for each VM and use range port. It isn't good for > > > >>>>>>firewall rules. > > > >>>>>> > > > >>>>>> > > > >>>>>> so I got a suggestion that use fix port. and connect > > > >>>>>> console with > > > >>>>>>'ssh vmuuid@hostip -p fixport'. this is simple for user. > > > >>>>>>We need one process for accept new connection from fix port > > > >>>>>>and when > > > >>>>>>new connection is coming, spawn sub-process for each vm. > > > >>>>>>But because the console only can open by one process, main > > > >>>>>>process > > > >>>>>>need responsible for dispatching console's output of all vms > > > >>>>>>and all > > > >>>>>>connection. > > > >>>>>>So the code will be a little complex then dynamic port. > > > >>>>>> > > > >>>>>> So this is dynamic port VS fix port and simple code VS > > > >>>>>> complex code. > > > >>>>>>From a usability point of view, I think the fixed port > > > >>>>>>suggestion is nicer. > > > >>>>>This means that a system administrator needs only to open one > > > >>>>>port to enable > > > >>>>>remote console access. If your initial implementation limits > > > >>>>>console access to > > > >>>>>one connection per VM would that simplify the code? > > > >>>>Yes, using a fixed port for all consoles of all VMs seems like > > > >>>>a cooler > > > >>>>idea. Besides the firewall issue, there's user experience: > > > >>>>instead of > > > >>>>calling getVmStats to tell the vm port, and then use ssh, only > > > >>>>one ssh > > > >>>>call is needed. (Taking this one step further - it would make > > > >>>>sense to > > > >>>>add another layer on top, directing console clients to the > > > >>>>specific host > > > >>>>currently running the Vm.) > > > >>>> > > > >>>>I did not take a close look at your implementation, and did not > > > >>>>research > > > >>>>this myself, but have you considered using sshd for this? I > > > >>>>suppose you > > > >>>>can configure sshd to collect the list of known "users" from > > > >>>>`getAllVmStats`, and force it to run a command that redirects > > > >>>>VM's > > > >>>>console to the ssh client. It has a potential of being a more > > > >>>>robust > > > >>>>implementation. > > > >>>I have considered using sshd and ssh tunnel. They > > > >>>can't implement fixed port and share console. > > > >>Would you elaborate on that? Usually sshd listens to a fixed port > > > >>22, > > > >>and allows multiple users to have independet shells. What do you > > > >>mean by > > > >>"share console"? > > > >> > > > >>>Current implement > > > >>>we can do anything that what we want. > > > >>Yes, it is completely under our control, but there are down > > > >>sides, too: > > > >>we have to maintain another process, and another entry point, > > > >>instead of > > > >>configuring a universally-used, well maintained and debugged > > > >>application. > > > >Think of the security implications of having another remote shell > > > >access point to a host. I'd much rather trust sshd if we can make > > > >it > > > >work. > > > > > > > > > > > >>Dan. > > > > > > At first glance, the standard sshd on the host is stronger and more > > > robust than a custom ssh server, but the risk using the host sshd > > > is > > > high. If we implement this feature via host ssd, when a hacker > > > attacks the sshd successfully, he will get access to the host > > > shell. > > > After all, the custom ssh server is not for accessing host shell, > > > but just for forwarding the data from the guest console (a host > > > /dev/pts/X device). If we just use a custom ssh server, the code in > > > this server only does 1. auth, 2. data forwarding, when the hacker > > > attacks, he just gets access to that virtual machine. Notice that > > > there is no code written about login to the host in the custom ssh > > > server, and the custom ssh server can be protected under selinux, > > > only allowing it to access /dev/pts/X. > > > > > > In fact using a custom VNC server in qemu is as risky as a custom > > > ssh server in vdsm. If we accepts the former one, then I can > > > accepts > > > the latter one. The consideration is how robust of the custom ssh > > > server, and the difficulty to maintain it. In He Jie's current > > > patch, the ssh auth and transport library is an open-source > > > third-party project, unless the project is well maintained and well > > > proven, using it can be risky. > > > > > > So my opinion is using neither the host sshd, nor a custom ssh > > > server. Maybe we can apply the suggestion from Dan Yasny, running a > > > standard sshd in a very small VM in every host, and forward data > > > from this VM to other guest consoles. The ssh part is in the VM, > > > then our work is just forwarding data from the VM via virto serial > > > channels, to the guest via the pty. > > > > I really dislike the idea of a service VM for something as > > fundamental as a VM > > console. The logistics of maintaining such a VM are a nightmare: > > provisioning, > > deployment, software upgrades, HA, etc. > > Why? It really sounds like an easy path to me - provisioning of a virtual > appliance is supposed to be simple, upgrades not necessary - same as with > ovirt-node, just a bit of config files preserved and the rest simply replaced, > and HA is taken care of by the platform How do you get the VM image to the hypervisor in the first place? Is this an extra step at install time that the admin must follow? You say that the VM is simple and will not need to be upgraded but I don't completely believe you. Inevitably, we will need to upgrade that VM (to fix a bug someone finds, or sync it up with the latest vdsm/engine code, or fix a security flaw). How will we conduct that upgrade? How do we handle a host going in and out of Maintenance mode?

> > On the other hand, maintaining this on multiple hypervisors means they should > all be up to date, compliant and configured. Not to mention the security > implications of maintaining an extra access point on lots of machines vs a > single virtual appliance VM. Bandwidth can be an issue, but I doubt serial > console traffic can be that heavy especially since it's there for admin access > and not routine work Don't we already want hypervisors to be up to date, compliant, and configured? Allowing serial console access will add complexity in one way or another. In my opinion it would be simpler to support a streaming service than a service VM. Are there any other uses for a service VM that could justify its complexity? > Am I missing a point here? > > > > > Maybe we can start simple and provide console access locally only. What > > sort of functionality would the vdsm api need to provide to enable only > > local access to the console? Presumably, it would set up a connection and > > provide the user with a port/pty to use to connect locally. For now it > > would be "BYOSSH - bring your own SSH" as clients would need to access the > > hosts with something like: > > > > ssh -t <host> "<connect command>" > > > > The above command could be wrapped in a vdsm-tool command. > > > > In the future, we can take a look at extending this feature via some sort of > > remote streaming API. Keep in mind that in order for this feature to be > > truly useful to ovirt-engine consumers, the console connection must survive > > a VM migration. To me, this means that vdsm will need to implement a > > generic streaming API like libvirt has. > > > > -- Adam Litke <agl(a)us.ibm.com&gt; IBM Linux Technology Center > > > > _______________________________________________ vdsm-devel mailing list > > vdsm-devel(a)lists.fedorahosted.org > > https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel > > > > -- > > > > Regards, > > Dan Yasny Red Hat Israel +972 9769 2280 >

From: "Adam Litke" <agl(a)us.ibm.com&gt; To: "Dan Yasny" <dyasny(a)redhat.com&gt; Cc: "VDSM Project Development" <vdsm-devel(a)lists.fedorahosted.org&gt;, "Zhou Zheng Sheng" <zhshzhou(a)linux.vnet.ibm.com&gt; Sent: Monday, 15 October, 2012 3:07:47 PM Subject: Re: [vdsm] [RFC]about the implement of text-based console On Mon, Oct 15, 2012 at 04:40:00AM -0400, Dan Yasny wrote: > > > ----- Original Message ----- > > From: "Adam Litke" <agl(a)us.ibm.com&gt; > > To: "Zhou Zheng Sheng" <zhshzhou(a)linux.vnet.ibm.com&gt; > > Cc: "VDSM Project Development" > > <vdsm-devel(a)lists.fedorahosted.org&gt; > > Sent: Friday, 12 October, 2012 3:10:57 PM > > Subject: Re: [vdsm] [RFC]about the implement of text-based > > console > > > > On Fri, Oct 12, 2012 at 04:55:20PM +0800, Zhou Zheng Sheng wrote: > > > > > > on 09/04/2012 22:19, Ryan Harper wrote: > > > >* Dan Kenigsberg <danken(a)redhat.com&gt; [2012-09-04 05:53]: > > > >>On Tue, Sep 04, 2012 at 03:05:37PM +0800, Xu He Jie wrote: > > > >>>On 09/03/2012 10:33 PM, Dan Kenigsberg wrote: > > > >>>>On Thu, Aug 30, 2012 at 04:26:31PM -0500, Adam Litke wrote: > > > >>>>>On Thu, Aug 30, 2012 at 11:32:02AM +0800, Xu He Jie wrote: > > > >>>>>>Hi, > > > >>>>>> > > > >>>>>> I submited a patch for text-based console > > > >>>>>>http://gerrit.ovirt.org/#/c/7165/ > > > >>>>>> > > > >>>>>>the issue I want to discussing as below: > > > >>>>>>1. fix port VS dynamic port > > > >>>>>> > > > >>>>>>Use fix port for all VM's console. connect console with > > > >>>>>>'ssh > > > >>>>>>vmUUID@ip -p port'. > > > >>>>>>Distinguishing VM by vmUUID. > > > >>>>>> > > > >>>>>> > > > >>>>>> The current implement was vdsm will allocated port for > > > >>>>>> console > > > >>>>>>dynamically and spawn sub-process when VM creating. > > > >>>>>>In sub-process the main thread responsible for accept new > > > >>>>>>connection > > > >>>>>>and dispatch output of console to each connection. > > > >>>>>>When new connection is coming, main processing create new > > > >>>>>>thread for > > > >>>>>>each new connection. Dynamic port will allocated > > > >>>>>>port for each VM and use range port. It isn't good for > > > >>>>>>firewall rules. > > > >>>>>> > > > >>>>>> > > > >>>>>> so I got a suggestion that use fix port. and connect > > > >>>>>> console with > > > >>>>>>'ssh vmuuid@hostip -p fixport'. this is simple for user. > > > >>>>>>We need one process for accept new connection from fix > > > >>>>>>port > > > >>>>>>and when > > > >>>>>>new connection is coming, spawn sub-process for each vm. > > > >>>>>>But because the console only can open by one process, > > > >>>>>>main > > > >>>>>>process > > > >>>>>>need responsible for dispatching console's output of all > > > >>>>>>vms > > > >>>>>>and all > > > >>>>>>connection. > > > >>>>>>So the code will be a little complex then dynamic port. > > > >>>>>> > > > >>>>>> So this is dynamic port VS fix port and simple code VS > > > >>>>>> complex code. > > > >>>>>>From a usability point of view, I think the fixed port > > > >>>>>>suggestion is nicer. > > > >>>>>This means that a system administrator needs only to open > > > >>>>>one > > > >>>>>port to enable > > > >>>>>remote console access. If your initial implementation > > > >>>>>limits > > > >>>>>console access to > > > >>>>>one connection per VM would that simplify the code? > > > >>>>Yes, using a fixed port for all consoles of all VMs seems > > > >>>>like > > > >>>>a cooler > > > >>>>idea. Besides the firewall issue, there's user experience: > > > >>>>instead of > > > >>>>calling getVmStats to tell the vm port, and then use ssh, > > > >>>>only > > > >>>>one ssh > > > >>>>call is needed. (Taking this one step further - it would > > > >>>>make > > > >>>>sense to > > > >>>>add another layer on top, directing console clients to the > > > >>>>specific host > > > >>>>currently running the Vm.) > > > >>>> > > > >>>>I did not take a close look at your implementation, and did > > > >>>>not > > > >>>>research > > > >>>>this myself, but have you considered using sshd for this? I > > > >>>>suppose you > > > >>>>can configure sshd to collect the list of known "users" > > > >>>>from > > > >>>>`getAllVmStats`, and force it to run a command that > > > >>>>redirects > > > >>>>VM's > > > >>>>console to the ssh client. It has a potential of being a > > > >>>>more > > > >>>>robust > > > >>>>implementation. > > > >>>I have considered using sshd and ssh tunnel. They > > > >>>can't implement fixed port and share console. > > > >>Would you elaborate on that? Usually sshd listens to a fixed > > > >>port > > > >>22, > > > >>and allows multiple users to have independet shells. What do > > > >>you > > > >>mean by > > > >>"share console"? > > > >> > > > >>>Current implement > > > >>>we can do anything that what we want. > > > >>Yes, it is completely under our control, but there are down > > > >>sides, too: > > > >>we have to maintain another process, and another entry point, > > > >>instead of > > > >>configuring a universally-used, well maintained and debugged > > > >>application. > > > >Think of the security implications of having another remote > > > >shell > > > >access point to a host. I'd much rather trust sshd if we can > > > >make > > > >it > > > >work. > > > > > > > > > > > >>Dan. > > > > > > At first glance, the standard sshd on the host is stronger and > > > more > > > robust than a custom ssh server, but the risk using the host > > > sshd > > > is > > > high. If we implement this feature via host ssd, when a hacker > > > attacks the sshd successfully, he will get access to the host > > > shell. > > > After all, the custom ssh server is not for accessing host > > > shell, > > > but just for forwarding the data from the guest console (a host > > > /dev/pts/X device). If we just use a custom ssh server, the > > > code in > > > this server only does 1. auth, 2. data forwarding, when the > > > hacker > > > attacks, he just gets access to that virtual machine. Notice > > > that > > > there is no code written about login to the host in the custom > > > ssh > > > server, and the custom ssh server can be protected under > > > selinux, > > > only allowing it to access /dev/pts/X. > > > > > > In fact using a custom VNC server in qemu is as risky as a > > > custom > > > ssh server in vdsm. If we accepts the former one, then I can > > > accepts > > > the latter one. The consideration is how robust of the custom > > > ssh > > > server, and the difficulty to maintain it. In He Jie's current > > > patch, the ssh auth and transport library is an open-source > > > third-party project, unless the project is well maintained and > > > well > > > proven, using it can be risky. > > > > > > So my opinion is using neither the host sshd, nor a custom ssh > > > server. Maybe we can apply the suggestion from Dan Yasny, > > > running a > > > standard sshd in a very small VM in every host, and forward > > > data > > > from this VM to other guest consoles. The ssh part is in the > > > VM, > > > then our work is just forwarding data from the VM via virto > > > serial > > > channels, to the guest via the pty. > > > > I really dislike the idea of a service VM for something as > > fundamental as a VM > > console. The logistics of maintaining such a VM are a nightmare: > > provisioning, > > deployment, software upgrades, HA, etc. > > Why? It really sounds like an easy path to me - provisioning of a > virtual > appliance is supposed to be simple, upgrades not necessary - same > as with > ovirt-node, just a bit of config files preserved and the rest > simply replaced, > and HA is taken care of by the platform How do you get the VM image to the hypervisor in the first place?

Is this an extra step at install time that the admin must follow?

You say that the VM is simple and will not need to be upgraded but I don't completely believe you. Inevitably, we will need to upgrade that VM (to fix a bug someone finds, or sync it up with the latest vdsm/engine code, or fix a security flaw). How will we conduct that upgrade?

How do we handle a host going in and out of Maintenance mode?

> > On the other hand, maintaining this on multiple hypervisors means > they should > all be up to date, compliant and configured. Not to mention the > security > implications of maintaining an extra access point on lots of > machines vs a > single virtual appliance VM. Bandwidth can be an issue, but I doubt > serial > console traffic can be that heavy especially since it's there for > admin access > and not routine work Don't we already want hypervisors to be up to date, compliant, and configured?

Allowing serial console access will add complexity in one way or another. In my opinion it would be simpler to support a streaming service than a service VM.

Are there any other uses for a service VM that could justify its complexity?

> Am I missing a point here? > > > > > Maybe we can start simple and provide console access locally > > only. What > > sort of functionality would the vdsm api need to provide to > > enable only > > local access to the console? Presumably, it would set up a > > connection and > > provide the user with a port/pty to use to connect locally. For > > now it > > would be "BYOSSH - bring your own SSH" as clients would need to > > access the > > hosts with something like: > > > > ssh -t <host> "<connect command>" > > > > The above command could be wrapped in a vdsm-tool command. > > > > In the future, we can take a look at extending this feature via > > some sort of > > remote streaming API. Keep in mind that in order for this > > feature to be > > truly useful to ovirt-engine consumers, the console connection > > must survive > > a VM migration. To me, this means that vdsm will need to > > implement a > > generic streaming API like libvirt has. > > > > -- Adam Litke <agl(a)us.ibm.com&gt; IBM Linux Technology Center > > > > _______________________________________________ vdsm-devel > > mailing list > > vdsm-devel(a)lists.fedorahosted.org > > https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel > > > > -- > > > > Regards, > > Dan Yasny Red Hat Israel +972 9769 2280 > -- Adam Litke <agl(a)us.ibm.com&gt; IBM Linux Technology Center

From: "Ryan Harper" <ryanh(a)us.ibm.com&gt; To: "Dan Yasny" <dyasny(a)redhat.com&gt; Cc: "Adam Litke" <agl(a)us.ibm.com&gt;, "VDSM Project Development" <vdsm-devel(a)lists.fedorahosted.org&gt; Sent: Monday, 15 October, 2012 10:55:21 PM Subject: Re: [vdsm] [RFC]about the implement of text-based console * Dan Yasny <dyasny(a)redhat.com&gt; [2012-10-15 03:41]: > > > > > >>Dan. > > > > > > At first glance, the standard sshd on the host is stronger and > > > more > > > robust than a custom ssh server, but the risk using the host > > > sshd > > > is > > > high. If we implement this feature via host ssd, when a hacker > > > attacks the sshd successfully, he will get access to the host > > > shell. > > > After all, the custom ssh server is not for accessing host > > > shell, > > > but just for forwarding the data from the guest console (a host > > > /dev/pts/X device). If we just use a custom ssh server, the > > > code in > > > this server only does 1. auth, 2. data forwarding, when the > > > hacker > > > attacks, he just gets access to that virtual machine. Notice > > > that > > > there is no code written about login to the host in the custom > > > ssh > > > server, and the custom ssh server can be protected under > > > selinux, > > > only allowing it to access /dev/pts/X. > > > > > > In fact using a custom VNC server in qemu is as risky as a > > > custom > > > ssh server in vdsm. If we accepts the former one, then I can > > > accepts > > > the latter one. The consideration is how robust of the custom > > > ssh > > > server, and the difficulty to maintain it. In He Jie's current > > > patch, the ssh auth and transport library is an open-source > > > third-party project, unless the project is well maintained and > > > well > > > proven, using it can be risky. > > > > > > So my opinion is using neither the host sshd, nor a custom ssh > > > server. Maybe we can apply the suggestion from Dan Yasny, > > > running a > > > standard sshd in a very small VM in every host, and forward > > > data > > > from this VM to other guest consoles. The ssh part is in the > > > VM, > > > then our work is just forwarding data from the VM via virto > > > serial > > > channels, to the guest via the pty. > > > > I really dislike the idea of a service VM for something as > > fundamental as a VM > > console. The logistics of maintaining such a VM are a nightmare: > > provisioning, > > deployment, software upgrades, HA, etc. > > Why? It really sounds like an easy path to me - provisioning of a > virtual appliance is supposed to be simple, upgrades not necessary > - > same as with ovirt-node, just a bit of config files preserved and > the > rest simply replaced, and HA is taken care of by the platform > > On the other hand, maintaining this on multiple hypervisors means > they > should all be up to date, compliant and configured. Not to mention > the > security implications of maintaining an extra access point on lots > of > machines vs a single virtual appliance VM. Bandwidth can be an > issue, > but I doubt serial console traffic can be that heavy especially > since it's there for admin access and not routine work So, we're replacing a single daemon with a complete operating system ?

which somehow we'll ensure the service VM is connected and running on all of the networks between the various clusters and datacenters within oVirt so that it can provide a single point of failure to the console of each VM?

> > Am I missing a point here? Keep it simple. We can re-use existing services that are already present on all of the hosts: virsh and ssh for remoting. By re-using existing services, there is no additional exposure.

-- Ryan Harper Software Engineer; Linux Technology Center IBM Corp., Austin, Tx ryanh(a)us.ibm.com

From: "Ryan Harper" <ryanh(a)us.ibm.com&gt; To: "Dan Yasny" <dyasny(a)redhat.com&gt; Cc: "Ryan Harper" <ryanh(a)us.ibm.com&gt;, "Adam Litke" <agl(a)us.ibm.com&gt;, "VDSM Project Development" <vdsm-devel(a)lists.fedorahosted.org&gt; Sent: Tuesday, 16 October, 2012 2:45:25 PM Subject: Re: [vdsm] [RFC]about the implement of text-based console * Dan Yasny <dyasny(a)redhat.com&gt; [2012-10-15 23:42]: > Hi Dan, > > > > Why? It really sounds like an easy path to me - provisioning of > > > a > > > virtual appliance is supposed to be simple, upgrades not > > > necessary > > > - > > > same as with ovirt-node, just a bit of config files preserved > > > and > > > the > > > rest simply replaced, and HA is taken care of by the platform > > > > > > On the other hand, maintaining this on multiple hypervisors > > > means > > > they > > > should all be up to date, compliant and configured. Not to > > > mention > > > the > > > security implications of maintaining an extra access point on > > > lots > > > of > > > machines vs a single virtual appliance VM. Bandwidth can be an > > > issue, > > > but I doubt serial console traffic can be that heavy especially > > > since it's there for admin access and not routine work > > > > So, we're replacing a single daemon with a complete operating > > system > > ? > > a daemon on all hosts vs a single VM. It looks to me like a single > access point for consoles can provide less of an attack surface. All of the hosts already run ssh, you're not turning that off, so the surface is the same.

> Especially when the virtual appliance comes pre-secured. I don't even know what that means. There isn't any magic just because we call it a virtual appliance.

> > > which somehow we'll ensure the service VM is connected and > > running on > > all of the networks between the various clusters and datacenters > > within > > oVirt so that it can provide a single point of failure to the > > console > > of > > each VM? > > Well, I don't see an SPOF here - a VM can be set up HA, right? > Moreover, since it doesn't really need to be powerful to push text > consoles through, you can have one per DC or even cluster, if you > have > too complex a network, a single appliance with minimal amount of > RAM > and a single cpu should not be a problem. > > Thinking about it, not every cluster even requires a serial console > appliance normally, you'd probably use it in clusters of Linux > server > VMs, but not deploy it with VDI and windows VMs I suppose All of this is still too much for just tunneling a connection to virsh. We can get remote console text from the VMs *today*. No need to make a "virtual appliance", no need to deploy it, no need to figure out how many we need. No need to figure out how to distribute the VM or dynamically build it. No need for a new features to have it implemented. This is a solved problem, we just need to connect to the existing solutions.

> > > Am I missing a point here? > > > > Keep it simple. We can re-use existing services that are already > > present on all of the hosts: virsh and ssh for remoting. By > > re-using > > existing services, there is no additional exposure. > > I have always assumed opening ssh on all the hosts is something > lots > of organizations frown upon. I mean I'm all for using sshd if we > can't > come up with something more elegant - it's a known and tested What I don't want to do is wait around for the best possible solution. Since we have these tools today, I'd like to use them (virsh + ssh). Over time, if we develope this more elegant design, we can transition to that.

> technology, but I have seen enough business demands that ssh be > closed > by default. And remote libvirt access is also something to be very > careful with If we were the last users of ssh on hosts, then I'd agree with you. But I don't see that being the case.

-- Ryan Harper Software Engineer; Linux Technology Center IBM Corp., Austin, Tx ryanh(a)us.ibm.com

On Fri, Oct 12, 2012 at 04:55:20PM +0800, Zhou Zheng Sheng wrote: > on 09/04/2012 22:19, Ryan Harper wrote: >> * Dan Kenigsberg <danken(a)redhat.com&gt; [2012-09-04 05:53]: >>> On Tue, Sep 04, 2012 at 03:05:37PM +0800, Xu He Jie wrote: >>>> On 09/03/2012 10:33 PM, Dan Kenigsberg wrote: >>>>> On Thu, Aug 30, 2012 at 04:26:31PM -0500, Adam Litke wrote: >>>>>> On Thu, Aug 30, 2012 at 11:32:02AM +0800, Xu He Jie wrote: >>>>>>> Hi, >>>>>>> >>>>>>> I submited a patch for text-based console >>>>>>> http://gerrit.ovirt.org/#/c/7165/ >>>>>>> >>>>>>> the issue I want to discussing as below: >>>>>>> 1. fix port VS dynamic port >>>>>>> >>>>>>> Use fix port for all VM's console. connect console with 'ssh >>>>>>> vmUUID@ip -p port'. >>>>>>> Distinguishing VM by vmUUID. >>>>>>> >>>>>>> >>>>>>> The current implement was vdsm will allocated port for console >>>>>>> dynamically and spawn sub-process when VM creating. >>>>>>> In sub-process the main thread responsible for accept new connection >>>>>>> and dispatch output of console to each connection. >>>>>>> When new connection is coming, main processing create new thread for >>>>>>> each new connection. Dynamic port will allocated >>>>>>> port for each VM and use range port. It isn't good for firewall rules. >>>>>>> >>>>>>> >>>>>>> so I got a suggestion that use fix port. and connect console with >>>>>>> 'ssh vmuuid@hostip -p fixport'. this is simple for user. >>>>>>> We need one process for accept new connection from fix port and when >>>>>>> new connection is coming, spawn sub-process for each vm. >>>>>>> But because the console only can open by one process, main process >>>>>>> need responsible for dispatching console's output of all vms and all >>>>>>> connection. >>>>>>> So the code will be a little complex then dynamic port. >>>>>>> >>>>>>> So this is dynamic port VS fix port and simple code VS complex code. >>>>>> >From a usability point of view, I think the fixed port suggestion is nicer. >>>>>> This means that a system administrator needs only to open one port to enable >>>>>> remote console access. If your initial implementation limits console access to >>>>>> one connection per VM would that simplify the code? >>>>> Yes, using a fixed port for all consoles of all VMs seems like a cooler >>>>> idea. Besides the firewall issue, there's user experience: instead of >>>>> calling getVmStats to tell the vm port, and then use ssh, only one ssh >>>>> call is needed. (Taking this one step further - it would make sense to >>>>> add another layer on top, directing console clients to the specific host >>>>> currently running the Vm.) >>>>> >>>>> I did not take a close look at your implementation, and did not research >>>>> this myself, but have you considered using sshd for this? I suppose you >>>>> can configure sshd to collect the list of known "users" from >>>>> `getAllVmStats`, and force it to run a command that redirects VM's >>>>> console to the ssh client. It has a potential of being a more robust >>>>> implementation. >>>> I have considered using sshd and ssh tunnel. They >>>> can't implement fixed port and share console. >>> Would you elaborate on that? Usually sshd listens to a fixed port 22, >>> and allows multiple users to have independet shells. What do you mean by >>> "share console"? >>> >>>> Current implement >>>> we can do anything that what we want. >>> Yes, it is completely under our control, but there are down sides, too: >>> we have to maintain another process, and another entry point, instead of >>> configuring a universally-used, well maintained and debugged >>> application. >> Think of the security implications of having another remote shell >> access point to a host. I'd much rather trust sshd if we can make it >> work. >> >> >>> Dan. > At first glance, the standard sshd on the host is stronger and more > robust than a custom ssh server, but the risk using the host sshd is > high. If we implement this feature via host ssd, when a hacker > attacks the sshd successfully, he will get access to the host shell. > After all, the custom ssh server is not for accessing host shell, > but just for forwarding the data from the guest console (a host > /dev/pts/X device). If we just use a custom ssh server, the code in > this server only does 1. auth, 2. data forwarding, when the hacker > attacks, he just gets access to that virtual machine. Notice that > there is no code written about login to the host in the custom ssh > server, and the custom ssh server can be protected under selinux, > only allowing it to access /dev/pts/X. > > In fact using a custom VNC server in qemu is as risky as a custom > ssh server in vdsm. If we accepts the former one, then I can accepts > the latter one. The consideration is how robust of the custom ssh > server, and the difficulty to maintain it. In He Jie's current > patch, the ssh auth and transport library is an open-source > third-party project, unless the project is well maintained and well > proven, using it can be risky. > > So my opinion is using neither the host sshd, nor a custom ssh > server. Maybe we can apply the suggestion from Dan Yasny, running a > standard sshd in a very small VM in every host, and forward data > from this VM to other guest consoles. The ssh part is in the VM, > then our work is just forwarding data from the VM via virto serial > channels, to the guest via the pty. I really dislike the idea of a service VM for something as fundamental as a VM console. The logistics of maintaining such a VM are a nightmare: provisioning, deployment, software upgrades, HA, etc. Maybe we can start simple and provide console access locally only. What sort of functionality would the vdsm api need to provide to enable only local access to the console? Presumably, it would set up a connection and provide the user with a port/pty to use to connect locally. For now it would be "BYOSSH - bring your own SSH" as clients would need to access the hosts with something like: ssh -t <host> "<connect command>" The above command could be wrapped in a vdsm-tool command. In the future, we can take a look at extending this feature via some sort of remote streaming API. Keep in mind that in order for this feature to be truly useful to ovirt-engine consumers, the console connection must survive a VM migration. To me, this means that vdsm will need to implement a generic streaming API like libvirt has.

On Tue, Oct 16, 2012 at 12:51:23AM +0800, Xu He Jie wrote: > [SNIP] > Hi, Adam, Could you explain more detail about how streaming API can > survive a VM migration? > > If we want to support migration, I think we should implement console > server out of vdsm. > Actually, It will work like proxy. So we call it as consoleProxy > now. That consoleProxy can deploy on same machine with engine, > or standalone, or virtual machine. I think its' working flow as below: > > 1. user request open console to engine. > 2. engine setTicket(uuid, ticket, hostofvm) to consoleProxy. > consoleProxy need provide api to engine. > 3. engine return ticket to user. > 4. user 'ssh UUID@consoleProxy' with ticket. > 5. consoleProxy connect 'virsh -c qemu+tls://hostofvm/system console'. > the host of running consoleProxy should have certificates of all > vdsm host. > 6. consoleProxy redirect output of 'virsh -c > qemu+tls://hostofvm/system console' with ssh protocol. > Same with currently implement. we can use system sshd or paramiko. > If we use paramiko, it almost reuse the code of consoleServer > that I have already writen. > > After vm migrated: > 1. engine tell consoleProxy that vm was migrated. > I guess engine can know vm finished migration? > And engine how to push the event of vm finished migration to > consoleProxy? Engine only have rest api didn't support event push? > Is streaming api can resolve this problem? > 2. consoleProxy kill 'virsh console'. > 3. reconnect to new host of vm with 'virsh console' again. > There will missing some character if the reconnection isn't > enough fast. > This is hardly to resolve except implement ssh in qemu. I guess > streaming api have some problem too. > 4. continue redirect 'virsh console'. > > Actually if we implement consoleProxy out of vdsm, we don't need > decide it will run on physical machine or > virtual machine now. > > A lot detail need to think. I'm not cover all problem. And I haven't > code to prove that work now. Just depend on thinking. > > Is this make sense? How is this handled with current displays like VNC and Spice?

_______________________________________________ vdsm-devel mailing list vdsm-devel(a)lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel

Ewoud Kohl van Wijngaarden píše v Po 15. 10. 2012 v 22:46 +0200: > On Tue, Oct 16, 2012 at 12:51:23AM +0800, Xu He Jie wrote: >> [SNIP] >> Hi, Adam, Could you explain more detail about how streaming API can >> survive a VM migration? >> >> If we want to support migration, I think we should implement console >> server out of vdsm. >> Actually, It will work like proxy. So we call it as consoleProxy >> now. That consoleProxy can deploy on same machine with engine, >> or standalone, or virtual machine. I think its' working flow as below: >> >> 1. user request open console to engine. >> 2. engine setTicket(uuid, ticket, hostofvm) to consoleProxy. >> consoleProxy need provide api to engine. >> 3. engine return ticket to user. >> 4. user 'ssh UUID@consoleProxy' with ticket. >> 5. consoleProxy connect 'virsh -c qemu+tls://hostofvm/system console'. >> the host of running consoleProxy should have certificates of all >> vdsm host. >> 6. consoleProxy redirect output of 'virsh -c >> qemu+tls://hostofvm/system console' with ssh protocol. >> Same with currently implement. we can use system sshd or paramiko. >> If we use paramiko, it almost reuse the code of consoleServer >> that I have already writen. >> >> After vm migrated: >> 1. engine tell consoleProxy that vm was migrated. >> I guess engine can know vm finished migration? >> And engine how to push the event of vm finished migration to >> consoleProxy? Engine only have rest api didn't support event push? >> Is streaming api can resolve this problem? >> 2. consoleProxy kill 'virsh console'. >> 3. reconnect to new host of vm with 'virsh console' again. >> There will missing some character if the reconnection isn't >> enough fast. >> This is hardly to resolve except implement ssh in qemu. I guess >> streaming api have some problem too. >> 4. continue redirect 'virsh console'. >> >> Actually if we implement consoleProxy out of vdsm, we don't need >> decide it will run on physical machine or >> virtual machine now. >> >> A lot detail need to think. I'm not cover all problem. And I haven't >> code to prove that work now. Just depend on thinking. >> >> Is this make sense? > > How is this handled with current displays like VNC and Spice? Extending spice to provide just serial console remoting actually seems the easiest way to provide remote text-only console because most of the code path is already mature (used for client to guest agent communication) and e.g. spicy to just provide a device where e.g. screen could connect or just provide the console itself. CCing spice-devel

> On 10/16/2012 12:18 AM, David Jaša wrote: >> Ewoud Kohl van Wijngaarden píše v Po 15. 10. 2012 v 22:46 +0200: >>> On Tue, Oct 16, 2012 at 12:51:23AM +0800, Xu He Jie wrote: >>>> [SNIP] >>>> Hi, Adam, Could you explain more detail about how streaming API >>>> can >>>> survive a VM migration? >>>> >>>> If we want to support migration, I think we should implement >>>> console >>>> server out of vdsm. >>>> Actually, It will work like proxy. So we call it as consoleProxy >>>> now. That consoleProxy can deploy on same machine with engine, >>>> or standalone, or virtual machine. I think its' working flow as >>>> below: >>>> >>>> 1. user request open console to engine. >>>> 2. engine setTicket(uuid, ticket, hostofvm) to consoleProxy. >>>> consoleProxy need provide api to engine. >>>> 3. engine return ticket to user. >>>> 4. user 'ssh UUID@consoleProxy' with ticket. >>>> 5. consoleProxy connect 'virsh -c qemu+tls://hostofvm/system >>>> console'. >>>> the host of running consoleProxy should have certificates of >>>> all >>>> vdsm host. >>>> 6. consoleProxy redirect output of 'virsh -c >>>> qemu+tls://hostofvm/system console' with ssh protocol. >>>> Same with currently implement. we can use system sshd or >>>> paramiko. >>>> If we use paramiko, it almost reuse the code of consoleServer >>>> that I have already writen. >>>> >>>> After vm migrated: >>>> 1. engine tell consoleProxy that vm was migrated. >>>> I guess engine can know vm finished migration? >>>> And engine how to push the event of vm finished migration to >>>> consoleProxy? Engine only have rest api didn't support event >>>> push? >>>> Is streaming api can resolve this problem? >>>> 2. consoleProxy kill 'virsh console'. >>>> 3. reconnect to new host of vm with 'virsh console' again. >>>> There will missing some character if the reconnection isn't >>>> enough fast. >>>> This is hardly to resolve except implement ssh in qemu. I >>>> guess >>>> streaming api have some problem too. >>>> 4. continue redirect 'virsh console'. >>>> >>>> Actually if we implement consoleProxy out of vdsm, we don't need >>>> decide it will run on physical machine or >>>> virtual machine now. >>>> >>>> A lot detail need to think. I'm not cover all problem. And I >>>> haven't >>>> code to prove that work now. Just depend on thinking. >>>> >>>> Is this make sense? >>> >>> How is this handled with current displays like VNC and Spice? >> >> Extending spice to provide just serial console remoting actually >> seems >> the easiest way to provide remote text-only console because most of >> the >> code path is already mature (used for client to guest agent >> communication) and e.g. spicy to just provide a device where e.g. >> screen >> could connect or just provide the console itself. >> >> CCing spice-devel > > would it allow users to script with/over it like they can with ssh? If I understand correctly the idea is to add another channel for spice that would connect to a char device in qemu that in turn connects to a serial port. The result is a spice client that can display and interact, but not a scripting extension. We could also create a unix domain socket to expose this connection on the client, and the client could then use that for scripting (but this will be instead of displaying, since you can't multiplex the console in a meaningful way - unless you run screen/tmux over it maybe): remote-viewer --spice-console-unix-domain-socket /tmp/spice.uds (This option assumes we want a single console channel - if we have multiple we will need to name them too) Anyone will be able to script it using for instance: socat UNIX-CONNECT:/tmp/spice.uds SYSTEM:"echo hello world" We could also turn it into a pty (socat can do that).

Itamar Heim píše v Čt 18. 10. 2012 v 20:32 +0200: > On 10/18/2012 12:13 PM, Alon Levy wrote: > >> On 10/16/2012 12:18 AM, David Jaša wrote: [snip] > >>> Extending spice to provide just serial console remoting > >>> actually > >>> seems > >>> the easiest way to provide remote text-only console because > >>> most of > >>> the > >>> code path is already mature (used for client to guest agent > >>> communication) and e.g. spicy to just provide a device where > >>> e.g. > >>> screen > >>> could connect or just provide the console itself. > >>> > >>> CCing spice-devel > >> > >> would it allow users to script with/over it like they can with > >> ssh? > > > > If I understand correctly the idea is to add another channel for > > spice that would connect to a char device in qemu that in turn > > connects to a serial port. The result is a spice client that can > > display and interact, but not a scripting extension. We could > > also create a unix domain socket to expose this connection on > > the client, and the client could then use that for scripting > > (but this will be instead of displaying, since you can't > > multiplex the console in a meaningful way - unless you run > > screen/tmux over it maybe): > > > > remote-viewer --spice-console-unix-domain-socket /tmp/spice.uds > > (This option assumes we want a single console channel - if we > > have multiple we will need to name them too) > > > > Anyone will be able to script it using for instance: > > socat UNIX-CONNECT:/tmp/spice.uds SYSTEM:"echo hello world" > > > > We could also turn it into a pty (socat can do that). > > i think using spice this way may be a very good solution, to proxy > a > serial console. > only caveat is it requires client to install spice, vs. just using > ssh. Jarda (To:) actually asked me if this feature (serial device pass through without any graphics) was feasible for purposes of connecting remotely to serial console. Jarda, would the solution outlined by Alon be good for you? Alon, one problem comes to my mind though: it would need either second spice server, or multi-client support (limited one would be enough to have simultaneously one graphics user and one serial device user). Do you think it is possible to implement such things without much effort?

David -- David Jaša, RHCE SPICE QE based in Brno GPG Key: 22C33E24 Fingerprint: 513A 060B D1B4 2A72 7F0D 0278 B125 CD00 22C3 3E24

On 09/04/2012 06:52 PM, Dan Kenigsberg wrote: > On Tue, Sep 04, 2012 at 03:05:37PM +0800, Xu He Jie wrote: >> On 09/03/2012 10:33 PM, Dan Kenigsberg wrote: >>> On Thu, Aug 30, 2012 at 04:26:31PM -0500, Adam Litke wrote: >>>> On Thu, Aug 30, 2012 at 11:32:02AM +0800, Xu He Jie wrote: >>>>> Hi, >>>>> >>>>> I submited a patch for text-based console >>>>> http://gerrit.ovirt.org/#/c/7165/ >>>>> >>>>> the issue I want to discussing as below: >>>>> 1. fix port VS dynamic port >>>>> >>>>> Use fix port for all VM's console. connect console with 'ssh >>>>> vmUUID@ip -p port'. >>>>> Distinguishing VM by vmUUID. >>>>> >>>>> >>>>> The current implement was vdsm will allocated port for console >>>>> dynamically and spawn sub-process when VM creating. >>>>> In sub-process the main thread responsible for accept new connection >>>>> and dispatch output of console to each connection. >>>>> When new connection is coming, main processing create new thread for >>>>> each new connection. Dynamic port will allocated >>>>> port for each VM and use range port. It isn't good for firewall >>>>> rules. >>>>> >>>>> >>>>> so I got a suggestion that use fix port. and connect console with >>>>> 'ssh vmuuid@hostip -p fixport'. this is simple for user. >>>>> We need one process for accept new connection from fix port and when >>>>> new connection is coming, spawn sub-process for each vm. >>>>> But because the console only can open by one process, main process >>>>> need responsible for dispatching console's output of all vms and all >>>>> connection. >>>>> So the code will be a little complex then dynamic port. >>>>> >>>>> So this is dynamic port VS fix port and simple code VS complex >>>>> code. >>>> >From a usability point of view, I think the fixed port suggestion >>>> is nicer. >>>> This means that a system administrator needs only to open one port >>>> to enable >>>> remote console access. If your initial implementation limits >>>> console access to >>>> one connection per VM would that simplify the code? >>> Yes, using a fixed port for all consoles of all VMs seems like a >>> cooler >>> idea. Besides the firewall issue, there's user experience: instead of >>> calling getVmStats to tell the vm port, and then use ssh, only one ssh >>> call is needed. (Taking this one step further - it would make sense to >>> add another layer on top, directing console clients to the specific >>> host >>> currently running the Vm.) >>> >>> I did not take a close look at your implementation, and did not >>> research >>> this myself, but have you considered using sshd for this? I suppose >>> you >>> can configure sshd to collect the list of known "users" from >>> `getAllVmStats`, and force it to run a command that redirects VM's >>> console to the ssh client. It has a potential of being a more robust >>> implementation. >> I have considered using sshd and ssh tunnel. They >> can't implement fixed port and share console. > Would you elaborate on that? Usually sshd listens to a fixed port 22, > and allows multiple users to have independet shells. What do you mean by > "share console"? sharable console is like qemu vnc, you can open multiple connection, but picture is same in all connection. virsh limited only one user can open console, so I think make it sharable is more powerful. Hmm... for sshd, I think I missing something. It could be implemented using sshd in the following way: Add new system user for that vm on setVmTicket. And change that user's login program to another program that can redirect console. To share console among multiple connection, It need that a process redirects the console to local unix socket, then we can copy console's output to multiple connection. This is just in my mind. I am going to give a try. Thanks for your suggestion!

> >> Current implement >> we can do anything that what we want. > Yes, it is completely under our control, but there are down sides, too: > we have to maintain another process, and another entry point, instead of > configuring a universally-used, well maintained and debugged > application. > > Dan. > _______________________________________________ vdsm-devel mailing list vdsm-devel(a)lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel

于 2012-8-31 5:26, Adam Litke 写道: > On Thu, Aug 30, 2012 at 11:32:02AM +0800, Xu He Jie wrote: >> Hi, >> >> I submited a patch for text-based console >> http://gerrit.ovirt.org/#/c/7165/ >> >> the issue I want to discussing as below: >> 1. fix port VS dynamic port >> >> Use fix port for all VM's console. connect console with 'ssh >> vmUUID@ip -p port'. >> Distinguishing VM by vmUUID. >> >> >> The current implement was vdsm will allocated port for console >> dynamically and spawn sub-process when VM creating. >> In sub-process the main thread responsible for accept new connection >> and dispatch output of console to each connection. >> When new connection is coming, main processing create new thread for >> each new connection. Dynamic port will allocated >> port for each VM and use range port. It isn't good for firewall rules. >> >> >> so I got a suggestion that use fix port. and connect console with >> 'ssh vmuuid@hostip -p fixport'. this is simple for user. >> We need one process for accept new connection from fix port and when >> new connection is coming, spawn sub-process for each vm. >> But because the console only can open by one process, main process >> need responsible for dispatching console's output of all vms and all >> connection. >> So the code will be a little complex then dynamic port. >> >> So this is dynamic port VS fix port and simple code VS complex code. > From a usability point of view, I think the fixed port suggestion is > nicer. > This means that a system administrator needs only to open one port to > enable > remote console access. If your initial implementation limits console > access to > one connection per VM would that simplify the code? Another thing we want to take care is the security. Enabling one port will make all console output accessable to the user. We should take care about this to ensure that one common user can not see the console of other vms belonging to another user.

>

I like solution 4 best.

From: "Zhou Zheng Sheng" <zhshzhou(a)linux.vnet.ibm.com&gt; To: "VDSM Project Development" <vdsm-devel(a)lists.fedorahosted.org&gt; Sent: Tuesday, November 27, 2012 4:22:20 AM Subject: Re: [vdsm] [RFC]about the implement of text-based console Hi all, For now in there is no agreement on the remote guest console solution, so I decide to do some investigation continue the discussion. Our goal VM serial console remote access in CLI mode. That means the client runs without X environment.

There are several proposals. 1. Sandboxed sshd VDSM runs a new host sshd instance in virtual machine/sandbox and redirects the virtio console to it. 2. Third-party sshd VDSM runs third-party sshd library/implementation and redirects virtio console to it. 3. Spice Extend spice to support console and implement a client to be run without GUI environment 4. oVirt shell -> Engine -> libvirt The user connects to Engine via oVirt CLI, then issues a "serial-console" command, then Engine locates the host and connect to the guest console. Currently there is a workaround, it invokes "virsh -c qemu+tls://host/qemu console vmid" from Engine side. 5. VDSM console streaming API VDSM exposes getConsoleReadStream() and getConsoleWriteStream() via XMLRPC binding. Then implement the related client in vdsClient and Engine Detailed discussion 1. Sandboxes Solution 1 and 2 allow users connect to console using their favorite ssh client. The login name is vmid, the password is set by setVmTicket() call of VDSM. The connection will be lost during migration. This is similar to VNC in oVirt. I take a look at several sandbox technologies, including libvirt-sandbox, lxc and selinux. a) libvirt-sandbox boots a VM using host kernel and initramfs, then passthru the host file system to the VM in read only mode. We can also add extra binding to the guest file system. It's very easy to use. To run shell in a VM, one can just issues virt-sandbox -c qemu:///session /bin/sh Then the VM will be ready in several seconds. However it will trigger some selinux violations. Currently there is no official support for selinux policy configuration from this project. In the project page this is put in the todo list. b) lxc utilize Linux container to run a process in sandbox. It needs to be configured properly. I find in the package lxc-templates there is an example configuration file for running sshd in lxc. c) sandbox command in the package policycoreutils-python makes use of selinux to run a process in sandbox, but there is no official or example policy files for sshd. In a word, for sandbox technologies, we have to configure the policies/file system binding/network carefully and test the compatibility with popular sshd implementations (openssh-server). When those sshd upgrade, the policy must be upgraded by us at the same time. Since the policies are not maintained by who implements sshd, this is a burden for us. Work to do Write and maintain the policies. Find ways for auth callback and redirecting data to openssh-server. pros Re-use existing pieces and technologies (host sshd, sandbox). User friendly, they can use existing ssh clients. cons Connection is lost in migration, this is not a big problem because 1) VNC connection share the same problem, 2) the user can reconnect manually. It's not easy to maintain the sandbox policies/file system binding/network for compatibility with sshd. 2. Third-party sshd implementations Almost the same as solution 1 but with better flexibility. VDSM can import a third-party sshd library and let that library deal with auth and transport. VDSM just have to implement the data forwarding. Many people consider this is insecure but I think the ticket solution for VNC is even not as secure as this. Currently most of us only trust openssh-server and think the quality of third-party sshd is low. I searched for a while and found twisted.conch from the popular twisted project. I'm not familiar with twisted.conch, but I still put it in this mail to collect opinions from potential twisted.conch experts. In a word, I prefer sandbox technologies to third-party sshd implementations unless there is a implementation as good as openssh-server. Work to do Integrate twisted.conch into VDSM pros Very flexible. If library provide auth callback to VDSM, then VDSM can just compares the login password to the VM ticket without knowing SSH detials. cons Third party implementations are not as secure and carefully maintained as sshd in the host (probably openssh-server). 3. Extend Spice to support console Is it possible to implement a spice client can be run in pure text mode without GUI environment? If we extend the protocol to support console stream but the client must be run in GUI, it will be less useful. pros No new VMs and server process, easy for maintenance. cons Must wait for Spice developers to commit the support. Need special client program in CLI, the user may prefer existing client program like ssh. It not a big problem because this feature can be put in to oVirt shell. 4. oVirt shell -> Engine -> libvirtd This is the current workaround described in http://wiki.ovirt.org/wiki/Features/Serial_Console_in_CLI#Currently_opera... The design is good but I do not like Engine talking to libvirtd directly, thus comes the VDSM console streaming API below. Work to do Provide console streaming API from Engine to be invoked in oVirt shell. Implement the "serial-console" command in oVirt shell. pros Support migration. Engine can reconnect to the guest automatically after migration while keeping the connection from oVirt shell. Fit well in the current oVirt architecture: no new server process introduced, no new VM introduced, easy to maintain and manage. cons Engine talking to libvirtd directly breaks the encapsulation of VDSM. Users only can get the console stream from Engine, they can not directly connect to the host as VNC and the above two sshd solutions do. 5. VDSM console streaming API Implement new APIs in VDSM to forward the raw data from console. It exposes getConsoleReadStream() and getConsoleWriteStream() via XMLRPC binding. Then Engine can get the console data stream via API instead of directly connecting to libvirtd. Other things will be the same as solution 4. Work to do Implement getConsoleReadStream() and getConsoleWriteStream() in VDSM. Provide console streaming API from Engine to be invoked in oVirt shell. Implement the "serial-console" command in oVirt shell. Optional: Implement a client program in vdsClient to consume the stream API. pros Same as solution 4 cons We can not allow ordinary user directly connect to VDSM and invoke the stream API, because there is no ACL in VDSM, once a client cert is setup for the ordinary user, he can call all the APIs in VDSM and get total control. So the ordinary user can only get the stream from Engine, and we leave Engine to do the ACL. I like solution 4 best. -- Thanks and best regards! Zhou Zheng Sheng / 周征晟 E-mail: zhshzhou(a)linux.vnet.ibm.com Telephone: 86-10-82454397 _______________________________________________ vdsm-devel mailing list vdsm-devel(a)lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel

One issue that was raised is console buffering. What happens if a client does not call getConsoleReadStream() fast enough? Will characters be dropped? This could create a reliability problem and would make scripting against this interface risky at best.

I don't really understand 5. What does those methods return the virtio dev path?

----- Original Message ----- > From: "Zhou Zheng Sheng" <zhshzhou(a)linux.vnet.ibm.com&gt; > To: "VDSM Project Development" <vdsm-devel(a)lists.fedorahosted.org&gt; > Sent: Tuesday, November 27, 2012 4:22:20 AM > Subject: Re: [vdsm] [RFC]about the implement of text-based console > > Hi all, > > For now in there is no agreement on the remote guest console > solution, > so I decide to do some investigation continue the discussion. > > Our goal > VM serial console remote access in CLI mode. That means the client > runs without X environment. Do you mean like running qemu with -curses?

Sorry, it's probably the fact that I don't have enough time to go into the code but I still don't get what you are trying to do. Having it in HTTP and XML-RPC is a bad idea but I imagine that the theoretical solution doesn't depend on any of them. Could you just show some pseudo code of a client using the stream?