Project for profiles and defaults for libvirt domains
by Martin Kletzander
Hi everyone!
First of all sorry for such wide distribution, but apparently that's the
best way to make sure we cooperate nicely. So please be considerate as
this is a cross-post between huge amount of mailing lists.
After some discussions with developers from different projects that work
with libvirt one cannot but notice some common patterns and workarounds.
So I set off to see how can we make all our lives better and our coding
more effective (and maybe more fun as well). If all goes well we will
create a project that will accommodate most of the defaulting, policies,
workarounds and other common algorithms around libvirt domain
definitions. And since early design gets you half way, I would like to
know your feedback on several key points as well as on the general idea.
Also correct me brutally in case I'm wrong.
In order to not get confused in the following descriptions, I will refer
to this project idea using the name `virtuned`, but there is really no
name for it yet (although an abbreviation for "Virtualization
Abstraction Definition and Hypervisor Delegation" would suit well,
IMHO).
Here are some common problems and use cases that virtuned could solve
(or help with). Don't take it as something that's impossible to solve
on your own, but rather something that could be de-duplicated from
multiple projects or "done right" instead of various hack-ish solutions.
1) Default devices/values
Libvirt itself must default to whatever values there were before any
particular element was introduced due to the fact that it strives to
keep the guest ABI stable. That means, for example, that it can't just
add -vmcoreinfo option (for KASLR support) or magically add the pvpanic
device to all QEMU machines, even though it would be useful, as that
would change the guest ABI.
For default values this is even more obvious. Let's say someone figures
out some "pretty good" default values for various HyperV enlightenment
feature tunables. Libvirt can't magically change them, but each one of
the projects building on top of it doesn't want to keep that list
updated and take care of setting them in every new XML. Some projects
don't even expose those to the end user as a knob, while others might.
One more thing could be automatically figuring out best values based on
libosinfo-provided data.
2) Policies
Lot of the time there are parts of the domain definition that need to be
added, but nobody really cares about them. Sometimes it's enough to
have few templates, another time you might want to have a policy
per-scenario and want to combine them in various ways. For example with
the data provided by point 1).
For example if you want PCI-Express, you need the q35 machine type, but
you don't really want to care about the machine type. Or you want to
use SPICE, but you don't want to care about adding QXL.
What if some of these policies could be specified once (using some DSL
for example), and used by virtuned to merge them in a unified and
predictable way?
3) Abstracting the XML
This is probably just usable for stateless apps, but it might happen
that some apps don't really want to care about the XML at all. They
just want an abstract view of the domain, possibly add/remove a device
and that's it. We could do that as well. I can't really tell how much
of a demand there is for it, though.
4) Identifying devices properly
In contrast to the previous point, stateful apps might have a problem
identifying devices after hotplug. For example, let's say you don't
care about the addresses and leave that up to libvirt. You hotplug a
device into the domain and dump the new XML of it. Depending on what
type of device it was, you might need to identify it based on different
values. It could be <target dev=''/> for disks, <mac address=''/> for
interfaces etc. For some devices it might not even be possible and you
need to remember the addresses of all the previous devices and then
parse them just to identify that one device and then throw them away.
With new enough libvirt you could use the user aliases for that, but
turns out it's not that easy to use them properly anyway. Also the
aliases won't help users identify that device inside the guest.
<rant>
We really should've gone with new attribute for the user alias instead
of using an existing one, given how many problems that is causing.
</rant>
5) Generating the right XML snippet for device hot-(un)plug
This is kind of related to some previous points.
When hot-plugging a device and creating an XML snippet for it, you want
to keep the defaults from point 1) and policies from 2) in mind. Or
something related to the already existing domain which you can describe
systematically. And adding something for identification (see previous
point).
Doing the hot-unplug is easy depending on how much information about
that device is saved by your application. The less you save about the
device (or show to the user in a GUI, if applicable) the harder it might
be to generate an XML that libvirt will accept. Again, some problems
with this should be fixed in libvirt, some of them are easy to
workaround. But having a common ground that takes care of this should
help some projects.
Hot-unplug could be implemented just based on the alias. This is
something that would fit into libvirt as well.
========================================================================
To mention some pre-existing solutions:
- I understand OpenStack has some really sensible and wisely chosen
and/or tested default values.
- I know KubeVirt has VirtualMachinePresets. That is something closely
related to points 1) and 2). Also their abstraction of the XML might
be usable for point 3).
- There was an effort on creating policy based configuration of libvirt
objects called libvirt-designer. This is closely related to points 2)
and 3). Unfortunately there was no much going on lately and part of
virt-manager repository has currently more features implemented with
the same ideas in mind, just not exported for public use.
We could utilize some of the above to various extents.
Let me know what you think and have a nice day.
Martin
6 years
Cockpit 164 released
by Martin Pitt
http://cockpit-project.org/blog/cockpit-164.html
Cockpit is the modern Linux admin interface. We release regularly. Here
are the release notes from version 164.
Storage: Usability improvements
-------------------------------
We recently ran a usability study targeting the storage area. Based on the
findings, we redesigned and adjusted the NFS, VDO, and RAID storage devices.
Previously, Cockpit used checkbox "arming" and drop-down buttons to reveal
available actions. Usability tests indicated these interface elements
were too hard and unintuitive to discover. Cockpit now expresses these
concepts in a more straightforward way.
Available RAID and VDO actions are now directly shown next to the
corresponding device.
NFS mounts are now clickable.
Clicking an NFS mount now shows a new details page with directly
available actions.
Screenshots:
http://cockpit-project.org/images/storage-raid-action-buttons.png
http://cockpit-project.org/images/storage-nfs-list.png
http://cockpit-project.org/images/storage-nfs-details.png
System: Show available package updates and missing registration
---------------------------------------------------------------
The System overview page now shows pending enhancements, bug fixes, and
security updates when they are available.
As unregistered RHEL systems cannot receive updates they now display a warning.
Screenshots:
http://cockpit-project.org/images/system-security-updates.png
http://cockpit-project.org/images/system-unregistered.png
System: Fix inconsistent tooltips
---------------------------------
Before this release, the System page had three different styles of tool tips.
Thee have been unified to use the standard Bootstrap tooltips like other
Cockpit pages. Relatedly, the performance profile used a static blue "i" for
additional information. We removed the icon and attached the tooltip directly
to the profile name.
The navigation redesign in Cockpit 156 increased the width of host navigation
items. As the navigation labels are no longer truncated, their corresponding
tooltips have been removed.
Screenshot:
http://cockpit-project.org/images/system-tuned-profile.png
Logs: Change severities
-----------------------
On the Logs page, the severity filter now displays the officially defined
syslog(3) levels (https://linux.die.net/man/3/syslog) instead of the previous
Cockpit-specific names.
Screenshot:
http://cockpit-project.org/images/logs-syslog-severity-levels.png
Thanks to Kirill Glebov for this improvement!
Machines: Add error notifications
---------------------------------
The Machines page now shows individual and dismissable error notifications for
all virtual machines.
Screenshot:
http://cockpit-project.org/images/machines-error-notifications.png
Thanks to suomiy for this improvement!
Accessibility improvements
--------------------------
This Cockpit release includes several accessibility fixes. Notably, ARIA
markers role tags, tabindex fixes, and several other improvements should help
with keyboard navigation and screen reader support.
There's still more accessibility related work to do, but these changes are
a step in the right direction.
Reloading the page in the browser now reloads Cockpit package manifests
-----------------------------------------------------------------------
Cockpit now refreshes the navigation menu on a browser reload. Installing,
removing, or updating Cockpit-related packages previously required logging
out and back in to see the changes.
Get it
------
You can get Cockpit here:
http://cockpit-project.org/running.html
Cockpit 164 is available in Fedora 27:
https://bodhi.fedoraproject.org/updates/cockpit-164-1.fc27
And Fedora 28:
https://bodhi.fedoraproject.org/updates/cockpit-164-1.fc28
Or download the tarball here:
https://github.com/cockpit-project/cockpit/releases/tag/164
Take care,
Martin Pitt
6 years, 1 month
Introducing the Cockpit Starter Kit for developing your own Cockpit
pages
by Martin Pitt
Hello Cockpit community,
in the past months we have worked on making Cockpit's test API and CI/CD system
available to third-party projects. So if you maintain or plan to create your
own Cockpit page, please have a look at Cockpit's "Starter Kit": It's an
example page and project which includes all the gory webpack/babel/eslint/
release tarballs/rpm build boilerplate, an integration test that uses
Cockpit's test VMs and test API, and whose PRs are validated on the Cockpit CI
infrastructure.
Please have a look at the recent blog post which explains this in much more
detail:
http://cockpit-project.org/blog/cockpit-starter-kit.html
Please talk to us on the mailing list, IRC (#cockpit) or GitHub issues if you
have questions!
Martin
6 years, 1 month
Questions about file watching
by Mark Reynolds
Hello,
So I am running into two issues with file watching on cockpit-151-2
[1] The first one appears to be a permissions issue. So I am trying to
watch a file that the Cockpit user does have immediate permissions to,
but the Cockpit user does have sudo rights. When I try to watch the
file, I get an error: "Not permitted to perform this action." - I'm
assuming it's permissions issue because if I watch a file in /tmp it
works fine (I also tried disabling selinux but no luck).
My code:
var logfile =
cockpit.file("/var/log/dirsrv/slapd-localhost/errors");
//var logfile = cockpit.file("/tmp/errors");
errorslog_watch = logfile.watch(function (content, tag, error) {
$('#errorslog-area').text(content);
console.log("We should have logged something: (" + content +
") err: " + error);
});
This generates in console:
We should have logged something: (null) err: Not permitted to
perform this action.
I'm not 100% sure what that errors means, but is there something what
will invoke sudo on the file access/watching?
[2] What I want to do is the equivalent of "tailing" a log file, and I
would like to use Cockpit's file watching functionality to do this.
However, when the log file changes the entire file content is returned.
Our log files can grow up to 100mb. Is there a way to only return the
last N lines of a file when it changes without loading the entire
content into memory?
Thanks,
Mark
6 years, 1 month
Cockpit 163 released
by Martin Pitt
http://cockpit-project.org/blog/cockpit-163.html
Cockpit is the modern Linux admin interface. We release regularly. Here
are the release notes from version 163.
Hide Docker storage pool reset button when it cannot work properly
------------------------------------------------------------------
Fedora Atomic used to configure a separate logical volume called
`docker-root-lv` as a Docker storage pool. Recently, Fedora Atomic switched to
using one large volume for both the root partition and Docker.
Due to this change in Fedora Atomic, Cockpit can no longer remove and free
previously added storage devices. As a result, we have removed the “Reset”
button on the storage pool page under these circumstances.
Drop "Transfer data asynchronously" VDO option on Storage page
--------------------------------------------------------------
Recent versions of VDO now automatically detect the correct setting for
asynchronous data transfer, so this option is no longer needed.
Manually setting the option may also lead to unsupported configurations in the
future.
Update jQuery to version 3.3.1
------------------------------
Cockpit updated jQuery from 2.2 to 3.3.1.
Third party projects should no longer use Cockpit’s bundled jQuery, as
announced in Cockpit 161 (http://cockpit-project.org/blog/cockpit-161.html).
All known consumers of this deprecated usage have been updated.
If you have a private cockpit extension that currently includes
"../base1/jquery.js", please verify that your project still works, remove the
reference to Cockpit’s bundled jQuery, and include your own copy of jQuery
through NPM or a CDN.
More information:
https://lists.fedorahosted.org/archives/list/cockpit-devel@lists.fedoraho...
Get it
------
You can get Cockpit here:
http://cockpit-project.org/running.html
Cockpit 163 is available in Fedora 27:
https://bodhi.fedoraproject.org/updates/cockpit-163-1.fc27
Or download the tarball here:
https://github.com/cockpit-project/cockpit/releases/tag/163
Take care,
Martin Pitt
6 years, 1 month
